Recent Advances in Reinforcement Learning

1. Front Matter

   PDF

2. Invited Talk Abstracts

   1. Invited Talk: UCRL and Autonomous Exploration

      • Peter Auer

      Pages 1-1

   2. Invited Talk: Increasing Representational Power and Scaling Inference in Reinforcement Learning

      • Kristian Kersting

      Pages 2-2

   3. Invited Talk: PRISM – Practical RL: Representation, Interaction, Synthesis, and Mortality

      • Peter Stone

      Pages 3-3

   4. Invited Talk: Towards Robust Reinforcement Learning Algorithms

      • Csaba Szepesvári

      Pages 4-4

3. Online Reinforcement Learning

   1. Automatic Discovery of Ranking Formulas for Playing with Multi-armed Bandits

      • Francis Maes, Louis Wehenkel, Damien Ernst

      Pages 5-17

   2. Goal-Directed Online Learning of Predictive Models

      • Sylvie C. W. Ong, Yuri Grinberg, Joelle Pineau

      Pages 18-29

   3. Gradient Based Algorithms with Loss Functions and Kernels for Improved On-Policy Control

      • Matthew Robards, Peter Sunehag

      Pages 30-41

4. Learning and Exploring MDPs

   1. Active Learning of MDP Models

      • Mauricio Araya-López, Olivier Buffet, Vincent Thomas, François Charpillet

      Pages 42-53

   2. Handling Ambiguous Effects in Action Learning

      • Boris Lesner, Bruno Zanuttini

      Pages 54-65

   3. Feature Reinforcement Learning in Practice

      • Phuong Nguyen, Peter Sunehag, Marcus Hutter

      Pages 66-77

5. Function Approximation Methods for Reinforcement Learning

   1. Reinforcement Learning with a Bilinear Q Function

      • Charles Elkan

      Pages 78-88

   2. ℓ1-Penalized Projected Bellman Residual

      • Matthieu Geist, Bruno Scherrer

      Pages 89-101

   3. Regularized Least Squares Temporal Difference Learning with Nested ℓ2 and ℓ1 Penalization

      • Matthew W. Hoffman, Alessandro Lazaric, Mohammad Ghavamzadeh, Rémi Munos

      Pages 102-114

   4. Recursive Least-Squares Learning with Eligibility Traces

      • Bruno Scherrer, Matthieu Geist

      Pages 115-127

   5. Value Function Approximation through Sparse Bayesian Modeling

      • Nikolaos Tziortziotis, Konstantinos Blekas

      Pages 128-139

6. Macro-actions in Reinforcement Learning

   1. Automatic Construction of Temporally Extended Actions for MDPs Using Bisimulation Metrics

      • Pablo Samuel Castro, Doina Precup

      Pages 140-152

   2. Unified Inter and Intra Options Learning Using Policy Gradient Methods

      • Kfir Y. Levy, Nahum Shimkin

      Pages 153-164

   3. Options with Exceptions

      • Munu Sairamesh, Balaraman Ravindran

      Pages 165-176

7. Policy Search and Bounds

   1. Robust Bayesian Reinforcement Learning through Tight Lower Bounds

      • Christos Dimitrakakis

      Pages 177-188

This book constitutes revised and selected papers of the 9th European Workshop on Reinforcement Learning, EWRL 2011, which took place in Athens, Greece in September 2011. The papers presented were carefully reviewed and selected from 40 submissions. The papers are organized in topical sections online reinforcement learning, learning and exploring MDPs, function approximation methods for reinforcement learning, macro-actions in reinforcement learning, policy search and bounds, multi-task and transfer reinforcement learning, multi-agent reinforcement learning, apprenticeship and inverse reinforcement learning and real-world reinforcement learning.

• bayesian inference
• multitask learning
• predictive state representation
• real-time control
• recommender system
• algorithm analysis and problem complexity

• NICTA and the Australian National University, Canberra, Australia

  Scott Sanner

• Research School of Computer Science, Australian National University, Canberra, Australia

  Marcus Hutter

Policies and ethics