Paper published in a book (Scientific congresses and symposiums)
A cautious approach to generalization in reinforcement learning
Fonteneau, Raphaël; Murphy, Susan; Wehenkel, Louis et al.
2010In Proceedings of the 2nd International Conference on Agents and Artificial Intelligence
Peer reviewed
 

Files


Full Text
Fonteneau2010ICAART.pdf
Publisher postprint (203.77 kB)
Download
Annexes
slides-22January2010@ICAART.pdf
Publisher postprint (766.09 kB)
Download
BSPACertificateICAART2010.jpg
Publisher postprint (621.05 kB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
Reinforcement Learning; Prior Knowledge; Cautious Generalization
Abstract :
[en] In the context of a deterministic Lipschitz continuous environment over continuous state spaces, finite action spaces, and a finite optimization horizon, we propose an algorithm of polynomial complexity which exploits weak prior knowledge about its environment for computing from a given sample of trajectories and for a given initial state a sequence of actions. The proposed Viterbi-like algorithm maximizes a recently proposed lower bound on the return depending on the initial state, and uses to this end prior knowledge about the environment provided in the form of upper bounds on its Lipschitz constants. It thereby avoids, in way depending on the initial state and on the prior knowledge, those regions of the state space where the sample is too sparse to make safe generalizations. Our experiments show that it can lead to more cautious policies than algorithms combining dynamic programming with function approximators. We give also a condition on the sample sparsity ensuring that, for a given initial state, the proposed algorithm produces an optimal sequence of actions in open-loop.
Disciplines :
Computer science
Author, co-author :
Fonteneau, Raphaël ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Murphy, Susan
Wehenkel, Louis  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Ernst, Damien  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Language :
English
Title :
A cautious approach to generalization in reinforcement learning
Publication date :
January 2010
Event name :
2nd International Conference on Agents and Artificial Intelligence
Event organizer :
Institute for Systems and Technologies of Information, Control and Communication
Event place :
Valencia, Spain
Event date :
from 22-01-2010 to 24-01-2010
Audience :
International
Main work title :
Proceedings of the 2nd International Conference on Agents and Artificial Intelligence
ISBN/EAN :
092312 1713
Pages :
10
Peer reviewed :
Peer reviewed
Funders :
F.R.S.-FNRS - Fonds de la Recherche Scientifique [BE]
FRIA - Fonds pour la Formation à la Recherche dans l'Industrie et dans l'Agriculture [BE]
Commentary :
Best Student Paper Award
Available on ORBi :
since 15 February 2010

Statistics


Number of views
263 (26 by ULiège)
Number of downloads
941 (14 by ULiège)

Scopus citations®
 
3
Scopus citations®
without self-citations
0

Bibliography


Similar publications



Contact ORBi