Paper published in a book (Scientific congresses and symposiums)
Model-free Monte Carlo-like policy evaluation
Fonteneau, Raphaël; Murphy, Susan; Wehenkel, Louis et al.
2010In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010)
Peer reviewed
 

Files


Full Text
Fonteneau2010AISTATS.pdf
Publisher postprint (1.04 MB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
reinforcement learning; model free policy evaluation; Monte Carlo
Abstract :
[en] We propose an algorithm for estimating the finite-horizon expected return of a closed loop control policy from an a priori given (off-policy) sample of one-step transitions. It averages cumulated rewards along a set of “broken trajectories” made of one-step transitions selected from the sample on the basis of the control policy. Under some Lipschitz continuity assumptions on the system dynamics, reward function and control policy, we provide bounds on the bias and variance of the estimator that depend only on the Lipschitz constants, on the number of broken trajectories used in the estimator, and on the sparsity of the sample of one-step transitions.
Disciplines :
Computer science
Author, co-author :
Fonteneau, Raphaël ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Murphy, Susan
Wehenkel, Louis  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Ernst, Damien  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Language :
English
Title :
Model-free Monte Carlo-like policy evaluation
Publication date :
May 2010
Event name :
Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010)
Event place :
Chia Laguna, Sardinia, Italy
Event date :
May 13-15 2010
Audience :
International
Main work title :
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010)
Collection name :
JMLR W&CP Volume 9
Pages :
217-224
Peer reviewed :
Peer reviewed
Funders :
F.R.S.-FNRS - Fonds de la Recherche Scientifique [BE]
FRIA - Fonds pour la Formation à la Recherche dans l'Industrie et dans l'Agriculture [BE]
Commentary :
This paper was also presented at the "Conférence Francophone sur l'Apprentissage Automatique (CAp) 2010" where it won the "Best student paper award".
Available on ORBi :
since 22 April 2010

Statistics


Number of views
127 (17 by ULiège)
Number of downloads
125 (2 by ULiège)

Scopus citations®
 
8
Scopus citations®
without self-citations
5

Bibliography


Similar publications



Contact ORBi