Reference : Approximate dynamic programming with a fuzzy parameterization
Scientific journals : Article
Engineering, computing & technology : Computer science
Approximate dynamic programming with a fuzzy parameterization
Busoniu, Lucian [ > > ]
Ernst, Damien mailto [Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation >]
De Schutter, Bart [ > > ]
Robert, Babuska [ > > ]
Pergamon Press - An Imprint of Elsevier Science
Yes (verified by ORBi)
United Kingdom
[en] approximate dynamic programming ; fuzzy approximation ; value iteration ; convergence analysis
[en] Dynamic programming (DP) is a powerful paradigm for general, nonlinear optimal control. Computing exact DP solutions is in general only possible when the process states and the control actions take values in a small discrete set. In practice, it is necessary to approximate the solutions. Therefore, we propose an algorithm for approximate DP that relies on a fuzzy partition of the state space, and on a discretization of the action space. This fuzzy Q-iteration algorithm works for deterministic processes, under the discounted return criterion. We prove that fuzzy Q-iteration asymptotically converges to a solution that lies within a bound of the optimal solution. A bound on the suboptimality of the solution obtained in a finite number of iterations is also derived. Under continuity assumptions on the dynamics and on the reward function, we show that fuzzy Q-iteration is consistent, i.e., that it asymptotically obtains the optimal solution as the approximation accuracy increases. These properties hold both when the parameters of the approximator are updated in a synchronous fashion, and when they are updated asynchronously. The asynchronous algorithm is proven to converge at least as fast as the synchronous one. The performance of fuzzy Q-iteration is illustrated in a two-link manipulator control problem.
Fonds de la Recherche Scientifique (Communauté française de Belgique) - F.R.S.-FNRS
Researchers ; Professionals ; Students

File(s) associated to this reference

Fulltext file(s):

Open access
aut10.pdfAuthor postprint1.48 MBView/Open

Bookmark and Share SFX Query

All documents in ORBi are protected by a user license.