[en] In this paper, we consider the periodic reference tracking problem in the framework of batch-mode reinforcement learning, which studies methods for solving optimal control problems from the sole knowledge of a set of trajectories. In particular, we extend an existing batch-mode reinforcement learning algorithm, known as Fitted Q Iteration, to the periodic reference tracking problem. The presented periodic reference tracking algorithm explicitly exploits a priori knowledge of the future values of the reference trajectory and its periodicity. We discuss the properties of our approach and illustrate it on the problem of reference tracking for a synthetic biology gene regulatory network known as the generalised repressilator. This system can produce decaying but long-lived oscillations, which makes it an interesting application for the tracking problem.
Disciplines :
Computer science
Author, co-author :
Sootla, Aivar
Strelkowa, Natajala
Ernst, Damien ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Barahona, Mauricio
Stan, Guy-Bart
Language :
English
Title :
On periodic reference tracking using batch-mode reinforcement learning with application to gene regulatory network control
Publication date :
December 2013
Event name :
52nd Annual Conference on Decision and Control (CDC 2013)
Event place :
Florence, Italy
Event date :
December 10-13, 2013
Audience :
International
Main work title :
Proceedings of the 52nd Annual Conference on Decision and Control (CDC 2013)
S. Hara, Y. Yamamoto, T. Omata, and M. Nakano, "Repetitive control system: A new type servo system for periodic exogenous signals, " IEEE Trans. Autom. Control, vol. 33, no. 7, pp. 659-668, 1988.
R. Fonteneau, S. Murphy, L. Wehenkel, and D. Ernst, "Batch mode reinforcement learning based on the synthesis of artificial trajectories, " Annals of Operations Research, vol. 208, no. 1, pp. 383-416, 2013.
L. Buşoniu, R. Babǔska, B. De Schutter, and D. Ernst, Reinforcement Learning and Dynamic Programming Using Function Approximators. CRC Pr I Llc, 2010.
R. Sutton and A. Barto, Reinforcement Learning, an Introduction. MIT Press, 1998.
M. Riedmiller, "Neural fitted Q iteration-first experiences with a data efficient neural reinforcement learning method, " in Machine Learning: ECML 2005, ser. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2005, vol. 3720, pp. 317-328.
G.-B. Stan, F. Belmudes, R. Fonteneau, F. Zeggwagh, M.-A. Lefebvre, C. Michelet, and D. Ernst, "Modelling the influence of activationinduced apoptosis of CD4+ and CD8+ T-cells on the immune system response of a HIV-infected patient, " IET Systems Biology, vol. 2, no. 2, pp. 94-102, 2008.
S. A. Murphy, "Optimal dynamic treatment regimes, " Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 65, no. 2, pp. 331-355, 2003.
D. Ernst, P. Geurts, and L. Wehenkel, "Tree-based batch mode reinforcement learning, " J Mach Learn Res, vol. 6, pp. 503-556, 2005.
M. Elowitz and S. Leibler, "A synthetic oscillatory network of transcriptional regulators, " Nature, vol. 403, no. 6767, pp. 335-338, 2000.
H. Smith, "Oscillations and multiple steady states in a cyclic gene model with repression, " J Math Biol, vol. 25, no. 15, pp. 169-190, Jul 1987.
N. Strelkowa and M. Barahona, "Switchable genetic oscillator operating in quasi-stable mode, " J R Soc Interface, vol. 7, no. 48, pp. 1071-1082, 2010.
A. Sootla, N. Strelkowa, D. Ernst, M. Barahona, and G. Stan, "Toggling the genetic switch using reinforcement learning, " March 2013, arXiv:1303.3183.
P. Geurts, D. Ernst, and L. Wehenkel, "Extremely randomized trees, " Machine Learning, vol. 63, no. 1, pp. 3-42, 2006.
D. Ormoneit and S. Sen, "Kernel-based reinforcement learning, " Machine Learning, vol. 49, no. 2-3, pp. 161-178, 2002.
L. Cai, N. Friedman, and X. S. Xie, "Stochastic protein expression in individual cells at the single molecule level, " Nature, vol. 440, pp. 358-362, 2006.
M. R. Bennett and J. Hasty, "Microfluidic devices for measuring gene network dynamics in single cells, " Nat Rev Genet, vol. 10, no. 9, pp. 628-638, September 2009.
S. Shimizu-Sato, E. Huq, J. M. Tepperman, and P. H. Quail, "A lightswitchable gene promoter system, " Nat Biotech, vol. 20, no. 10, pp. 1041-1044, 2002.
A. Levskaya, O. D. Weiner, W. A. Lim, and C. A. Voigt, "Spatiotemporal control of cell signalling using a light-switchable protein interaction, " Nature, vol. 461, pp. 997-1001, 2009.
F. Pedregosa et al., "Scikit-learn: Machine learning in python, " J Machine Learning Research, vol. 12, pp. 2825-2830, 2011.
"Joblib: running python function as pipeline jobs." [Online]. Available: http://packages.python.org/joblib.
J. D. Hunter, "Matplotlib: A 2d graphics environment, " Computing In Science & Engineering, vol. 9, no. 3, pp. 90-95, 2007.
E. Jones, T. Oliphant, P. Peterson, et al., "SciPy: Open source scientific tools for Python, " 2001-. [Online]. Available: http://www.scipy.org/.
Similar publications
Sorry the service is unavailable at the moment. Please try again later.
This website uses cookies to improve user experience. Read more
Save & Close
Accept all
Decline all
Show detailsHide details
Cookie declaration
About cookies
Strictly necessary
Performance
Strictly necessary cookies allow core website functionality such as user login and account management. The website cannot be used properly without strictly necessary cookies.
This cookie is used by Cookie-Script.com service to remember visitor cookie consent preferences. It is necessary for Cookie-Script.com cookie banner to work properly.
Performance cookies are used to see how visitors use the website, eg. analytics cookies. Those cookies cannot be used to directly identify a certain visitor.
Used to store the attribution information, the referrer initially used to visit the website
Cookies are small text files that are placed on your computer by websites that you visit. Websites use cookies to help users navigate efficiently and perform certain functions. Cookies that are required for the website to operate properly are allowed to be set without your permission. All other cookies need to be approved before they can be set in the browser.
You can change your consent to cookie usage at any time on our Privacy Policy page.