Solving Optimal Control Problems for Monotone Systems Using the Koopman Operator

[en] Koopman operator theory offers numerous techniques for analysis and control of complex systems. In particular, in this chapter we will argue that for the problem of convergence to an equilibrium, the Koopman operator can be used to take advantage of the geometric properties of controlled systems, thus making the optimal solutions more transparent, and easier to analyse and implement. The motivation for the study of the convergence problem comes from biological applications, where easy-to-implement and easy-to-analyse solutions are of particular value. At the moment, theoretical results have been developed for a class of nonlinear systems called monotone systems. However, the core ideas presented here can be applied heuristically to non-monotone systems. Furthermore, the convergence problem can serve as a building block for solving other control problems such as switching between stable equilibria, or inducing oscillations. These applications are illustrated on biologically inspired numerical examples.

Disciplines :

Mathematics
Electrical & electronics engineering

Author, co-author :

Sootla, Aivar

Stan, Guy-Bart

Ernst, Damien ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids

Language :

English

Title :

Solving Optimal Control Problems for Monotone Systems Using the Koopman Operator

Publication date :

February 2020

Main work title :

The Koopman Operator in Systems and Control

Author, co-author :

Mauroy, Alexandre

Mezic, Igor

Susuki, Yoshihiko

Publisher :

Springer

ISBN/EAN :

978-3-030-35713-9

Collection name :

Lecture Notes in Control and Information Sciences

Pages :

335-357

Peer reviewed :

Peer reviewed

Available on ORBi :

since 25 February 2020

Statistics

Number of views

206 (16 by ULiège)

Number of downloads

232 (15 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

Adams, B.M., Banks, H.T., Kwon, H.-D., Tran, H.T.: Dynamic multidrug therapies for HIV: optimal and STI control approaches. Math. Biosci. Eng. 1(2), 223–241 (2004)
Angeli, D., Sontag, E.D.: Monotone control systems. IEEE Trans. Autom. Control 48(10), 1684–1698 (2003)
Berman, A., Plemmons, R.J.: Nonnegative Matrices in the Mathematical Sciences, vol. 9. SIAM, Philadelphia (1994)
Brophy, J.A.N., Voigt, C.A.: Principles of genetic circuit design. Nat. Methods 11(5), 508–520 (2014)
Ernst, D., Stan, G.-B., Goncalves, J., Wehenkel, L.: Clinical data based optimal STI strategies for HIV: a reinforcement learning approach. In: IEEE Conference Decision Control, pp. 667– 672. IEEE, San Diego (2006)
Ernst, D., Geurts, P., Wehenkel, L.: Tree-based batch mode reinforcement learning. J. Mach. Learn. Res. 6, 503–556 (2005)
Forni, F., Sepulchre, R.: Differentially positive systems. IEEE Trans. Autom. Control 61(2), 346–359 (2016)
Freemont, P.S., Kitney, R.I.: Synthetic Biology-A Primer (revised Edition). World Scientific, Singapore (2015)
Hirsch, M.W., Smith, H., et al.: Monotone dynamical systems. In: Handbook of Differential Equations: Ordinary Differential Equations, vol. 2, pp. 239–357. Elsevier, Amsterdam (2005)
Hsiao, V., Swaminathan, A., Murray, R.M.: Control theory for synthetic biology: recent advances in system characterization, control design, and controller implementation for synthetic biology. IEEE Control Syst. 38(3), 32–62 (2018)
Kahl, L.J., Endy, D.: A survey of enabling technologies in synthetic biology. J. Biol. Eng. 7(1):13 (2013)
Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for control (2017). arXiv:1707.01146
Kim, E.S., Arcak, M., Seshia, S.A.: Directed specifications and assumption mining for monotone dynamical systems. In: Proceedings of the Conference on Hybrid Systems: Computation Control, pp. 21–30. ACM, New York (2016)
Korda, M., Mezić, I.: Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control. Automatica 93, 149–160 (2018)
Lan, Y., Mezić, I.: Linearization in the large of nonlinear systems and Koopman operator spectrum. Phys. D 242, 42–53 (2013)
Lugagne, J.-B., Carrillo, S.S., Kirch, M., Köhler, A., Batt, G., Hersen, P.: Balancing a genetic toggle switch by real-time feedback control and periodic forcing. Nat. Commun. 8(1), 1671 (2017)
Mauroy, A., Forni, F., Sepulchre, R.: An operator-theoretic approach to differential positivity. In: IEEE Conference on Decision Control, pp. 7028–7033 (2015)
Mauroy, A.: Converging to and escaping from the global equilibrium: Isostables and optimal control. In: IEEE Conference on Decision Control, pp. 5888–5893 (2014)
Mauroy, A., Mezić, I.: Global stability analysis using the eigenfunctions of the Koopman operator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
Mauroy, A., Mezić, I., Moehlis, J.: Isostables, isochrons, and Koopman spectrum for the actionangle representation of stable fixed point dynamics. Phys. D 261, 19–30 (2013)
Menolascina, F., Di Bernardo, M., Di Bernardo, D.: Analysis, design and implementation of a novel scheme for in-vivo control of synthetic gene regulatory networks. Autom. Spec. Issue Syst. Biol. 47(6), 1265–1270 (2011)
Milias-Argeitis, A., Summers, S., Stewart-Ornstein, J., Zuleta, I., Pincus, D., El-Samad, H., Khammash, M., Lygeros, J.: In silico feedback for in vivo regulation of a gene expression circuit. Nat. Biotechnol. 29(12), 1114–1116 (2011)
Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of PDEs (2017). arXiv:1710.06759
Peitz, S.: Controlling nonlinear PDEs using low-dimensional bilinear approximations obtained from data (2018). arXiv:1801.06419
Purnick, P.E.M., Weiss, R.: The second wave of synthetic biology: from modules to systems. Nat. Rev. Mol. Cell Biol. 10(6), 410–422 (2009)
Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5–28 (2010)
Sontag, E.D.: Mathematical Control Theory: Deterministic Finite Dimensional Systems, vol. 6. Springer, Berlin (2013)
Sontag, E.D.: Monotone and near-monotone biochemical networks. Syst. Synth. Biol. 1(2), 59–87 (2007)
Sootla, A., Mauroy, A., Gonçalves, J.: Shaping pulses to control monotone bistable systems using Koopman operator. In: Proceedings of the Symposium on Nonlinear Control Systems, pp. 710–715 (2016)
Sootla, A., Mauroy, A.: Properties of isostables and basins of attraction of monotone systems. In: Proceedings of the American Control Conference, pp. 7365–7370 (2016)
Sootla, A., Strelkowa, N., Ernst, D., Barahona, M., Stan, G.-B.: On reference tracking using reinforcement learning with application to gene regulatory networks. In: Proceedings of the Conference on Decision Control, pp. 4086–4091, Florence (2013)
Sootla, A., Strelkowa, N., Ernst, D., Barahona, M., Stan, G.-B.: Toggling the genetic switch using reinforcement learning. In: Proceedings of the French Meeting on Planning, Decision Making and Learning, Liège (2014)
Sootla, A., Ernst, D.: Pulse-based control using Koopman operator under parametric uncertainty. IEEE Trans. Autom. Control 63(3), 791–796 (2018)
Sootla, A., Mauroy, A.: Geometric properties and computation of isostables and basins of attraction of monotone systems. IEEE Trans. Autom. Control 62(12), 6183–6194 (2017)
Sootla, A., Oyarzún, D., Angeli, D., Stan, G.-B.: Shaping pulses to control bistable systems analysis, computation and counterexamples. Automatica 63, 254–264 (2016)
Sootla, A., Mauroy, A., Ernst, D.: An optimal control formulation of pulse-based control using Koopman operator. Automatica 91, 217–224 (2018)
Strelkowa, N., Barahona, M.: Switchable genetic oscillator operating in quasi-stable mode. J. R. Soc. Interface 7(48), 1071–1082 (2010)
Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On dynamic mode decomposition: theory and applications. J. Comput. Dyn. 1(2), 391–421 (2014)
Uhlendorf, J., Miermont, A., Delaveau, T., Charvin, G., Fages, F., Bottani, S., Batt, G., Hersen, P.: Long-term model predictive control of gene expression at the population and single-cell levels. Proc. Nat. Acad. Sci. 109(35), 14271–14276 (2012)
Wilson, D., Moehlis, J.: An energy-optimal methodology for synchronization of excitable media. SIAM J. Appl. Dyn. Syst. 13(2), 944–957 (2014)