References of "Sootla, Aivar"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailAn Optimal Control Formulation of Pulse-Based Control Using Koopman Operator
Sootla, Aivar; Mauroy, Alexandre; Ernst, Damien ULiege

in Automatica (2018), 91

In many applications, and in systems/synthetic biology in particular, it is desirable to compute control policies that force the trajectory of a bistable system from one equilibrium (the initial point) to ... [more ▼]

In many applications, and in systems/synthetic biology in particular, it is desirable to compute control policies that force the trajectory of a bistable system from one equilibrium (the initial point) to another equilibrium (the target point), or in other words to solve the switching problem. It was recently shown that, for monotone bistable systems, this problem admits easyto-implement open-loop solutions in terms of temporal pulses (i.e., step functions of fixed length and fixed magnitude). In this paper, we develop this idea further and formulate a problem of convergence to an equilibrium from an arbitrary initial point. We show that this problem can be solved using a static optimization problem in the case of monotone systems. Changing the initial point to an arbitrary state allows to build closed-loop, event-based or open-loop policies for the switching/convergence problems. In our derivations we exploit the Koopman operator, which offers a linear infinite-dimensional representation of an autonomous nonlinear system. One of the main advantages of using the Koopman operator is the powerful computational tools developed for this framework. Besides the presence of numerical solutions, the switching/convergence problem can also serve as a building block for solving more complicated control problems and can potentially be applied to non-monotone systems. We illustrate this argument on the problem of synchronizing cardiac cells by defibrillation. Potentially, our approach can be extended to problems with different parametrizations of control signals since the only fundamental limitation is the finite time application of the control signal. [less ▲]

Detailed reference viewed: 117 (20 ULiège)
Full Text
Peer Reviewed
See detailPulse-Based Control Using Koopman Operator Under Parametric Uncertainty
Sootla, Aivar; Ernst, Damien ULiege

in IEEE Transactions on Automatic Control (2018), 63(3), 791-796

In applications, such as biomedicine and systems/synthetic biology, technical limitations in actuation complicate implementation of time-varying control signals. In order to alleviate some of these ... [more ▼]

In applications, such as biomedicine and systems/synthetic biology, technical limitations in actuation complicate implementation of time-varying control signals. In order to alleviate some of these limitations, it may be desirable to derive simple control policies, such as step functions with fixed magnitude and length (or temporal pulses). In this technical note, we further develop a recently proposed pulse-based solution to the convergence problem, i.e., minimizing the convergence time to the target exponentially stable equilibrium, for monotone systems. In particular, we extend this solution to monotone systems with parametric uncertainty. Our solutions also provide worst-case estimates on convergence times. Furthermore, we indicate how our tools can be used for a class of non-monotone systems, and more importantly how these tools can be extended to other control problems. We illustrate our approach on switching under parametric uncertainty and regulation around a saddle point problems in a genetic toggle switch system. [less ▲]

Detailed reference viewed: 54 (11 ULiège)
Full Text
Peer Reviewed
See detailPolicy transfer using Value Function as Prior Information
Aittahar, Samy ULiege; Sootla, Aivar; Ernst, Damien ULiege

Conference (2016, September 19)

This work proposes an approach based on reward shaping techniques in a reinforcement learning setting to approximate the opti- mal decision-making process (also called the optimal policy) in a desired ... [more ▼]

This work proposes an approach based on reward shaping techniques in a reinforcement learning setting to approximate the opti- mal decision-making process (also called the optimal policy) in a desired task with a limited amount of data. We extract prior information from an existing family of policies have been used as a heuristic to help the construction of the new one under this challenging condition. We use this approach to study the relationship between the similarity of two tasks and the minimal amount of data needed to compute a near-optimal pol- icy for the second one using the prior information of the existing policy. Preliminary results show that for the least similar existing task consid- ered compared to the desired one, only 10% of the dataset was needed to compute the corresponding near-optimal policy. [less ▲]

Detailed reference viewed: 18 (1 ULiège)
Full Text
Peer Reviewed
See detailToggling a genetic switch using reinforcement learning
Sootla, Aivar; Strelkowa, Natalja; Ernst, Damien ULiege et al

in Proceedings of the 9th French Meeting on Planning, Decision Making and Learning (2014, May)

In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the fitted Q ... [more ▼]

In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the fitted Q iteration. This algorithm infers the control law directly from the measurements of the system’s response to external control inputs without the use of a mathematical model of the system. The measurement data set can either be collected from wet-lab experiments or artificially created by computer simulations of dynamical models of the system. The algorithm is applicable to a wide range of biological systems due to its ability to deal with nonlinear and stochastic system dynamics. To illustrate the application of the algorithm to a gene regulatory network, the regulation of the toggle switch system is considered. The control objective of this problem is to drive the concentrations of two specific proteins to a target region in the state space. [less ▲]

Detailed reference viewed: 47 (4 ULiège)
Full Text
Peer Reviewed
See detailOn periodic reference tracking using batch-mode reinforcement learning with application to gene regulatory network control
Sootla, Aivar; Strelkowa, Natajala; Ernst, Damien ULiege et al

in Proceedings of the 52nd Annual Conference on Decision and Control (CDC 2013) (2013, December)

In this paper, we consider the periodic reference tracking problem in the framework of batch-mode reinforcement learning, which studies methods for solving optimal control problems from the sole knowledge ... [more ▼]

In this paper, we consider the periodic reference tracking problem in the framework of batch-mode reinforcement learning, which studies methods for solving optimal control problems from the sole knowledge of a set of trajectories. In particular, we extend an existing batch-mode reinforcement learning algorithm, known as Fitted Q Iteration, to the periodic reference tracking problem. The presented periodic reference tracking algorithm explicitly exploits a priori knowledge of the future values of the reference trajectory and its periodicity. We discuss the properties of our approach and illustrate it on the problem of reference tracking for a synthetic biology gene regulatory network known as the generalised repressilator. This system can produce decaying but long-lived oscillations, which makes it an interesting application for the tracking problem. [less ▲]

Detailed reference viewed: 19 (1 ULiège)