References of "Ernst, Damien"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailEmpirical Analysis of Policy Gradient Algorithms where Starting States are Sampled accordingly to Most Frequently Visited States
Aittahar, Samy ULiege; Fonteneau, Raphaël ULiege; Ernst, Damien ULiege

in IFAC-PapersOnLine (in press)

In this paper, we propose an extension to the policy gradient algorithms by allowing starting states to be sampled from a probability distribution that may differ from the one used to specify the ... [more ▼]

In this paper, we propose an extension to the policy gradient algorithms by allowing starting states to be sampled from a probability distribution that may differ from the one used to specify the reinforcement learning task. In particular, we suggest that, between policy updates, starting states should be sampled from a probability density function which approximates the state visitation frequency of the current policy. Results generated from various environments clearly demonstrate a performance improvement in terms of mean cumulative rewards and substantial update stability compared to vanilla policy gradient algorithms where the starting state distributions are either as specified by the environment or uniform distributions over the state space. A sensitivity analysis over a subset of the hyper-parameters of our algorithm also suggests that they should be adapted after each policy update to maximise the improvements of the policies. [less ▲]

Detailed reference viewed: 147 (27 ULiège)
Full Text
See detailLes e-fuels, des carburants dans le vent
Theunis, Laetitia; Ernst, Damien ULiege

Article for general public (2020)

Detailed reference viewed: 32 (1 ULiège)
Full Text
Peer Reviewed
See detailThe impact of different COVID-19 containment measures on electricity consumption in Europe
Bahmanyar, Alizera; Abouzar, Estebsari; Ernst, Damien ULiege

in Energy Research and Social Science (2020), 68

As of March 13, 2020, the director general of the World Health Organization (WHO) considered Europe as the centre of the global COVID-19 outbreak. All countries within Europe had a confirmed case of COVID ... [more ▼]

As of March 13, 2020, the director general of the World Health Organization (WHO) considered Europe as the centre of the global COVID-19 outbreak. All countries within Europe had a confirmed case of COVID-19 by March 17. In response to the pandemic, different European countries took different approaches. This paper compares the impact of different containment measures taken by European countries in response to COVID-19 on their electricity consumption profiles. The comparisons are made for Spain, Italy, Belgium and the UK as countries with severe restrictions, and for the Netherlands and Sweden as countries with less restrictive measures. The results show that the consumption profiles reflect the difference in peoples’ activities in different countries using various measures. [less ▲]

Detailed reference viewed: 288 (11 ULiège)
Full Text
Peer Reviewed
See detailAllocation of locally generated electricity in renewable energy communities
Manuel de Villena Millan, Miguel ULiege; Mathieu, Sébastien ULiege; Vermeulen, Eric et al

E-print/Working paper (2020)

This paper introduces a methodology to perform an ex-post allocation of locally generated electricity among the members of a renewable energy community. Such an ex-post allocation takes place in a ... [more ▼]

This paper introduces a methodology to perform an ex-post allocation of locally generated electricity among the members of a renewable energy community. Such an ex-post allocation takes place in a settlement phase where the financial exchanges of the community are based on the production and consumption profiles of each member. The proposed methodology consists of an optimisation framework which (i) minimises the sum of individual electricity costs of the community members, and (ii) can enforce minimum self-sufficiency rates –proportion of electricity consumption covered by local production– on each member, enhancing the economic gains of some of them. The latter capability aims to ensure that members receive enough incentives to participate in the renewable energy community. This framework is designed so as to provide a practical approach that is ready to use by community managers, which is compliant with current legislation on renewable energy communities. It computes a set of optimal repartition keys, which represent the percentage of total local production given to each member – one key per metering period per member. These keys are computed based on an initial set of keys provided in the simulation, which are typically contractual i.e., agreed upon between the member and the manager the renewable energy community. This methodology is tested in a broad range of scenarios, illustrating its ability to optimise the operational costs of a renewable energy community. [less ▲]

Detailed reference viewed: 225 (8 ULiège)
Full Text
See detailEx-post allocation of electricity and real-time control strategy for renewable energy communities
Aittahar, Samy ULiege; Manuel de Villena Millan, Miguel ULiege; Ernst, Damien ULiege et al

Speech/Talk (2020)

A presentation that describes recent algorithmic developments for operating a renewable energy community to minimize energy costs.

Detailed reference viewed: 48 (5 ULiège)
Full Text
Peer Reviewed
See detailShort-term active distribution network operation under uncertainty
Mathieu, Sébastien ULiege; Ernst, Damien ULiege; Gemine, Quentin

in Proceeedings of the16th International Conference on Probabilistic Methods Applied to Power Systems (PMAPS 2020) (2020, August)

Electrical distribution systems need to integrate more and more renewable energy generation in their network. Since networks cannot be quickly upgraded at a low cost, new generators are connected to the ... [more ▼]

Electrical distribution systems need to integrate more and more renewable energy generation in their network. Since networks cannot be quickly upgraded at a low cost, new generators are connected to the network under non-firm access contracts. These contracts allow distribution system operators to specify dynamic access limits according to a given regulatory policy, e.g. “last-in, first-out” or a similar policy. Due to operational delays, access limits must be communicated before realtime, e.g. ten minutes ahead. This paper presents an operational method to compute these dynamic access limits using correlated probabilistic forecasts of power consumption and production processes. The method is illustrated on a test-case based on real data where no additional production would be allowed under firm access. Results show that the method allows to safely integrate additional production capacity while limiting congestion events, provided that efficient probabilistic forecasts able to anticipate sudden and important changes are available. [less ▲]

Detailed reference viewed: 136 (9 ULiège)
Full Text
See detailAn Artificial Intelligence Solution for Electricity Procurement in Forward Markets
Théate, Thibaut ULiege; Mathieu, Sébastien ULiege; Ernst, Damien ULiege

E-print/Working paper (2020)

Retailers and major consumers of electricity generally purchase an important percentage of their estimated electricity needs years ahead in the forward market. This long-term electricity procurement task ... [more ▼]

Retailers and major consumers of electricity generally purchase an important percentage of their estimated electricity needs years ahead in the forward market. This long-term electricity procurement task consists of determining when to buy electricity so that the resulting energy cost is minimised, and the forecast consumption is covered. In this scientific article, the focus is set on a yearly base load product from the Belgian forward market, named calendar (CAL), which is tradable up to three years ahead of the delivery period. This research paper introduces a novel algorithm providing recommendations to either buy electricity now or wait for a future opportunity based on the history of CAL prices. This algorithm relies on deep learning forecasting techniques and on an indicator quantifying the deviation from a perfectly uniform reference procurement policy. On average, the proposed approach surpasses the benchmark procurement policies considered and achieves a reduction in costs of 1.65% with respect to the perfectly uniform reference procurement policy achieving the mean electricity price. Moreover, in addition to automating the complex electricity procurement task, this algorithm demonstrates more consistent results throughout the years compared to the benchmark policies. Eventually, the generality of the solution presented makes it well suited for solving other commodity procurement problems. [less ▲]

Detailed reference viewed: 103 (9 ULiège)
Full Text
See detailModelling and Assessing the Impact of the DSO’s Remuneration Strategy on its Interaction with Electricity Users
Manuel de Villena Millan, Miguel ULiege; Gautier, Axel ULiege; Ernst, Damien ULiege et al

E-print/Working paper (2020)

This paper presents a simulation-based methodology for assessing the impact of employing different distribution system operator’s remuneration strategies on the economic sustainability of electrical ... [more ▼]

This paper presents a simulation-based methodology for assessing the impact of employing different distribution system operator’s remuneration strategies on the economic sustainability of electrical distribution systems. The proposed methodology accounts for the uncertainties posed by the integration of distributed electricity generation resources, and the roll out of smart-meters. The different remuneration strategies analysed in this paper include notably new distribution tariffs based on individual peak power consumption and time-dependent rates that are contingent on the time of energy consumption, both requiring smart-meters to work. The distributed electricity generation resources are modelled through an optimisation framework and an investment decision process that gradually deploys household photovoltaic installations depending on their profitability and the electricity charges, including the distribution rates. The impact of the distribution system operator’s remuneration strategy is measured by an accurate modelling of the remuneration mechanism of this entity, which can adapt to various distribution tariff designs. We analyse this impact over a discrete time horizon. Our methodology is illustrated with several examples of distribution tariffs including old –based on energy consumption or on per-connection fees– as well as new –based on power consumption or time-of use fees– designs. Finally, we provide a comprehensive sensitivity analysis of the proposed simulation environment to the main parameters of the methodology. [less ▲]

Detailed reference viewed: 193 (58 ULiège)
Full Text
See detailThe Role of Hydrogen in the Dutch Electricity System
Berger, Mathias ULiege; Radu, David-Constantin ULiege; Ryszka, Karolina et al

Report (2020)

This technical report investigates the role power-to-gas, hydrogen and battery storage technologies may play in the Dutch electricity system using a recently published optimization-based energy system ... [more ▼]

This technical report investigates the role power-to-gas, hydrogen and battery storage technologies may play in the Dutch electricity system using a recently published optimization-based energy system model. [less ▲]

Detailed reference viewed: 75 (6 ULiège)
Full Text
Peer Reviewed
See detailA bio-inspired bistable recurrent cell allows for long-lasting memory
Vecoven, Nicolas ULiege; Ernst, Damien ULiege; Drion, Guillaume ULiege

E-print/Working paper (2020)

Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as ... [more ▼]

Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network level, and long term memory is shaped by network-wide recurrent connection weights. Biological neurons on the other hand are capable of holding information at the cellular level for an arbitrary long amount of time through a process called bistability. Through bistability, cells can stabilize to different stable states depending on their own past state and inputs, which permits the durable storing of past information in neuron state. In this work, we take inspiration from biological neuron bistability to embed RNNs with long-lasting memory at the cellular level. This leads to the introduction of a new bistable biologically-inspired recurrent cell that is shown to strongly improves RNN performance on time-series which require very long memory, despite using only cellular connections (all recurrent connections are from neurons to themselves, i.e. a neuron state is not influenced by the state of other neurons). Furthermore, equipping this cell with recurrent neuromodulation permits to link them to standard GRU cells, taking a step towards the biological plausibility of GRU. [less ▲]

Detailed reference viewed: 50 (4 ULiège)
Full Text
Peer Reviewed
See detailLearning optimal environments using projected stochastic gradient ascent
Bolland, Adrien ULiege; Boukas, Ioannis ULiege; Cornet, François ULiege et al

E-print/Working paper (2020)

In this work, we generalize the direct policy search algorithms to an algorithm we call Direct Environment Search with (projected stochastic) Gradient Ascent (DESGA). The latter can be used to jointly ... [more ▼]

In this work, we generalize the direct policy search algorithms to an algorithm we call Direct Environment Search with (projected stochastic) Gradient Ascent (DESGA). The latter can be used to jointly learn a reinforcement learning (RL) environment and a policy with maximal expected return over a joint hypothesis space of environments and policies. We illustrate the performance of DESGA on two benchmarks. First, we consider a parametrized space of Mass-Spring Damper (MSD) environments. Then, we use our algorithm for optimizing the size of the components and the operation of a small-scale and autonomous energy system, i.e. a solar off-grid microgrid, composed of photovoltaic panels, batteries, etc. The results highlight the excellent performances of the DESGA algorithm. [less ▲]

Detailed reference viewed: 51 (10 ULiège)
Full Text
Peer Reviewed
See detailA Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding
Boukas, Ioannis ULiege; Ernst, Damien ULiege; Théate, Thibaut ULiege et al

E-print/Working paper (2020)

The large integration of variable energy resources is expected to shift a large part of the energy exchanges closer to real-time, where more accurate forecasts are available. In this context, the short ... [more ▼]

The large integration of variable energy resources is expected to shift a large part of the energy exchanges closer to real-time, where more accurate forecasts are available. In this context, the short-term electricity markets and in particular the intraday market are considered a suitable trading floor for these exchanges to occur. A key component for the successful renewable energy sources integration is the usage of energy storage. In this paper, we propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market where exchanges occur through a centralized order book. The goal of the storage device operator is the maximization of the profits received over the entire trading horizon, while taking into account the operational constraints of the unit. The sequential decision-making problem of trading in the intraday market is modelled as a Markov Decision Process. An asynchronous version of the fitted Q iteration algorithm is chosen for solving this problem due to its sample efficiency. The large and variable number of the existing orders in the order book motivates the use of high-level actions and an alternative state representation. Historical data are used for the generation of a large number of artificial trajectories in order to address exploration issues during the learning process. The resulting policy is back-tested and compared against a benchmark strategy that is the current industrial standard. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy. [less ▲]

Detailed reference viewed: 629 (72 ULiège)
Full Text
Peer Reviewed
See detailThe Role of Power-to-Gas and Carbon Capture Technologies in Cross-Sector Decarbonisation Strategies
Berger, Mathias ULiege; Radu, David-Constantin ULiege; Fonteneau, Raphaël ULiege et al

in Electric Power Systems Research (2020), 180

This paper proposes an optimisation-based framework to tackle long-term centralised planning problems of multi-sector, integrated energy systems including electricity, hydrogen, natural gas, synthetic ... [more ▼]

This paper proposes an optimisation-based framework to tackle long-term centralised planning problems of multi-sector, integrated energy systems including electricity, hydrogen, natural gas, synthetic methane and carbon dioxide. The model selects and sizes the set of power generation, energy conversion and storage as well as carbon capture technologies minimising the cost of supplying energy demand in the form of electricity, hydrogen, natural gas or synthetic methane across the power, heating, transportation and industry sectors whilst accounting for policy drivers, such as energy independence, carbon dioxide emissions reduction targets, or support schemes. The usefulness of the model is illustrated by a case study evaluating the potential of sector coupling via power-to-gas and carbon capture technologies to achieve deep decarbonisation targets in the Belgian context. Results, on the one hand, indicate that power-to-gas can only play a minor supporting role in cross-sector decarbonisation strategies in Belgium, as electrolysis plants are deployed in moderate quantities whilst methanation plants do not appear in any studied scenario. On the other hand, given the limited renewable potential, post-combustion and direct air carbon capture technologies clearly play an enabling role in any decarbonisation strategy, but may also exacerbate the dependence on fossil fuels. [less ▲]

Detailed reference viewed: 468 (52 ULiège)
Full Text
See detailLa transition énergétique peut réunir tous les Belges
Ernst, Damien ULiege

Article for general public (2020)

Detailed reference viewed: 123 (9 ULiège)
Full Text
Peer Reviewed
See detailCritical Time Windows for Renewable Resource Complementarity Assessment
Berger, Mathias ULiege; Radu, David-Constantin ULiege; Fonteneau, Raphaël ULiege et al

in Energy (2020)

This paper proposes a framework to assess the complementarity between geographically dispersed variable renewable energy resources over arbitrary time scales. More precisely, the framework relies on the ... [more ▼]

This paper proposes a framework to assess the complementarity between geographically dispersed variable renewable energy resources over arbitrary time scales. More precisely, the framework relies on the concept of critical time windows, which offers an accurate, time-domain description of low-probability power production events impacting power system operation and planning. A scalar criticality indicator is also derived to quantify the spatiotemporal complementarity that renewable generation sites may exhibit, and it is leveraged to propose optimisation models seeking to identify deployment patterns with maximum complementarity. The usefulness of the framework is shown in a case study investigating the complementarity between wind regimes in continental western Europe and southern Greenland, using roughly 300 candidate locations and 10 years of reanalysis and simulated data with hourly resolution. Besides showing that the occurrence of low wind power production events can be reduced on a regional scale by exploiting diversity in local wind patterns, results highlight the fact that aggregating wind power production sites located on different continents may result in a lower occurrence of system-wide low wind power production events and point to potential bene ts of intercontinental electrical interconnections. [less ▲]

Detailed reference viewed: 919 (92 ULiège)
Full Text
Peer Reviewed
See detailSolving Optimal Control Problems for Monotone Systems Using the Koopman Operator
Sootla, Aivar; Stan, Guy-Bart; Ernst, Damien ULiege

in Mauroy, Alexandre; Mezic, Igor; Susuki, Yoshihiko (Eds.) The Koopman Operator in Systems and Control (2020)

Koopman operator theory offers numerous techniques for analysis and control of complex systems. In particular, in this chapter we will argue that for the problem of convergence to an equilibrium, the ... [more ▼]

Koopman operator theory offers numerous techniques for analysis and control of complex systems. In particular, in this chapter we will argue that for the problem of convergence to an equilibrium, the Koopman operator can be used to take advantage of the geometric properties of controlled systems, thus making the optimal solutions more transparent, and easier to analyse and implement. The motivation for the study of the convergence problem comes from biological applications, where easy-to-implement and easy-to-analyse solutions are of particular value. At the moment, theoretical results have been developed for a class of nonlinear systems called monotone systems. However, the core ideas presented here can be applied heuristically to non-monotone systems. Furthermore, the convergence problem can serve as a building block for solving other control problems such as switching between stable equilibria, or inducing oscillations. These applications are illustrated on biologically inspired numerical examples. [less ▲]

Detailed reference viewed: 171 (16 ULiège)
Full Text
Peer Reviewed
See detailIntroducing neuromodulation in deep neural networks to learn adaptive behaviours
Vecoven, Nicolas ULiege; Ernst, Damien ULiege; Wehenkel, Antoine ULiege et al

in PLoS ONE (2020)

Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a ... [more ▼]

Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation, the biological mechanism that dynamically controls intrinsic properties of neurons and their response to external stimuli in a context-dependent manner. In this paper, we take inspiration from cellular neuromodulation to construct a new deep neural network architecture that is specifically designed to learn adaptive behaviours. The network adaptation capabilities are tested on navigation benchmarks in a meta-reinforcement learning context and compared with state-of-the-art approaches. Results show that neuromodulation is capable of adapting an agent to different tasks and that neuromodulation-based approaches provide a promising way of improving adaptation of artificial systems. [less ▲]

Detailed reference viewed: 453 (55 ULiège)
Full Text
See detailLes voitures électriques bientôt majoritaires
Lahaye, Christian; Ernst, Damien ULiege

Article for general public (2020)

Detailed reference viewed: 88 (3 ULiège)
Full Text
See detailThe Smart Grids lab at the University of Liège
Ernst, Damien ULiege; Fonteneau, Raphaël ULiege

Speech/Talk (2020)

Detailed reference viewed: 198 (16 ULiège)