References of "Ernst, Damien"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailGYM-ANM: Reinforcement learning environments for active network management tasks in electricity distribution systems
Henry, Robin; Ernst, Damien ULiege

in Energy and AI (2021), 5

Active network management (ANM) of electricity distribution networks include many complex stochastic sequential optimization problems. These problems need to be solved for integrating renewable energies ... [more ▼]

Active network management (ANM) of electricity distribution networks include many complex stochastic sequential optimization problems. These problems need to be solved for integrating renewable energies and distributed storage into future electrical grids. In this work, we introduce Gym-ANM, a framework for designing reinforcement learning (RL) environments that model ANM tasks in electricity distribution networks. These environments provide new playgrounds for RL research in the management of electricity networks that do not require an extensive knowledge of the underlying dynamics of such systems. Along with this work, we are releasing an implementation of an introductory toy-environment, ANM6-Easy, designed to emphasize common challenges in ANM. We also show that state-of-the-art RL algorithms can already achieve good performance on ANM6-Easy when compared against a model predictive control (MPC) approach. Finally, we provide guidelines to create new Gym-ANM environments differing in terms of (a) the distribution network topology and parameters, (b) the observation space, (c) the modelling of the stochastic processes present in the system, and (d) a set of hyperparameters influencing the reward signal. Gym-ANM can be downloaded at https://github.com/robinhenry/gym-anm. [less ▲]

Detailed reference viewed: 95 (30 ULiège)
Full Text
Peer Reviewed
See detailA Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding
Boukas, Ioannis ULiege; Ernst, Damien ULiege; Théate, Thibaut ULiege et al

in Machine Learning (2021)

The large integration of variable energy resources is expected to shift a large part of the energy exchanges closer to real-time, where more accurate forecasts are available. In this context, the short ... [more ▼]

The large integration of variable energy resources is expected to shift a large part of the energy exchanges closer to real-time, where more accurate forecasts are available. In this context, the short-term electricity markets and in particular the intraday market are considered a suitable trading floor for these exchanges to occur. A key component for the successful renewable energy sources integration is the usage of energy storage. In this paper, we propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market where exchanges occur through a centralized order book. The goal of the storage device operator is the maximization of the profits received over the entire trading horizon, while taking into account the operational constraints of the unit. The sequential decision-making problem of trading in the intraday market is modelled as a Markov Decision Process. An asynchronous version of the fitted Q iteration algorithm is chosen for solving this problem due to its sample efficiency. The large and variable number of the existing orders in the order book motivates the use of high-level actions and an alternative state representation. Historical data are used for the generation of a large number of artificial trajectories in order to address exploration issues during the learning process. The resulting policy is back-tested and compared against a number of benchmark strategies. Finally, the impact of the storage characteristics on the total revenues collected in the intraday market is evaluated. [less ▲]

Detailed reference viewed: 1349 (111 ULiège)
Full Text
Peer Reviewed
See detailA bio-inspired bistable recurrent cell allows for long-lasting memory
Vecoven, Nicolas ULiege; Ernst, Damien ULiege; Drion, Guillaume ULiege

in PLoS ONE (2021), 16(6), 1-13

Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as ... [more ▼]

Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network level, and long term memory is shaped by network-wide recurrent connection weights. Biological neurons on the other hand are capable of holding information at the cellular level for an arbitrary long amount of time through a process called bistability. Through bistability, cells can stabilize to different stable states depending on their own past state and inputs, which permits the durable storing of past information in neuron state. In this work, we take inspiration from biological neuron bistability to embed RNNs with long-lasting memory at the cellular level. This leads to the introduction of a new bistable biologically-inspired recurrent cell that is shown to strongly improves RNN performance on time-series which require very long memory, despite using only cellular connections (all recurrent connections are from neurons to themselves, i.e. a neuron state is not influenced by the state of other neurons). Furthermore, equipping this cell with recurrent neuromodulation permits to link them to standard GRU cells, taking a step towards the biological plausibility of GRU. With this link, this work paves the way for studying more complex and biologically plausible neuromodulation schemes as gating mechanisms in RNNs. [less ▲]

Detailed reference viewed: 108 (15 ULiège)
Full Text
Peer Reviewed
See detailRemote Renewable Hubs for Carbon-Neutral Synthetic Fuel Production
Berger, Mathias ULiege; Radu, David-Constantin ULiege; Detienne, Ghislain et al

in Frontiers in Energy Research (2021)

This paper studies the economics of carbon-neutral synthetic fuel production from renewable electricity in remote areas where high-quality renewable resources are abundant. To this end, a graph-based ... [more ▼]

This paper studies the economics of carbon-neutral synthetic fuel production from renewable electricity in remote areas where high-quality renewable resources are abundant. To this end, a graph-based optimisation modelling framework directly applicable to the strategic planning of remote renewable energy supply chains is proposed. More precisely, a hypergraph abstraction of planning problems is introduced, wherein nodes can be viewed as optimisation subproblems with their own parameters, variables, constraints and local objective. Nodes typically represent a subsystem such as a technology, a plant or a process. Hyperedges, on the other hand, express the connectivity between subsystems. The framework is leveraged to study the economics of carbon-neutral synthetic methane production from solar and wind energy in North Africa and its delivery to Northwestern European markets. The full supply chain is modelled in an integrated fashion, which makes it possible to accurately capture the interaction between various technologies on an hourly time scale. Results suggest that the cost of synthetic methane production and delivery would be slightly under 150 €/MWh (higher heating value) by 2030 for a system supplying 10 TWh annually and relying on a combination of solar photovoltaic and wind power plants, assuming a uniform weighted average cost of capital of 7%. A comprehensive sensitivity analysis is also carried out in order to assess the impact of various techno-economic parameters and assumptions on synthetic methane cost, including the availability of wind power plants, the investment costs of electrolysis, methanation and direct air capture plants, their operational flexibility, the energy consumption of direct air capture plants, and financing costs. The most expensive configuration (around 200 €/MWh) relies on solar photovoltaic power plants alone, while the cheapest configuration (around 88 €/MWh) makes use of a combination of solar PV and wind power plants and is obtained when financing costs are set to zero. [less ▲]

Detailed reference viewed: 708 (57 ULiège)
Full Text
Peer Reviewed
See detailResidential Energy Communities: How to minimize the investment risk from an investor perspective
Vangulick, David ULiege; Manuel de Villena Millan, Miguel ULiege; Fonteneau, Raphaël ULiege et al

in Proceedings of the CIRED 2021 Conference (2021, June)

The success of local renewable energy communities, now foreseen by new the European Union directives but also growing worldwide, will rely on the appetite of consumers and investors. This is not obvious ... [more ▼]

The success of local renewable energy communities, now foreseen by new the European Union directives but also growing worldwide, will rely on the appetite of consumers and investors. This is not obvious when the target local area is a residential community where people have varying expectations. Based on Bayesian game theory (also called game of incomplete information), the purpose of this paper is to define an approach for determining, from the point of view of the renewable energy investor, the level of production capacity and optimum energy price to be offered to the consumers. [less ▲]

Detailed reference viewed: 236 (4 ULiège)
Full Text
Peer Reviewed
See detailGym-ANM: Open-source software to leverage reinforcement learning for power system management in research and education
Henry, Robin; Ernst, Damien ULiege

in Software Impacts (2021), 9

Gym-ANM is a Python package that facilitates the design of reinforcement learning (RL) environments that model active network management (ANM) tasks in electricity networks. Here, we describe how to ... [more ▼]

Gym-ANM is a Python package that facilitates the design of reinforcement learning (RL) environments that model active network management (ANM) tasks in electricity networks. Here, we describe how to implement new environments and how to write code to interact with pre-existing ones. We also provide an overview of ANM6-Easy, an environment designed to highlight common ANM challenges. Finally, we discuss the potential impact of Gym-ANM on the scientific community, both in terms of research and education. We hope this package will facilitate collaboration between the power system and RL communities in the search for algorithms to control future energy systems. [less ▲]

Detailed reference viewed: 179 (11 ULiège)
Full Text
Peer Reviewed
See detailWarming-up recurrent neural networks to maximize reachable multi-stability greatly improves learning
Vecoven, Nicolas ULiege; Ernst, Damien ULiege; Drion, Guillaume ULiege

E-print/Working paper (2021)

Training recurrent neural networks is known to be difficult when time dependencies become long. Consequently, training standard gated cells such as gated recurrent units and long-short term memory on ... [more ▼]

Training recurrent neural networks is known to be difficult when time dependencies become long. Consequently, training standard gated cells such as gated recurrent units and long-short term memory on benchmarks where long-term memory is required remains an arduous task. In this work, we propose a general way to initialize any recurrent network connectivity through a process called “warmup” to improve its capability to learn arbitrarily long time dependencies. This initialization process is designed to maximize network reachable multi-stability, i.e. the number of attractors within the network that can be reached through relevant input trajectories. Warming-up is performed before training, using stochastic gradient descent on a specifically designed loss. We show that warming-up greatly improves recurrent neural network performance on long-term memory benchmarks for multiple recurrent cell types, but can sometimes impede precision. We therefore introduce a parallel recurrent network structure with partial warm-up that is shown to greatly improve learning on long time-series while maintaining high levels of precision. This approach provides a general framework for improving learning abilities of any recurrent cell type when long-term memory is required. [less ▲]

Detailed reference viewed: 49 (2 ULiège)
Full Text
Peer Reviewed
See detailModel Reduction in Capacity Expansion Planning Problems via Renewable Generation Site Selection
Radu, David-Constantin ULiege; Dubois, Antoine ULiege; Berger, Mathias ULiege et al

in Proceedings of the 2021 IEEE Madrid PowerTech (2021, June)

The accurate representation of variable renewable generation (RES, e.g., wind, solar PV) assets in capacity expansion planning (CEP) studies is paramount to capture spatial and temporal correlations that ... [more ▼]

The accurate representation of variable renewable generation (RES, e.g., wind, solar PV) assets in capacity expansion planning (CEP) studies is paramount to capture spatial and temporal correlations that may exist between sites and impact both power system design and operation. However, it typically has a high computational cost. This paper proposes a method to reduce the spatial dimension of CEP problems while preserving an accurate representation of renewable energy sources. A two-stage approach is proposed to this end. In the first stage, relevant sites are identified via a screening routine that discards the locations with little impact on system design. In the second stage, the subset of relevant RES sites previously identified is used in a CEP problem to determine the optimal configuration of the power system. The proposed method is tested on a realistic EU case study and its performance is benchmarked against a CEP set-up in which the entire set of candidate RES sites is available. The method shows great promise, with the screening stage consistently identifying 90% of the optimal RES sites while discarding up to 54% of the total number of candidate locations. This leads to a peak memory reduction of up to 41% and solver runtime gains between 31% and 46%, depending on the weather year considered. [less ▲]

Detailed reference viewed: 168 (19 ULiège)
Full Text
Peer Reviewed
See detailLow-voltage network topology and impedance identification using smart meter measurements
Benzerga, Amina ULiege; Maruli, Daniele; Sutera, Antonio ULiege et al

in Proceedings of the 2021 IEEE Madrid PowerTech (2021, June)

Distribution system operators have been upgrading their network over several decades, though not always keeping digital records of all changes. As a result, the operators do not always know exactly how ... [more ▼]

Distribution system operators have been upgrading their network over several decades, though not always keeping digital records of all changes. As a result, the operators do not always know exactly how their customers are connected to a network. Some of these customers are equipped with smart meters, providing voltage and current time-series. These measurements can be used to identify the network topology and the line impedances. This paper presents a method to identify radially operated low-voltage networks which can be applied with limited number of smart meters. The resulting identified model provides the map of the network and impedances of the inferred lines, allowing to perform subsequent analyses (e.g. power-flow). Simulation results on a case study with 128 nodes show an average error of 0.69% in computed voltages, while only 40% of the nodes are equipped with smart meters. [less ▲]

Detailed reference viewed: 203 (11 ULiège)
Full Text
See detailLe rendez-vous manqué de la Wallonie avec les communautés d’énergie renouvelable
Goffin, Philippe; El Mokhtari, Yakhlef; Ernst, Damien ULiege

Article for general public (2021)

Detailed reference viewed: 497 (2 ULiège)
Full Text
See detailM4Depth: A motion-based approach for monocular depth estimation on video sequences
Fonder, Michaël ULiege; Ernst, Damien ULiege; Van Droogenbroeck, Marc ULiege

E-print/Working paper (2021)

Getting the distance to objects is crucial for autonomous vehicles. In instances where depth sensors cannot be used, this distance has to be estimated from RGB cameras. As opposed to cars, the task of ... [more ▼]

Getting the distance to objects is crucial for autonomous vehicles. In instances where depth sensors cannot be used, this distance has to be estimated from RGB cameras. As opposed to cars, the task of estimating depth from on-board mounted cameras is made complex on drones because of the lack of constrains on motion during flights. In this paper, we present a method to estimate the distance of objects seen by an on-board mounted camera by using its RGB video stream and drone motion information. Our method is built upon a pyramidal convolutional neural network architecture and uses time recurrence in pair with geometric constraints imposed by motion to produce pixel-wise depth maps. In our architecture, each level of the pyramid is designed to produce its own depth estimate based on past observations and information provided by the previous level in the pyramid. We introduce a spatial reprojection layer to maintain the spatio-temporal consistency of the data between the levels. We analyse the performance of our approach on Mid-Air, a public drone dataset featuring synthetic drone trajectories recorded in a wide variety of unstructured outdoor environments. Our experiments show that our network outperforms state-of-the-art depth estimation methods and that the use of motion information is the main contributing factor for this improvement. The code of our method is publicly available on GitHub; see https://github.com/michael-fonder/M4Depth [less ▲]

Detailed reference viewed: 160 (23 ULiège)
Full Text
Peer Reviewed
See detailSparse Training Theory for Scalable and Efficient Agents
Mocanu; Mocanu; Pinto, Tiago et al

in Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems - Blue Sky Ideas Track (2021, May)

A fundamental task for artificial intelligence is learning. Deep Neural Networks have proven to cope perfectly with all learnin paradigms, i.e. supervised, unsupervised, and reinforcement learning ... [more ▼]

A fundamental task for artificial intelligence is learning. Deep Neural Networks have proven to cope perfectly with all learnin paradigms, i.e. supervised, unsupervised, and reinforcement learning. Nevertheless, traditional deep learning approaches make use of cloud computing facilities and do not scale well to autonomous agents with low computational resources. Even in the cloud, theysuffer from computational and memory limitations, and they cannot be used to model adequately large physical worlds for agents which assume networks with billions of neurons. These issues are addressed in the last few years by the emerging topic of sparse training, which trains sparse networks from scratch. This paper discusses sparse training state-of-the-art, its challenges and limitations while introducing a couple of new theoretical research directions which has the potential of alleviating sparse training limitations to push deep learning scalability well beyond its current boundaries. Nevertheless, the theoretical advancements impact in complex multi-agents settings is discussed from a real-world perspective, using the smart grid case study. [less ▲]

Detailed reference viewed: 81 (2 ULiège)
Full Text
Peer Reviewed
See detailModelling and Assessing the Impact of the DSO Remuneration Strategy on its Interaction with Electricity Users
Manuel de Villena Millan, Miguel ULiege; Gautier, Axel ULiege; Ernst, Damien ULiege et al

in International Journal of Electrical Power and Energy Systems (2021), 126(Part A), 106585

This paper presents a simulation-based methodology for assessing the impact of employing different distribution system operator’s remuneration strategies on the economic sustainability of electrical ... [more ▼]

This paper presents a simulation-based methodology for assessing the impact of employing different distribution system operator’s remuneration strategies on the economic sustainability of electrical distribution systems. The proposed methodology accounts for the uncertainties posed by the integration of distributed electricity generation resources, and the roll out of smart-meters. The different remuneration strategies analysed in this paper include notably new distribution tariffs based on individual peak power consumption and time-dependent rates that are contingent on the time of energy consumption, both requiring smart-meters to work. The distributed electricity generation resources are modelled through an optimisation framework and an investment decision process that gradually deploys household photovoltaic installations depending on their profitability and the electricity charges, including the distribution rates. The impact of the distribution system operator’s remuneration strategy is measured by an accurate modelling of the remuneration mechanism of this entity, which can adapt to various distribution tariff designs. We analyse this impact over a discrete time horizon. Our methodology is illustrated with several examples of distribution tariffs including old –based on energy consumption or on per-connection fees– as well as new –based on power consumption or time-of use fees– designs. Finally, we provide a comprehensive sensitivity analysis of the proposed simulation environment to the main parameters of the methodology. [less ▲]

Detailed reference viewed: 641 (95 ULiège)
Full Text
Peer Reviewed
See detailQVMix and QVMix-Max: Extending the Deep Quality-Value Family of Algorithms to Cooperative Multi-Agent Reinforcement Learning
Leroy, Pascal ULiege; Ernst, Damien ULiege; Geurts, Pierre ULiege et al

in Proceedings of the AAAI-21 Workshop on Reinforcement Learning in Games (2021, February)

This paper introduces four new algorithms that can be used for tackling multi-agent reinforcement learning (MARL) problems occurring in cooperative settings. All algorithms are based on the Deep Quality ... [more ▼]

This paper introduces four new algorithms that can be used for tackling multi-agent reinforcement learning (MARL) problems occurring in cooperative settings. All algorithms are based on the Deep Quality-Value (DQV) family of algorithms, a set of techniques that have proven to be successful when dealing with single-agent reinforcement learning problems (SARL). The key idea of DQV algorithms is to jointly learn an approximation of the state-value function V , alongside an approximation of the state-action value function Q. We follow this principle and generalise these algorithms by introducing two fully decentralised MARL algorithms (IQV and IQV-Max) and two algorithms that are based on the centralised training with decentralised execution training paradigm (QVMix and QVMix-Max). We compare our algorithms with state-of-the-art MARL techniques on the popular StarCraft Multi-Agent Challenge (SMAC) environment. We show competitive results when QVMix and QVMix-Max are compared to well-known MARL techniques such as QMIX and MAVEN and show that QVMix can even outperform them on some of the tested environments, being the algorithm which performs best overall. We hypothesise that this is due to the fact that QVMix suffers less from the overestimation bias of the Q function. [less ▲]

Detailed reference viewed: 459 (16 ULiège)
Full Text
See detailGraph-Based Optimization Modeling Language: A Tutorial
Berger, Mathias ULiege; Bolland, Adrien ULiege; Miftari, Bardhyl ULiege et al

E-print/Working paper (2021)

This paper introduces the graph-based optimization modeling language (GBOML), which enables the easy implementation of a broad class of structured mixed-integer linear programs typically found in ... [more ▼]

This paper introduces the graph-based optimization modeling language (GBOML), which enables the easy implementation of a broad class of structured mixed-integer linear programs typically found in applications ranging from energy system planning to supply chain management. More precisely, the language is particularly well-suited for representing problems involving the optimization of discrete-time dynamical systems over a finite time horizon and possessing a block decomposable structure that can be encoded by a sparse connected hypergraph. The language combines elements of both algebraic and object-oriented modeling languages in order to facilitate problem encoding and post-processing. This document discusses the abstract problem class that can be represented using the modeling language, details its grammar and provides two relevant examples of applications. The first example deals with the deployment of a microgrid system, while the second example focuses on the design and analysis of remote carbon-neutral fuel supply chains. [less ▲]

Detailed reference viewed: 395 (82 ULiège)
Full Text
See detailDistributional Reinforcement Learning with Unconstrained Monotonic Neural Networks
Théate, Thibaut ULiege; Wehenkel, Antoine ULiege; Bolland, Adrien ULiege et al

E-print/Working paper (2021)

The distributional reinforcement learning (RL) approach advocates for representing the complete probability distribution of the random return instead of only modelling its expectation. A distributional RL ... [more ▼]

The distributional reinforcement learning (RL) approach advocates for representing the complete probability distribution of the random return instead of only modelling its expectation. A distributional RL algorithm may be characterised by two main components, namely the representation and parameterisation of the distribution and the probability metric defining the loss. This research considers the unconstrained monotonic neural network (UMNN) architecture, a universal approximator of continuous monotonic functions which is particularly well suited for modelling different representations of a distribution (PDF, CDF, quantile function). This property enables the decoupling of the effect of the function approximator class from that of the probability metric. The paper firstly introduces a methodology for learning different representations of the random return distribution. Secondly, a novel distributional RL algorithm named unconstrained monotonic deep Q-network (UMDQN) is presented. Lastly, in light of this new algorithm, an empirical comparison is performed between three probability quasimetrics, namely the Kullback-Leibler divergence, Cramer distance and Wasserstein distance. The results call for a reconsideration of all probability metrics in distributional RL, which contrasts with the dominance of the Wasserstein distance in recent publications. [less ▲]

Detailed reference viewed: 26 (3 ULiège)
Full Text
Peer Reviewed
See detailAssessing the Impact of Offshore Wind Siting Strategies on the Design of the European Power System
Radu, David-Constantin ULiege; Berger, Mathias ULiege; Dubois, Antoine ULiege et al

E-print/Working paper (2021)

This paper provides a detailed account of the impact of different offshore wind siting strategies on the design of the European power system. To this end, a two-stage method is proposed. In the first ... [more ▼]

This paper provides a detailed account of the impact of different offshore wind siting strategies on the design of the European power system. To this end, a two-stage method is proposed. In the first stage, a highly-granular siting problem identifies a suitable set of sites where offshore wind plants could be deployed according to a pre-specified criterion. Two siting schemes are analysed and compared within a realistic case study. These schemes essentially select a pre-specified number of sites so as to maximise their aggregate power output and their spatiotemporal complementarity, respectively. In addition, two variants of these siting schemes are provided, wherein the number of sites to be selected is specified on a country-by-country basis rather than Europe-wide. In the second stage, the subset of previously identified sites is passed to a capacity expansion planning framework that sizes the power generation, transmission and storage assets that should be deployed and operated in order to satisfy pre-specified electricity demand levels at minimum cost. Results show that the complementarity-based siting criterion leads to system designs which are up to 5% cheaper than the ones relying the power output-based criterion when offshore wind plants are deployed with no consideration for country-based deployment targets. On the contrary, the power output-based scheme leads to system designs which are consistently 2% cheaper than the ones leveraging the complementarity-based siting strategy when such constraints are enforced. The robustness of the results is supported by a sensitivity analysis on offshore wind capital expenditure and inter-annual weather variability, respectively. [less ▲]

Detailed reference viewed: 158 (29 ULiège)