Article (Scientific journals)
A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding
Boukas, Ioannis; Ernst, Damien; Théate, Thibaut et al.
2021In Machine Learning
Peer Reviewed verified by ORBi
 

Files


Full Text
Boukas2021_Article_ADeepReinforcementLearningFram.pdf
Publisher postprint (4.16 MB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
European continuous intraday markets; Energy storage control; Markov decision process; Deep reinforcement learning; Asynchronous fitted Q iteration
Abstract :
[en] The large integration of variable energy resources is expected to shift a large part of the energy exchanges closer to real-time, where more accurate forecasts are available. In this context, the short-term electricity markets and in particular the intraday market are considered a suitable trading floor for these exchanges to occur. A key component for the successful renewable energy sources integration is the usage of energy storage. In this paper, we propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market where exchanges occur through a centralized order book. The goal of the storage device operator is the maximization of the profits received over the entire trading horizon, while taking into account the operational constraints of the unit. The sequential decision-making problem of trading in the intraday market is modelled as a Markov Decision Process. An asynchronous version of the fitted Q iteration algorithm is chosen for solving this problem due to its sample efficiency. The large and variable number of the existing orders in the order book motivates the use of high-level actions and an alternative state representation. Historical data are used for the generation of a large number of artificial trajectories in order to address exploration issues during the learning process. The resulting policy is back-tested and compared against a number of benchmark strategies. Finally, the impact of the storage characteristics on the total revenues collected in the intraday market is evaluated.
Disciplines :
Energy
Author, co-author :
Boukas, Ioannis  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart-Microgrids
Ernst, Damien  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Théate, Thibaut ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Bolland, Adrien ;  Université de Liège - ULiège > Master ingé. civ. électr., à fin.
Huynen, Alexandre
Buchwald, Martin
Wynants, Christelle
Cornélusse, Bertrand  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart-Microgrids
Language :
English
Title :
A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding
Publication date :
July 2021
Journal title :
Machine Learning
ISSN :
0885-6125
eISSN :
1573-0565
Publisher :
Kluwer Academic Publishers, Netherlands
Peer reviewed :
Peer Reviewed verified by ORBi
Available on ORBi :
since 12 February 2019

Statistics


Number of views
1689 (131 by ULiège)
Number of downloads
912 (42 by ULiège)

Scopus citations®
 
21
Scopus citations®
without self-citations
21
OpenCitations
 
14

Bibliography


Similar publications



Contact ORBi