Graph Neural Networks; Reinforcement Learning; Tertiary Voltage Control; power systems
Abstract :
[en] Transmission systems have experienced an increase in the occurrence frequency and intensity of high voltage events over the past few years. Since traditional approaches to optimal power flow do not scale well to real-life systems, it has become urgent to develop new methods to help operators improve tertiary voltage control. In this paper, we propose to train a graph neural network to choose voltage setpoints by interacting with a power grid simulator using reinforcement learning techniques. Moreover, we introduce the hyper heterogeneous multi graph formalism to account for topology variations of real-life systems (assets disconnection, bus-splitting, etc.). Our approach is validated on an artificial case study based on the case60nordic power grid.
Disciplines :
Energy Electrical & electronics engineering Computer science
Author, co-author :
Donon, Balthazar ; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Cubelier, François ; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Karangelos, Efthymios ; Université de Liège - ULiège > Département d'électricité, électronique et informatique (Institut Montefiore) > Méthodes stochastiques
Wehenkel, Louis ; Université de Liège - ULiège > Département d'électricité, électronique et informatique (Institut Montefiore) > Méthodes stochastiques
Crochepierre, Laure; Réseau de Transport d'Électricité (RTE) > Research & Development
Pache, Camille; Réseau de Transport d'Électricité (RTE) > Research & Development
Saludjian, Lucas; Réseau de Transport d'Électricité (RTE) > Research & Development
Panciatici, Patrick; Réseau de Transport d'Électricité (RTE) > Research & Development
Language :
English
Title :
Topology-Aware Reinforcement Learning for Tertiary Voltage Control
Publication date :
27 June 2024
Journal title :
Electric Power Systems Research
ISSN :
0378-7796
eISSN :
1873-2046
Publisher :
Elsevier, Amsterdam, Netherlands
Special issue title :
Proceedings of the 23rd Power Systems Computation Conference (PSCC 2024)
Volume :
234
Pages :
110658
Peer reviewed :
Peer Reviewed verified by ORBi
Tags :
CÉCI : Consortium des Équipements de Calcul Intensif
P. Mandoulidis, et al., Controlling long-term overvoltages in lightly loaded transmission systems, in: 13th Mediterranean Conference on Power Generation, Transmission, Distribution and Energy Conversion, Vol. 2022, MEDPOWER 2022, 2022, pp. 7–12.
Castillo, A., Essays on the ACOPF Problem: Formulations, Approximations, and Applications in the Electricity Markets. (Ph.D. thesis), 2016.
Goodfellow, I., Bengio, Y., Courville, A., Deep Learning. 2016, MIT Press.
C. Szegedy, et al., Going deeper with convolutions, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Los Alamitos, CA, USA, 2015, pp. 1–9.
Sutskever, I., Vinyals, O., Le, Q.V., Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, vol. 27, 2014.
Silver, D., et al. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484–489.
Sutton, R.S., Barto, A.G., Reinforcement Learning: An Introduction. 1998, MIT Press.
Pan, X., Chen, M., Zhao, T., Low, S.H., DeepOPF: A feasibility-optimized deep neural network approach for AC optimal power flow problems. IEEE Syst. J. 17 (2023), 673–683.
B.L. Thayer, T.J. Overbye, Deep Reinforcement Learning for Electric Transmission Voltage Control, in: 2020 IEEE Electric Power and Energy Conference, EPEC, 2020, pp. 1–8.
H. Hagmar, L.A. Tuan, R. Eriksson, Deep Reinforcement Learning for Long-Term Voltage Stability Control, in: 11th Bulk Power Systems Dynamics and Control Symposium, IREP 2022, 2022.
Zhen, H., et al. Design and tests of reinforcement-learning-based optimal power flow solution generator. Energy Rep. 8 (2022), 43–50.
Zhou, Y., et al. A data-driven method for fast AC optimal power flow solutions via deep reinforcement learning. J. Mod. Power Syst. Clean Energy 8 (2020), 1128–1139.
Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G., The graph neural network model. IEEE Trans. Neural Netw. 20 (2009), 61–80.
T.N. Kipf, M. Welling, Semi-Supervised Classification with Graph Convolutional Networks, in: International Conference on Learning Representations, ICLR 2017, 2017.
Liao, W., Bak-Jensen, B., Pillai, J.R., Wang, Y., Wang, Y., A review of graph neural networks and their applications in power systems. Journal of Modern Power Systems and Clean Energy, vol. 10, 2022.
Lopez-Cardona, A., Bernardez, G., Barlet-Ros, P., Cabellos-Aparicio, A., Proximal policy optimization with graph neural networks for optimal power flow. 2022 Preprint arXiv:2212.12470.
Owerko, D., Gama, F., Ribeiro, A., Unsupervised optimal power flow using graph neural networks. 2022 Preprint arXiv:2210.09277.
Li, J., Zhang, R., Wang, H., Liu, Z., Lai, H., Zhang, Y., Deep reinforcement learning for optimal power flow with renewables using graph information. 2022 Preprint arXiv:2112.11461.
F. Diehl, Warm-Starting AC Optimal Power Flow with Graph Neural Networks, in: NeurIPS 2019 Workshop on Tackling Climate Change with Machine Learning, 2019.
Donon, B., Deep Statistical Solvers & Power Systems Applications. (Ph.D. thesis), 2022.
B. Donon, et al., Topology-Aware Reinforcement Learning for Tertiary Voltage Control - Supplementary Material, Tech. Rep., 2023,.
Karpathy, A., Deep reinforcement learning: Pong from pixels. Andrej Karpathy blog, 2016 http://karpathy.github.io/2016/05/31/rl/.
Capitanescu, F., Suppressing ineffective control actions in optimal power flow problems. IET Gener. Transm. Distrib. 14 (2020), 2520–2527.
Bezanson, J., Edelman, A., Karpinski, S., Shah, V., Julia: A fresh approach to numerical computing. SIAM Rev. 59 (2017), 65–98.
Dunning, I., Huchette, J., Lubin, M., JuMP: A modeling language for mathematical optimization. SIAM Rev. 59 (2017), 295–320.
C. Coffrin, R. Bent, K. Sundar, Y. Ng, M. Lubin, PowerModels.jl: An Open-Source Framework for Exploring Power Formulations, in: 2018 Power Systems Computation Conference, PSCC, 2018.
Van Rossum, G., Drake, F.L., Python 3 reference manual. 2009.
Bradbury, J., et al. JAX: composable transformations of Python+NumPy programs. 2018.
Thurner, L., et al. pandapower - an open source python tool for convenient modeling, analysis and optimization of electric power systems. IEEE Trans. Power Syst., 2018.
Heek, J., et al. Flax: A neural network library and ecosystem for JAX. 2023.
Kidger, P., On Neural Differential Equations. (Ph.D. thesis), 2021.
D.P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization, in: International Conference on Learning Representations, ICLR, 2015.
