Gym-TORAX: Open-source software for integrating RL with plasma control simulators

This paper presents Gym-TORAX, a Python package enabling the implementation
of Reinforcement Learning (RL) environments for simulating plasma dynamics and
control in tokamaks. Users define succinctly a set of control actions and
observations, and a control objective from which Gym-TORAX creates a Gymnasium
environment that wraps TORAX for simulating the plasma dynamics. The objective
is formulated through rewards depending on the simulated state of the plasma
and control action to optimize specific characteristics of the plasma, such as
performance and stability. The resulting environment instance is then
compatible with a wide range of RL algorithms and libraries and will facilitate
RL research in plasma control. In its current version, one environment is
readily available, based on a ramp-up scenario of the International
Thermonuclear Experimental Reactor (ITER).