Forest Fire Control with Learning from Demonstration and Reinforcement Learning

This paper describes a novel approach to control forest fires in a simulated environment using connectionist reinforcement learning (RL) algorithms. A forest fire simulator is introduced that allows to benchmark several popular model-free RL algorithms that are combined with multilayer perceptrons that serve as a value function approximator. For our experiments, we test in total four different algorithms: Q-Learning, SARSA, Dueling Q-Networks and a novel algorithm called Dueling-SARSA. To enable the algorithms to better cope with the difficulty to contain the forest fires when they start learning, we use demonstration data that is inserted in an experience-replay memory buffer before learning. In the experiments, the performance of these algorithms are compared under different experimental setups ranging from the complexity of the simulated environment to how much demonstration data is initially given. The results show that the demonstration data are necessary to learn very good policies for controlling the forest fires in our simulator and that the novel Dueling-SARSA algorithm performs best.