IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARL

We introduce IMP-MARL, an open-source suite of multi-agent reinforcement
learning (MARL) environments for large-scale Infrastructure Management Planning
(IMP), offering a platform for benchmarking the scalability of cooperative MARL
methods in real-world engineering applications. In IMP, a multi-component
engineering system is subject to a risk of failure due to its components'
damage condition. Specifically, each agent plans inspections and repairs for a
specific system component, aiming to minimise maintenance costs while
cooperating to minimise system failure risk. With IMP-MARL, we release several
environments including one related to offshore wind structural systems, in an
effort to meet today's needs to improve management strategies to support
sustainable and reliable energy systems. Supported by IMP practical engineering
environments featuring up to 100 agents, we conduct a benchmark campaign, where
the scalability and performance of state-of-the-art cooperative MARL methods
are compared against expert-based heuristic policies. The results reveal that
centralised training with decentralised execution methods scale better with the
number of agents than fully centralised or decentralised RL approaches, while
also outperforming expert-based heuristic policies in most IMP environments.
Based on our findings, we additionally outline remaining cooperation and
scalability challenges that future MARL methods should still address. Through
IMP-MARL, we encourage the implementation of new environments and the further
development of MARL methods.