Paper published in a book (Scientific congresses and symposiums)
QVMix and QVMix-Max: Extending the Deep Quality-Value Family of Algorithms to Cooperative Multi-Agent Reinforcement Learning
Leroy, Pascal; Ernst, Damien; Geurts, Pierre et al.
2021In Proceedings of the AAAI-21 Workshop on Reinforcement Learning in Games
Peer reviewed
 

Files


Full Text
2012.12062v1.pdf
Publisher postprint (3.17 MB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
multi-agent reinforcement learning; Deep Quality-Value
Abstract :
[en] This paper introduces four new algorithms that can be used for tackling multi-agent reinforcement learning (MARL) problems occurring in cooperative settings. All algorithms are based on the Deep Quality-Value (DQV) family of algorithms, a set of techniques that have proven to be successful when dealing with single-agent reinforcement learning problems (SARL). The key idea of DQV algorithms is to jointly learn an approximation of the state-value function V , alongside an approximation of the state-action value function Q. We follow this principle and generalise these algorithms by introducing two fully decentralised MARL algorithms (IQV and IQV-Max) and two algorithms that are based on the centralised training with decentralised execution training paradigm (QVMix and QVMix-Max). We compare our algorithms with state-of-the-art MARL techniques on the popular StarCraft Multi-Agent Challenge (SMAC) environment. We show competitive results when QVMix and QVMix-Max are compared to well-known MARL techniques such as QMIX and MAVEN and show that QVMix can even outperform them on some of the tested environments, being the algorithm which performs best overall. We hypothesise that this is due to the fact that QVMix suffers less from the overestimation bias of the Q function.
Disciplines :
Computer science
Author, co-author :
Leroy, Pascal  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Ernst, Damien  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Geurts, Pierre  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Algorith. des syst. en interaction avec le monde physique
Louppe, Gilles  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Big Data
Pisane, Jonathan
Sabatelli, Matthia ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Algorith. des syst. en interaction avec le monde physique
Language :
English
Title :
QVMix and QVMix-Max: Extending the Deep Quality-Value Family of Algorithms to Cooperative Multi-Agent Reinforcement Learning
Publication date :
February 2021
Event name :
AAAI-21 Workshop on Reinforcement Learning in Games
Event date :
08-02-2021
Audience :
International
Main work title :
Proceedings of the AAAI-21 Workshop on Reinforcement Learning in Games
Pages :
8
Peer reviewed :
Peer reviewed
Available on ORBi :
since 26 December 2020

Statistics


Number of views
565 (30 by ULiège)
Number of downloads
129 (11 by ULiège)

Bibliography


Similar publications



Contact ORBi