QMIX-GNN: A Graph Neural Network-Based Heterogeneous Multi-Agent Reinforcement Learning Model for Improved Collaboration and Decision-Making

Taiyin Zhao, Tian Chen, Bing Zhang

In multi-agent reinforcement learning, the fully centralized approach suffers from issues such as explosion of the joint state and action spaces, leading to performance degradation. On the other hand, the fully decentralized approach relies on agents that focus solely on maximizing their own rewards, making effective collaboration difficult and complicating adaption to scenarios that require cooperation among multiple agents. The Centralized Training and Decentralized Execution (CTDE) framework combines both fully centralized and fully decentralized approaches. During the training phase, a virtual central node receives the observations and actions of all agents for training, while during the execution phase each agent makes decisions based only on its own observations. However, in this framework the agents do not fully consider the information of other agents or the complex interactions between them during execution, which affects the correctness of their decisions. Therefore, this paper proposes a heterogeneous multi-agent reinforcement learning model based on graph neural networks, which we call QMIX-GNN. This model efficiently and flexibly handles input data of different dimensions, enabling the fusion of heterogeneous multi-agent information and providing this fused information to the agents. In turn, this allows them to perceive more comprehensive information and improve the correctness of their decisions. Experimental results demonstrate that the QMIX-GNN model performs better than other methods on complex multi-agent collaborative tasks.