From Agents to Robots: A Training and Evaluation Platform for Multi-robot Reinforcement Learning

Multi-robot reinforcement learning (MRRL) is a promising approach to solving cooperation problems and has been widely adopted in many applications. In the past decades, researchers have proposed various approaches to improve the efficiency of MRRL. However, most of them are trained and evaluated only in simulated environments with simple interaction scenarios. The problem of how these methods perform in the real-world environment with complex interaction scenarios remains unsolved. To meet this emergent need, we introduce a scalable multi-robot reinforcement learning platform (SMART) for training and evaluation. Specifically, SMART consists of two components: 1) a simulation environment with an uncertainty-aware social agent model that provides a variety of complex interaction scenarios for training and 2) a real-world multi-robot system for realistic performance evaluation. To evaluate the generalizability of MRRL baselines, we introduce a novel generalization metric that takes into account their performance across changes in the environment as well as the policies of other agents. Furthermore, we conduct a case study on the multi-vehicle cooperative lane change and summarize the unique challenges of MRRL, which are rarely considered previously. Finally, we open-source the simulation environments, associated benchmark tasks, and state-of-the-art baselines to encourage and empower MRRL research. Our code is available at https://github.com/Blackmamba-xuan/MRST.