Pattern-RL: Multi-robot cooperative pattern formation via deep reinforcement learning

Jia Wang, Jiannong Cao, Milos Stojmenovic, Miao Zhao, Jinlin Chen, Shan Jiang

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

1 Citation (Scopus)

Abstract

Autonomous, arbitrary pattern formation is one of the most critical applications in multi-robot systems, where robots are required to form into circles, lines, and meshes or any other desired configuration. This task is important in military applications, search and rescue operations, and visual inspection of infrastructure and equipment tasks to name a few. Most existing works are very rigid, and only able to form certain shapes, where slight target changes can cause failure in the predefined pattern-specific rules and trigger algorithm redesign. We propose a novel, deep reinforcement learning-based method that generates general-purpose pattern formation strategies, in the form of deep neural networks (DNN), for any target pattern. Our method uses the trial-and-error feedback of each round of training to gradually generate the pattern formation strategy. Thus, robots are able to query the trained DNN model to select their optimal directions and speeds in a fully distributed manner. Considering that reinforcement learning models do not perform well with large state spaces and highly variant training samples, we employ auto-encoders to learn the condensed representation for each state and compute model-free policy gradients for arbitrary pattern formation. We experimentally show that groups of robots are able to form various general target patterns while minimizing the number of completion time steps.

Original languageEnglish
Title of host publicationProceedings - 18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019
EditorsM. Arif Wani, Taghi M. Khoshgoftaar, Dingding Wang, Huanjing Wang, Naeem Seliya
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages210-215
Number of pages6
ISBN (Electronic)9781728145495
DOIs
Publication statusPublished - Dec 2019
Event18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019 - Boca Raton, United States
Duration: 16 Dec 201919 Dec 2019

Publication series

NameProceedings - 18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019

Conference

Conference18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019
CountryUnited States
CityBoca Raton
Period16/12/1919/12/19

Keywords

  • Cooperation
  • Multi agent
  • Multi Robot System
  • Pattern Formation
  • Reinforcement Learning

ASJC Scopus subject areas

  • Strategy and Management
  • Artificial Intelligence
  • Computer Science Applications
  • Decision Sciences (miscellaneous)
  • Signal Processing
  • Media Technology

Cite this