Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control

Yifan Zhao, Gangyan Xu, Yali Duy, Meng Fangz

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

Adaptive traffic signal control is widely recognized as an effective solution to improve urban mobility and reduce congestion in metropolises. Recently, reinforcement learning has been adopted for this transportation problem. While centralized reinforcement learning inevitably faces action space explosion, decentralized reinforcement learning allows agents to develop policies based on local observations but suffers from unstable training. In this paper, we present CommNetPF, a multi-agent decentralized reinforcement learning model incorporating communication and neighbourhood policy fingerprints for adaptive traffic signal control. With policy fingerprints in communication, agents learn to produce cooperative policies and the model converges faster. Experiments in scenarios of adaptive traffic signal control show that CommNetPF outperforms several strong baselines in terms of control performance and convergence speed.

Original languageEnglish
Title of host publication2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020
PublisherIEEE Computer Society
Pages266-273
Number of pages8
ISBN (Electronic)9781728169040
DOIs
Publication statusPublished - Aug 2020
Externally publishedYes
Event16th IEEE International Conference on Automation Science and Engineering, CASE 2020 - Hong Kong, Hong Kong
Duration: 20 Aug 202021 Aug 2020

Publication series

NameIEEE International Conference on Automation Science and Engineering
Volume2020-August
ISSN (Print)2161-8070
ISSN (Electronic)2161-8089

Conference

Conference16th IEEE International Conference on Automation Science and Engineering, CASE 2020
Country/TerritoryHong Kong
CityHong Kong
Period20/08/2021/08/20

Keywords

  • adaptive traffic signal control
  • multi-agent reinforcement learning
  • policy fingerprints
  • reinforcement learning

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this