Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control

Yifan Zhao; Gangyan Xu; Yali Duy; Meng Fangz

doi:10.1109/CASE48305.2020.9216981

Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control

Yifan Zhao
, Gangyan Xu
, Yali Duy
, Meng Fangz

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

5 Citations (Scopus)

Abstract

Adaptive traffic signal control is widely recognized as an effective solution to improve urban mobility and reduce congestion in metropolises. Recently, reinforcement learning has been adopted for this transportation problem. While centralized reinforcement learning inevitably faces action space explosion, decentralized reinforcement learning allows agents to develop policies based on local observations but suffers from unstable training. In this paper, we present CommNetPF, a multi-agent decentralized reinforcement learning model incorporating communication and neighbourhood policy fingerprints for adaptive traffic signal control. With policy fingerprints in communication, agents learn to produce cooperative policies and the model converges faster. Experiments in scenarios of adaptive traffic signal control show that CommNetPF outperforms several strong baselines in terms of control performance and convergence speed.

Original language	English
Title of host publication	2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020
Publisher	IEEE Computer Society
Pages	266-273
Number of pages	8
ISBN (Electronic)	9781728169040
DOIs	https://doi.org/10.1109/CASE48305.2020.9216981
Publication status	Published - Aug 2020
Externally published	Yes
Event	16th IEEE International Conference on Automation Science and Engineering, CASE 2020 - Hong Kong, Hong Kong Duration: 20 Aug 2020 → 21 Aug 2020

Publication series

Name	IEEE International Conference on Automation Science and Engineering
Volume	2020-August
ISSN (Print)	2161-8070
ISSN (Electronic)	2161-8089

Conference

Conference	16th IEEE International Conference on Automation Science and Engineering, CASE 2020
Country/Territory	Hong Kong
City	Hong Kong
Period	20/08/20 → 21/08/20

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 11 Sustainable Cities and Communities

Keywords

adaptive traffic signal control
multi-agent reinforcement learning
policy fingerprints
reinforcement learning

ASJC Scopus subject areas

Control and Systems Engineering
Electrical and Electronic Engineering

More information

10.1109/CASE48305.2020.9216981

Cite this

Zhao, Y., Xu, G., Duy, Y., & Fangz, M. (2020). Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control. In 2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020 (pp. 266-273). Article 9216981 (IEEE International Conference on Automation Science and Engineering; Vol. 2020-August). IEEE Computer Society. https://doi.org/10.1109/CASE48305.2020.9216981

@inproceedings{c52ad9b433fd4af8888b74f2d2d6d5e2,

title = "Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control",

abstract = "Adaptive traffic signal control is widely recognized as an effective solution to improve urban mobility and reduce congestion in metropolises. Recently, reinforcement learning has been adopted for this transportation problem. While centralized reinforcement learning inevitably faces action space explosion, decentralized reinforcement learning allows agents to develop policies based on local observations but suffers from unstable training. In this paper, we present CommNetPF, a multi-agent decentralized reinforcement learning model incorporating communication and neighbourhood policy fingerprints for adaptive traffic signal control. With policy fingerprints in communication, agents learn to produce cooperative policies and the model converges faster. Experiments in scenarios of adaptive traffic signal control show that CommNetPF outperforms several strong baselines in terms of control performance and convergence speed.",

keywords = "adaptive traffic signal control, multi-agent reinforcement learning, policy fingerprints, reinforcement learning",

author = "Yifan Zhao and Gangyan Xu and Yali Duy and Meng Fangz",

note = "Funding Information: This work was supported in part by the CCF-Tencent Open Research Fund, the National Natural Science Foundation of China under Grant 71804034, and the Research Foundation of STIC under Grant JCYJ20180306171958907 Publisher Copyright: {\textcopyright} 2020 IEEE.; 16th IEEE International Conference on Automation Science and Engineering, CASE 2020 ; Conference date: 20-08-2020 Through 21-08-2020",

year = "2020",

month = aug,

doi = "10.1109/CASE48305.2020.9216981",

language = "English",

series = "IEEE International Conference on Automation Science and Engineering",

publisher = "IEEE Computer Society",

pages = "266--273",

booktitle = "2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020",

address = "United States",

}

Zhao, Y, Xu, G, Duy, Y & Fangz, M 2020, Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control. in 2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020., 9216981, IEEE International Conference on Automation Science and Engineering, vol. 2020-August, IEEE Computer Society, pp. 266-273, 16th IEEE International Conference on Automation Science and Engineering, CASE 2020, Hong Kong, Hong Kong, 20/08/20. https://doi.org/10.1109/CASE48305.2020.9216981

Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control. / Zhao, Yifan; Xu, Gangyan; Duy, Yali et al.
2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020. IEEE Computer Society, 2020. p. 266-273 9216981 (IEEE International Conference on Automation Science and Engineering; Vol. 2020-August).

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control

AU - Zhao, Yifan

AU - Xu, Gangyan

AU - Duy, Yali

AU - Fangz, Meng

N1 - Funding Information: This work was supported in part by the CCF-Tencent Open Research Fund, the National Natural Science Foundation of China under Grant 71804034, and the Research Foundation of STIC under Grant JCYJ20180306171958907 Publisher Copyright: © 2020 IEEE.

PY - 2020/8

Y1 - 2020/8

N2 - Adaptive traffic signal control is widely recognized as an effective solution to improve urban mobility and reduce congestion in metropolises. Recently, reinforcement learning has been adopted for this transportation problem. While centralized reinforcement learning inevitably faces action space explosion, decentralized reinforcement learning allows agents to develop policies based on local observations but suffers from unstable training. In this paper, we present CommNetPF, a multi-agent decentralized reinforcement learning model incorporating communication and neighbourhood policy fingerprints for adaptive traffic signal control. With policy fingerprints in communication, agents learn to produce cooperative policies and the model converges faster. Experiments in scenarios of adaptive traffic signal control show that CommNetPF outperforms several strong baselines in terms of control performance and convergence speed.

AB - Adaptive traffic signal control is widely recognized as an effective solution to improve urban mobility and reduce congestion in metropolises. Recently, reinforcement learning has been adopted for this transportation problem. While centralized reinforcement learning inevitably faces action space explosion, decentralized reinforcement learning allows agents to develop policies based on local observations but suffers from unstable training. In this paper, we present CommNetPF, a multi-agent decentralized reinforcement learning model incorporating communication and neighbourhood policy fingerprints for adaptive traffic signal control. With policy fingerprints in communication, agents learn to produce cooperative policies and the model converges faster. Experiments in scenarios of adaptive traffic signal control show that CommNetPF outperforms several strong baselines in terms of control performance and convergence speed.

KW - adaptive traffic signal control

KW - multi-agent reinforcement learning

KW - policy fingerprints

KW - reinforcement learning

UR - https://www.scopus.com/pages/publications/85094143411

U2 - 10.1109/CASE48305.2020.9216981

DO - 10.1109/CASE48305.2020.9216981

M3 - Conference article published in proceeding or book

AN - SCOPUS:85094143411

T3 - IEEE International Conference on Automation Science and Engineering

SP - 266

EP - 273

BT - 2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020

PB - IEEE Computer Society

T2 - 16th IEEE International Conference on Automation Science and Engineering, CASE 2020

Y2 - 20 August 2020 through 21 August 2020

ER -

Zhao Y, Xu G, Duy Y, Fangz M. Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control. In 2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020. IEEE Computer Society. 2020. p. 266-273. 9216981. (IEEE International Conference on Automation Science and Engineering). doi: 10.1109/CASE48305.2020.9216981

Learning Multi-Agent Communication with Policy Fingerprints for Adaptive Traffic Signal Control

Abstract

Publication series

Conference

UN SDGs

Keywords

ASJC Scopus subject areas

More information

Other files and links

Fingerprint

Cite this