A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning

Yixuan Lin; Kaiqing Zhang; Zhuoran Yang; Zhaoran Wang; Tamer Basar; Romeil Sandhu; Ji Liu

doi:10.1109/CDC40024.2019.9029257

A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning

Yixuan Lin
, Kaiqing Zhang
, Zhuoran Yang
, Zhaoran Wang
, Tamer Basar
, Romeil Sandhu
, Ji Liu

Electrical Engineering

Stony Brook University
University of Illinois at Urbana-Champaign
Princeton University
Northwestern University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

36 Scopus citations

Abstract

This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors. A randomized communication-efficient multi-agent actor-critic algorithm is proposed for possibly unidirectional communication relationships depicted by a directed graph. It is shown that the algorithm can solve the problem for strongly connected graphs by allowing each agent to transmit only two scalar-valued variables at one time.

Original language	English
Title of host publication	2019 IEEE 58th Conference on Decision and Control, CDC 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	5562-5567
Number of pages	6
ISBN (Electronic)	9781728113982
DOIs	https://doi.org/10.1109/CDC40024.2019.9029257
State	Published - Dec 2019
Event	58th IEEE Conference on Decision and Control, CDC 2019 - Nice, France Duration: Dec 11 2019 → Dec 13 2019

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
Volume	2019-December
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	58th IEEE Conference on Decision and Control, CDC 2019
Country/Territory	France
City	Nice
Period	12/11/19 → 12/13/19

Access to Document

10.1109/CDC40024.2019.9029257

Cite this

Lin, Y., Zhang, K., Yang, Z., Wang, Z., Basar, T., Sandhu, R., & Liu, J. (2019). A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning. In 2019 IEEE 58th Conference on Decision and Control, CDC 2019 (pp. 5562-5567). Article 9029257 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2019-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC40024.2019.9029257

@inproceedings{210a78bbf2784d80851c67557ab3df1c,

title = "A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning",

abstract = "This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors. A randomized communication-efficient multi-agent actor-critic algorithm is proposed for possibly unidirectional communication relationships depicted by a directed graph. It is shown that the algorithm can solve the problem for strongly connected graphs by allowing each agent to transmit only two scalar-valued variables at one time.",

author = "Yixuan Lin and Kaiqing Zhang and Zhuoran Yang and Zhaoran Wang and Tamer Basar and Romeil Sandhu and Ji Liu",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 58th IEEE Conference on Decision and Control, CDC 2019 ; Conference date: 11-12-2019 Through 13-12-2019",

year = "2019",

month = dec,

doi = "10.1109/CDC40024.2019.9029257",

language = "English",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "5562--5567",

booktitle = "2019 IEEE 58th Conference on Decision and Control, CDC 2019",

}

Lin, Y, Zhang, K, Yang, Z, Wang, Z, Basar, T, Sandhu, R & Liu, J 2019, A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning. in 2019 IEEE 58th Conference on Decision and Control, CDC 2019., 9029257, Proceedings of the IEEE Conference on Decision and Control, vol. 2019-December, Institute of Electrical and Electronics Engineers Inc., pp. 5562-5567, 58th IEEE Conference on Decision and Control, CDC 2019, Nice, France, 12/11/19. https://doi.org/10.1109/CDC40024.2019.9029257

A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning. / Lin, Yixuan; Zhang, Kaiqing; Yang, Zhuoran et al.
2019 IEEE 58th Conference on Decision and Control, CDC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 5562-5567 9029257 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2019-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning

AU - Lin, Yixuan

AU - Zhang, Kaiqing

AU - Yang, Zhuoran

AU - Wang, Zhaoran

AU - Basar, Tamer

AU - Sandhu, Romeil

AU - Liu, Ji

PY - 2019/12

Y1 - 2019/12

N2 - This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors. A randomized communication-efficient multi-agent actor-critic algorithm is proposed for possibly unidirectional communication relationships depicted by a directed graph. It is shown that the algorithm can solve the problem for strongly connected graphs by allowing each agent to transmit only two scalar-valued variables at one time.

AB - This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors. A randomized communication-efficient multi-agent actor-critic algorithm is proposed for possibly unidirectional communication relationships depicted by a directed graph. It is shown that the algorithm can solve the problem for strongly connected graphs by allowing each agent to transmit only two scalar-valued variables at one time.

UR - https://www.scopus.com/pages/publications/85082474927

U2 - 10.1109/CDC40024.2019.9029257

DO - 10.1109/CDC40024.2019.9029257

M3 - Conference contribution

AN - SCOPUS:85082474927

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 5562

EP - 5567

BT - 2019 IEEE 58th Conference on Decision and Control, CDC 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 58th IEEE Conference on Decision and Control, CDC 2019

Y2 - 11 December 2019 through 13 December 2019

ER -

Lin Y, Zhang K, Yang Z, Wang Z, Basar T, Sandhu R et al. A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning. In 2019 IEEE 58th Conference on Decision and Control, CDC 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 5562-5567. 9029257. (Proceedings of the IEEE Conference on Decision and Control). doi: 10.1109/CDC40024.2019.9029257

A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this