Multi-Agent Correlated Equilibrium Q(λ) Learning for Coordinated Smart Generation Control of Interconnected Power Grids

T. Yu, H. Z. Wang, B. Zhou, Ka Wing Chan, J. Tang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

60 Citations (Scopus)


This paper proposes an optimal coordinated control methodology based on the multi-agent reinforcement learning (MARL) for the multi-area smart generation control (SGC) under the control performance standards (CPS). A new MARL algorithm called correlated Q(λ) learning (CEQ(λ)) is presented to form an optimal joint equilibrium strategy for the coordinated load frequency control of interconnected control areas, and a SGC framework is proposed to facilitate information sharing and strategic interaction among multi-areas so as to enhance the overall long-run performance of the control areas. Furthermore, a novel time-varying equilibrium factor is introduced into the equilibrium selection function to identify the optimum equilibrium policies in various power system operation scenarios. The performance of CEQ(λ) based SGC strategy has been fully tested and benchmarked on a two-area power system and the China Southern Power Grid. Comparative studies have not only demonstrated the superior equilibrium optimization and dynamic performance of the proposed SGC strategy but also confirmed its fast convergence and high flexibility in designing the equilibrium factor for the desirable operating state of correlated equilibria.
Original languageEnglish
Article number6913586
Pages (from-to)1669-1679
Number of pages11
JournalIEEE Transactions on Power Systems
Issue number4
Publication statusPublished - 1 Jul 2015


  • Coordinated control
  • correlated equilibrium
  • CPS
  • multi-agent system
  • Q(λ) learning
  • SGC

ASJC Scopus subject areas

  • Energy Engineering and Power Technology
  • Electrical and Electronic Engineering

Cite this