Q-learning approach for hierarchical AGC scheme of interconnected power grids

B. Zhou, Ka Wing Chan, T. Yu

Research output: Journal article publicationConference articleAcademic researchpeer-review

8 Citations (Scopus)


This paper formulates automatic generation control (AGC) for the power dispatch center as a two-layer hierarchical control framework, which can be divided into two discrete-time Markov decision process (DTMDP) sub-problems. The first one focuses on the solution of optimum AGC regulating commands under control performance standards, while the second works on the dynamic optimization allocation of the commands to various types of AGC units. The model-free Q-learning and multicriteria reward function are proposed and designed specifically for two DTMDP subproblems, respectively. The proposed methodology can enhance the overall performance of the hierarchical AGC scheme from the viewpoint of long-term optimal objective. The effectiveness and efficiency of the AGC scheme are fully studied via simulation tests on a two-area interconnected hydro-thermal power system model, and test results are benchmarked against another heuristic algorithm and practical engineering approaches.
Original languageEnglish
Pages (from-to)43-52
Number of pages10
JournalEnergy Procedia
Publication statusPublished - 1 Jan 2011
Event1st International Conference on Smart Grid and Clean Energy Technologies, ICSGCE 2011 - Chengdu, China
Duration: 27 Sept 201130 Sept 2011


  • AGC
  • CPS
  • Dynamic generation allocation
  • Hierarchical control
  • Q-learning
  • Stochastic optimization

ASJC Scopus subject areas

  • General Energy


Dive into the research topics of 'Q-learning approach for hierarchical AGC scheme of interconnected power grids'. Together they form a unique fingerprint.

Cite this