Stochastic optimal CPS control for interconnected power grids using multi-step backtrack Q (λ) learning

Tao Yu, Bin Zhou, Ka Wing Chan

Research output: Journal article publicationJournal articleAcademic researchpeer-review

13 Citations (Scopus)


This paper presents the application of multi-step backtrack Q (λ) learning based on stochastic optimal control to effectively solve the long time-delay link for thermal plants under Non-Markovian environment. The moving averages of CPS1/CPS2 are used as the state input, and the CPS control and relaxed control objectives are formulated as MDP reward function by means of linear weighted aggregative approach. The optimal CPS control methodology open avenues to on-line feedback learning rule to maximize the long-run discounted reward. Statistic experiments show that the Q (λ) controllers can enhance obviously the robustness and dynamic performance of AGC systems, and reduce the number of pulses and pulse reversals while the CPS compliances are ensured. The proposed strategy also provides a convenient means for controlling the degree of compliance and relaxation by online tune relaxation factors to implement the desirable CPS relaxed control.
Original languageChinese (Simplified)
Pages (from-to)179-186
Number of pages8
JournalDiangong Jishu Xuebao/Transactions of China Electrotechnical Society
Issue number6
Publication statusPublished - 1 Jun 2011


  • Automatic generation control
  • Control performance standard (CPS)
  • Multi-step Q (λ) learning
  • Non-Markovian environment
  • Stochastic optimal control

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this