Q-learning approach for hierarchical AGC scheme of interconnected power grids

This paper formulates automatic generation control (AGC) for the power dispatch center as a two-layer hierarchical control framework, which can be divided into two discrete-time Markov decision process (DTMDP) sub-problems. The first one focuses on the solution of optimum AGC regulating commands under control performance standards, while the second works on the dynamic optimization allocation of the commands to various types of AGC units. The model-free Q-learning and multicriteria reward function are proposed and designed specifically for two DTMDP subproblems, respectively. The proposed methodology can enhance the overall performance of the hierarchical AGC scheme from the viewpoint of long-term optimal objective. The effectiveness and efficiency of the AGC scheme are fully studied via simulation tests on a two-area interconnected hydro-thermal power system model, and test results are benchmarked against another heuristic algorithm and practical engineering approaches.