Stochastic optimal CPS control for interconnected power grids using multi-step backtrack Q (λ) learning

This paper presents the application of multi-step backtrack Q (λ) learning based on stochastic optimal control to effectively solve the long time-delay link for thermal plants under Non-Markovian environment. The moving averages of CPS1/CPS2 are used as the state input, and the CPS control and relaxed control objectives are formulated as MDP reward function by means of linear weighted aggregative approach. The optimal CPS control methodology open avenues to on-line feedback learning rule to maximize the long-run discounted reward. Statistic experiments show that the Q (λ) controllers can enhance obviously the robustness and dynamic performance of AGC systems, and reduce the number of pulses and pulse reversals while the CPS compliances are ensured. The proposed strategy also provides a convenient means for controlling the degree of compliance and relaxation by online tune relaxation factors to implement the desirable CPS relaxed control.