TY - JOUR
T1 - Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning
AU - Zhu, Ziqing
AU - Chan, Ka Wing
AU - Bu, Siqi
AU - Or, Siu Wing
AU - Gao, Xiang
AU - Xia, Shiwei
N1 - Funding Information:
This work was supported in part by the Research Grants Council of the HKSAR Government under Grant R5020-18, in part by the Innovation and Technology Commission of the HKSAR Government to the Hong Kong Branch of National Rail Transit Electrification and Automation Engineering Technology Research Center under Grant K-BBY1, and in part by the National Natural Science Foundation of China under Grant 52077075.
Publisher Copyright:
© 1969-2012 IEEE.
PY - 2021/11/1
Y1 - 2021/11/1
N2 - In this letter, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to explicitly determine the factors affecting energy providers (EPs) willingness of using the market power to uplift the price in the bidding procedure, which could be simulated using the win-or-learn-fast policy hill climbing (WoLF-PHC) algorithm as a multi-agent reinforcement learning (MARL) method. Firstly, empirical and numerical connections between WoLF-PHC and RDEs is proved. Then, by formulating RDEs of the bidding procedure, three factors affecting the bidding strategy preference are revealed, including the load demand, severity of congestion, and the price cap. Finally, the impact of these factors on the converged bidding price is demonstrated in case studies, by simulating the bidding procedure driven by WoLF-PHC.
AB - In this letter, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to explicitly determine the factors affecting energy providers (EPs) willingness of using the market power to uplift the price in the bidding procedure, which could be simulated using the win-or-learn-fast policy hill climbing (WoLF-PHC) algorithm as a multi-agent reinforcement learning (MARL) method. Firstly, empirical and numerical connections between WoLF-PHC and RDEs is proved. Then, by formulating RDEs of the bidding procedure, three factors affecting the bidding strategy preference are revealed, including the load demand, severity of congestion, and the price cap. Finally, the impact of these factors on the converged bidding price is demonstrated in case studies, by simulating the bidding procedure driven by WoLF-PHC.
KW - bidding strategy
KW - evolutionary game theory
KW - Games
KW - Generators
KW - Heuristic algorithms
KW - market power
KW - multi-agent reinforcement learning
KW - Power system dynamics
KW - Reinforcement learning
KW - Stability criteria
KW - Stakeholders
KW - Market power
UR - http://www.scopus.com/inward/record.url?scp=85111609440&partnerID=8YFLogxK
U2 - 10.1109/TPWRS.2021.3099693
DO - 10.1109/TPWRS.2021.3099693
M3 - Journal article
AN - SCOPUS:85111609440
SN - 0885-8950
VL - 36
SP - 5975
EP - 5978
JO - IEEE Transactions on Power Systems
JF - IEEE Transactions on Power Systems
IS - 6
ER -