Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

Ziqing Zhu; Ka Wing Chan; Siqi Bu; Siu Wing Or; Xiang Gao; Shiwei Xia

doi:10.1109/TPWRS.2021.3099693

Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

Ziqing Zhu, Ka Wing Chan, Siqi Bu, Siu Wing Or, Xiang Gao, Shiwei Xia

Research output: Journal article publication › Journal article › Academic research › peer-review

Abstract

In this letter, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to explicitly determine the factors affecting energy providers (EPs) willingness of using the market power to uplift the price in the bidding procedure, which could be simulated using the win-or-learn-fast policy hill climbing (WoLF-PHC) algorithm as a multi-agent reinforcement learning (MARL) method. Firstly, empirical and numerical connections between WoLF-PHC and RDEs is proved. Then, by formulating RDEs of the bidding procedure, three factors affecting the bidding strategy preference are revealed, including the load demand, severity of congestion, and the price cap. Finally, the impact of these factors on the converged bidding price is demonstrated in case studies, by simulating the bidding procedure driven by WoLF-PHC.

Original language	English
Journal	IEEE Transactions on Power Systems
DOIs	https://doi.org/10.1109/TPWRS.2021.3099693
Publication status	Accepted/In press - 2021

Keywords

bidding strategy
evolutionary game theory
Games
Generators
Heuristic algorithms
market power
multi-agent reinforcement learning
Power system dynamics
Reinforcement learning
Stability criteria
Stakeholders

ASJC Scopus subject areas

Energy Engineering and Power Technology
Electrical and Electronic Engineering

More information

10.1109/TPWRS.2021.3099693

Cite this

@article{f25a90f6f454456cb06332a1874c5930,

title = "Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning",

abstract = "In this letter, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to explicitly determine the factors affecting energy providers (EPs) willingness of using the market power to uplift the price in the bidding procedure, which could be simulated using the win-or-learn-fast policy hill climbing (WoLF-PHC) algorithm as a multi-agent reinforcement learning (MARL) method. Firstly, empirical and numerical connections between WoLF-PHC and RDEs is proved. Then, by formulating RDEs of the bidding procedure, three factors affecting the bidding strategy preference are revealed, including the load demand, severity of congestion, and the price cap. Finally, the impact of these factors on the converged bidding price is demonstrated in case studies, by simulating the bidding procedure driven by WoLF-PHC.",

keywords = "bidding strategy, evolutionary game theory, Games, Generators, Heuristic algorithms, market power, multi-agent reinforcement learning, Power system dynamics, Reinforcement learning, Stability criteria, Stakeholders",

author = "Ziqing Zhu and Chan, {Ka Wing} and Siqi Bu and Or, {Siu Wing} and Xiang Gao and Shiwei Xia",

note = "Publisher Copyright: IEEE",

year = "2021",

doi = "10.1109/TPWRS.2021.3099693",

language = "English",

journal = "IEEE Transactions on Power Systems",

issn = "0885-8950",

publisher = "IEEE",

}

TY - JOUR

T1 - Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

AU - Zhu, Ziqing

AU - Chan, Ka Wing

AU - Bu, Siqi

AU - Or, Siu Wing

AU - Gao, Xiang

AU - Xia, Shiwei

N1 - Publisher Copyright: IEEE

PY - 2021

Y1 - 2021

N2 - In this letter, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to explicitly determine the factors affecting energy providers (EPs) willingness of using the market power to uplift the price in the bidding procedure, which could be simulated using the win-or-learn-fast policy hill climbing (WoLF-PHC) algorithm as a multi-agent reinforcement learning (MARL) method. Firstly, empirical and numerical connections between WoLF-PHC and RDEs is proved. Then, by formulating RDEs of the bidding procedure, three factors affecting the bidding strategy preference are revealed, including the load demand, severity of congestion, and the price cap. Finally, the impact of these factors on the converged bidding price is demonstrated in case studies, by simulating the bidding procedure driven by WoLF-PHC.

AB - In this letter, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to explicitly determine the factors affecting energy providers (EPs) willingness of using the market power to uplift the price in the bidding procedure, which could be simulated using the win-or-learn-fast policy hill climbing (WoLF-PHC) algorithm as a multi-agent reinforcement learning (MARL) method. Firstly, empirical and numerical connections between WoLF-PHC and RDEs is proved. Then, by formulating RDEs of the bidding procedure, three factors affecting the bidding strategy preference are revealed, including the load demand, severity of congestion, and the price cap. Finally, the impact of these factors on the converged bidding price is demonstrated in case studies, by simulating the bidding procedure driven by WoLF-PHC.

KW - bidding strategy

KW - evolutionary game theory

KW - Games

KW - Generators

KW - Heuristic algorithms

KW - market power

KW - multi-agent reinforcement learning

KW - Power system dynamics

KW - Reinforcement learning

KW - Stability criteria

KW - Stakeholders

UR - http://www.scopus.com/inward/record.url?scp=85111609440&partnerID=8YFLogxK

U2 - 10.1109/TPWRS.2021.3099693

DO - 10.1109/TPWRS.2021.3099693

M3 - Journal article

AN - SCOPUS:85111609440

JO - IEEE Transactions on Power Systems

JF - IEEE Transactions on Power Systems

SN - 0885-8950

ER -

Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

Abstract

Keywords

ASJC Scopus subject areas

More information

Other files and links

Cite this