Optimal Bi-Level Bidding and Dispatching Strategy between Active Distribution Network and Virtual Alliances Using Distributed Robust Multi-Agent Deep Reinforcement Learning

Ziqing Zhu, Ka Wing Chan, Shiwei Xia, Siqi Bu

Research output: Journal article publicationJournal articleAcademic researchpeer-review


The deregulated active distribution network (ADN) would incorporate numerous autonomous stakeholders, including some emerging distributed virtual alliances (DVAs) like virtual microgrids and virtual power plants. Those DVAs would autonomously participate in energy market trading through bidding among themselves and dispatching conducted by the ADN. In this paper, the optimal bidding and dispatching model for DVAs and ADN in the day-ahead market is first developed as a stochastic dynamic programming model with the risk of misconduct considered, and then re-formulated as a Markov Decision Process to be solved by a new Distributed Robust Multi-Agent Deep Deterministic Policy Gradient algorithm based on the concept of robust Nash equilibrium (RNE). This algorithm is a fully distributed online optimization that would converge to RNE. It is an effective risk-averse method to obtain the optimal bidding strategies of DVAs and the optimal dispatching decisions of distribution system operator (DSO). Its high computational performance is demonstrated in the case studies, and the strategic decisions of DVAs and DSO are thoroughly analyzed.

Original languageEnglish
Pages (from-to)2833-2843
Number of pages11
JournalIEEE Transactions on Smart Grid
Issue number4
Publication statusPublished - 1 Jul 2022


  • active distribution network
  • bidding strategy
  • Costs
  • Dispatching
  • distributed virtual alliance
  • Heuristic algorithms
  • Indexes
  • Load modeling
  • Real-time systems
  • reinforcement learning.
  • Uncertainty

ASJC Scopus subject areas

  • Computer Science(all)

Cite this