DRL-Based Adaptive Energy Management for Hybrid Electric Storage Systems Under Dynamic Spatial-Temporal Traffic in Urban Rail Transits

Hybrid electric storage systems (HESSs) of stationary batteries and supercapacitors have received increasing attention to the reduction of power/energy redundancy from their single-form counterparts. The real-time traffic in fast-expanding urban rail transit networks (URTNs) brings considerable challenges of time-varying, high-power, and large-scale regenerative braking energy recovery to the HESS energy management. This paper proposes an adaptive energy management method for effectively operating HESSs in URTNs by considering complex and dynamic spatial-temporal traffic characteristics and utilizing deep reinforcement learning. First, a Copula-based multi-dimensional spatial-temporal passenger flow correlation model is developed to generate dynamic multi-train operation scenarios. Second, the HESS energy management problem is formulated as a Markov decision process, where a dueling double deep Q-network algorithm is used to optimize a coordinated energy management strategy composed of voltage threshold adjustments and power distributions between batteries and supercapacitors for minimizing the URTN operation cost. Third, comparative studies are analyzed to verify the performance of the proposed method using real-world subway line data. The results demonstrate the superior performance of the proposed method in improving the energy-saving effect and economic benefits of the HESSs.