Abstract
The integration of urban rail transit (URT) traction networks (TNs) with hybrid energy storage systems (HESSs) has become technologically and socioeconomically crucial to enabling highly efficient and convenient mass public transportation within urban areas while promoting the carbon-neutral transformation of URTs. The spatial-temporal uncertainties and complexities arising from passenger demand, urban traffic congestion, widespread distribution, operational disturbances, etc. have imposed significant challenges and limitations on the stable, efficient, sustainable, and intelligent operations of the HESS-integrated URT TNs, especially for those involving distributed HESSs (DHESSs).This thesis reports using reinforcement learning (RL) as a machine-learning base technique to develop three different levels of energy management and configuration strategies for HESS-integrated URT TNs. These include: (1) a supervised RL-based energy-efficient train trajectory optimization (SRL–EETTO) approach for automatic train operation at the 1st (train) level, (2) a multi-task RL-based sizing and control optimization (MTRL–SCO) approach for HESS-integrated traction substation operation at the 2nd (substation) level, (3) a multi-task multi-agent RL-based multi-time scale energy management (MTMARL–MTSEM) approach for DHESS-integrated TN operation at the 3rd (network) level, and (4) a multi-task multi-agent RL-based data driven multi-objective configuration optimization (MTMARL–DDMOCO) approach for furthering the DHESS-integrated TN operation at the 3rd (network) level. The research background, problem formulation, approach establishment, and case study verifications are also described for each energy management and configuration strategy.
At the 1st (train) level, the proposed SRL–EETTO approach is aimed to expand the capability of automatic train operation systems in addressing the real-time responsiveness and dynamic online challenges to state-of-the-art TTO approaches and their associated safety, punctuality, and ride comfort issues. A real-time train control model under uncertain disturbances is formulated as a Markov decision process, and a supervised twin-delayed deep deterministic policy gradient algorithm with improved effectiveness is developed to solve the real-time train control model. Satisfactory performances on reduced traction energy use and multiple evaluation indices are verified for the proposed SRL–EETTO approach, and the optimal configuration of the train trajectory set is investigated.
At the 2nd (substation) level, the proposed MTRL–SCO approach is intended to enhance the coordinated operations of the supercapacitor–battery HESSs and their integrated traction substations under dynamic spatial-temporal URT traffic. A dynamic traffic model is devised to characterize the multi-train traction load uncertainty induced by passenger flow fluctuations, real-time traffic regulations, and train parameters. An MTRL algorithm based on a dueling double deep Q network with knowledge transfer is presented to learn a generalized HESS control policy adapting to multiple train service patterns by leveraging a shareable cross-task experience. Simulations have validated the superior computational performance, sizing decisions, and control behaviors of the proposed MTRL–SCO approach.
At the 3rd (network) level, the proposed MTMARL–MTSEM approach strives for the economic and low-carbon operation of TNs integrating with photovoltaic–regenerative braking (PV–RB) DHESSs. A two-stage stochastic scheduling is performed on a long-time scale to minimize daily operation and carbon trading costs at the upper level and correct day-ahead scheduling deviations against multi-source uncertainties at the middle level. A real-time energy management algorithm based on MTMARL is established to optimize PV–RB power flow and promote utilization through decentralized coordination of DHESSs at the lower level. Representative daily TN operation scenarios are selected to demonstrate the improved economic and low-carbon benefits and PV–RB energy utilization of the proposed MTMARL–MTSEM approach.
Furthering the 3rd (network) level, the proposed MTMARL–DDMOCO approach is focused on promoting an optimal synergy between the economic and energy efficiencies of the DHESS-integrated TN operation and the travel time of the passengers. A multi-objective configuration optimization model considering the electrothermal aging of batteries is formulated to optimize DHESS capacities and train operation parameters based on the developed MTMARL–MTSEM approach. The non-dominated sorting genetic algorithm is incorporated with ensemble learning-based load prediction models to solve the multi-objective configuration optimization model in a data-driven manner. The configuration decisions of the proposed MTMARL–DDMOCO approach are analyzed thoroughly.
| Date of Award | 21 May 2025 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Sponsors | Research Grants Council (RGC) & Innovation and Technology Commission |
| Supervisor | Siu Wing Or (Chief supervisor) |