Safe Reinforcement Learning for Event-Triggered Control of Automated Vehicles With Uncertainty

Safety is of vital importance to transportation cyber-physical systems (T-CPS). With the deeper integration of automated vehicles (AVs) in road traffic, safe control algorithms for AVs have become a crucial demand for maintaining the overall safety of T-CPS. One of the major concerns with respect to the DRL-based cyber systems of AVs is the lack of safety guarantees, which may cause physical collisions. Safe reinforcement learning (SRL) is a promising way to enhance the safety performance of traditional DRL. However, current SRL methods are designed in a time-triggered (TT) paradigm, where the controller is uniformly triggered in the time domain. Despite the safety enhancement, this TT-SRL paradigm lacks data efficiency in safety-critical scenarios, and most importantly, still cannot guarantee safety during the intervals between time steps. The main objective of this paper is to address the safety issue of the existing TT-SRL methods while improving the data efficiency by proposing an event-triggered SRL (ET-SRL) framework. In the proposed method, safety is guaranteed by designing an additional safe controller to correct the unsafe actions generated by DRL. Event-triggered control barrier functions (CBFs) are used to provide safety constraints in a discrete manner. A state and dynamics synchronization module is also designed to estimate model uncertainties considering both the state dynamics and the input dynamics. The proposed method is trained and applied in simulated car-following experiments. The effectiveness of the proposed method is demonstrated by comparison to both DRL-based and model-based baselines in both artificial safety-critical scenarios and a real trajectory dataset.