Integrated optimal control strategies for freeway traffic mixed with connected automated vehicles: A model-based reinforcement learning approach

Tianlu Pan, Renzhong Guo, William H.K. Lam, Renxin Zhong, Weixi Wang, Biao He

Research output: Journal article publicationJournal articleAcademic researchpeer-review

38 Citations (Scopus)


This paper proposes an integrated freeway traffic flow control framework that aims to minimize the total travel cost, improve greenness and safety for freeway traffic mixed with connected automated vehicles (CAVs) and regular human-piloted vehicles (RHVs). The proposed framework devises an integrated action of several control strategies such as ramp metering, lane changing control (LCC) for CAVs and lane changing recommendation (LCR) for RHVs, variable speed limit control (VSLC) for CAVs and variable speed limit recommendation (VSLR) for RHVs with minimum safety gap control measures for lane changing and merging maneuvers. The CAVs are assumed to follow the system control instructions fully and immediately. In contrast, the RHVs would make decisions in response to the recommendations disseminated and also the behaviors of CAVs. The compliance rate of drivers to the LCR is captured by the underlying traffic flow model. A set of constraints is imposed to restrict VSLC/VSLR and LCC/LCR measures from changing too frequently or too sharply on both temporal and spatial dimensions to avoid excessive nuisance to passengers and traffic flow instability. A reinforcement learning based solution algorithm is proposed. First, a control parameterization technique is adopted to reduce the dimension of the original optimal control problem to increase computational efficiency. Then, a gradient-free Cross-Entropy-Method based algorithm is used to search the optimal parameters to circumvent the non-differentiability of the traffic flow model. The feasibility and effectiveness of the proposed framework are illustrated via numerical examples for a variety of penetration rates of CAVs under various traffic conditions. A sensitivity analysis is conducted to demonstrate the impacts of several important parameters such as the reaction time of the CAVs. It is found that the integrated control strategy can reduce the total travel cost by reducing the lane changing maneuvers and vehicles queuing at the bottleneck meanwhile smooth the traffic flow and suppress the adverse impact of shockwaves. The effect of ramp metering is not significant when the penetration rate of CAVs is high enough. Speed harmonization (with minimum gap control) in conjunction with LCC/LCR would be a better integrated control strategy under high penetration rate of CAVs.

Original languageEnglish
Article number102987
JournalTransportation Research Part C: Emerging Technologies
Publication statusPublished - Feb 2021


  • Connected automated vehicle
  • Integrated traffic control
  • Multiclass multilane cell transmission model
  • Penetration rate
  • Vehicle automation and communication system

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Automotive Engineering
  • Transportation
  • Computer Science Applications


Dive into the research topics of 'Integrated optimal control strategies for freeway traffic mixed with connected automated vehicles: A model-based reinforcement learning approach'. Together they form a unique fingerprint.

Cite this