Enhancing Driving Representation with Vector Quantized Encoding for Behavior Planning in Autonomous Driving

Behavior planning for autonomous vehicles (AVs) poses a significant challenge, largely due to the complexities of extracting comprehensive environmental information through neural network-based planning agents. In contrast to the traditional approach of employing larger-scale neural networks to enhance behavior planning, this paper introduces an efficient representation enhancement scheme utilizing a vector quantizer (VQ) mechanism. We integrate the architecture of the VQ-variational autoencoder with reinforcement learning (RL)-based planning strategies to develop a novel VQ- RL method. This method involves mapping the output of the state encoder to a finite set of discrete embedding vectors that could more effectively represent the environment. These vectors serve as inputs to the decoder, which generates tactical behavior actions based on these latent abstract representations. Additionally, we have developed a dual-objective learning method that updates both the optimal value objective for RL and the VQ-based environmental representation objective simultaneously. The effectiveness of the proposed VQ-RL method is demonstrated through its application to the lane-changing task for AVs. Comparisons with state-of-the-art standard RL methods show that our strategy, enhanced by VQ-based representation, leads to a deeper understanding of the autonomous driving environment and significantly improves behavior planning performance and adaptability.