A visual reasoning-based approach for driving experience improvement in the AR-assisted head-up displays

Yongshi Liang, Pai Zheng, Liqiao Xia

Research output: Journal article publicationJournal articleAcademic researchpeer-review

12 Citations (Scopus)


Enabled by advanced data analytics and intelligent computing, augmented reality head-up displays (AR-HUDs) are appraised with a certain degree of intelligence towards an in-car assistance system providing more convenience for drivers and ensuring safer traffic. Nevertheless, current AR-HUDs systems fail to analyze perceptual results with recommended driving strategies as the cognitive intelligence, while solely rely on driver's own decision-makings. To pave the way, this work stepwise proposes a visual reasoning-based approach for presenting drivers with perceptual, predictive, and reasoning information onto AR-HUDs toward cognitive intelligence. Firstly, a Driving Scenario Knowledge Graph comprising many road elements and empirical knowledge is established appropriately. Then, by analyzing the video streams and images collected by an in-car visual camera, the driving scene can be perceived comprehensively, including 1) identifying road elements and 2) moving elements’ intention recognition. Afterwards, a graph-based driving scenario reasoning model, driving scenario-adaptive KAGNET, is built for achieving driving strategy recommendations. Moreover, the analyzed information is shown on the HUDs via pre-defined AR graphics to support drivers intuitively. A case study is given lastly to prove its feasibility. As an explorative study, some limitations and future work are emphasized to attract further study and open discussion in this area for pursuing the better implementation of AR-HUDs.

Original languageEnglish
Article number101888
Number of pages9
JournalAdvanced Engineering Informatics
Publication statusPublished - Jan 2023


  • Augmented reality
  • Graph neural network
  • Head-up displayVisual reasoning
  • Smart traffic
  • Visual reasoning

ASJC Scopus subject areas

  • Information Systems
  • Building and Construction
  • Artificial Intelligence


Dive into the research topics of 'A visual reasoning-based approach for driving experience improvement in the AR-assisted head-up displays'. Together they form a unique fingerprint.

Cite this