Enhancing Pilot Engagement and Situational Awareness with Augmented Reality: A Neuro-Ergonomic Evaluation of an Innovative AR-HUD

Introducing augmented reality (AR) technology into aviation cockpit design, especially headup display (HUD) design, will help innovate a new generation of human-machine interaction (HMI) paradigm. However, former research on AR-assisted HMI design provides a less objective assessment of human factors such as pilot situational awareness (SA) and workload. This research innovatively developed a low-cost, easy-to-assemble AR-HUD based on projection reflection technology. The projection-based AR-HUD can provide key information and necessary warnings and reminders to the pilot in different flight stages through projection. This study aims to evaluate the positive effects of the projection-based AR-HUD on flight tasks under low visual conditions with the help of multi-modal evaluation technologies, namely Functional Near-Infrared Spectroscopy (fNIRS), eye tracker, and Situation Awareness Rating Technique (SAGAT). At the same time, the characteristics of eye movement and brain activity during different flight phases were explored. The results show that AR-HUD benefits enhancing the pilot’s engagement in executing flight tasks, especially during the landing phase, thereby maintaining level-1 SA. Specifically, AR-HUD supports visuospatial processing when integrating synthetic cues. This AR-HUD design prototype explores a low-cost HMI paradigm that can achieve natural interaction and provides ideas for future HMI interaction in the cockpit.