Dual modal data-driven virtual reality-based mild cognitive Impairment assessment using MCIformer

Background Mild Cognitive Impairment (MCI) assessment is critical for identifying cognitive decline and enabling early intervention to reduce the risk of Alzheimer's disease (AD). Virtual Reality (VR)-based cognitive assessments offer enhanced engagement, ecological validity, and user-friendliness. However, most existing VR-based systems rely primarily on task performance metrics, overlooking subtle changes in motor behaviour and underlying neural activity patterns that are indicative of early cognitive decline. Methods We developed a dual-modal, data-driven VR-based MCI assessment method integrating kinematic and functional near-infrared spectroscopy (fNIRS) data. A VR-based experiment involving healthy and MCI participants was conducted to collect synchronised kinematic and neural data. Kinematic features—smoothness, coordination, and stability—were extracted from segmented movement trajectories and structured as time series to capture subtle motor deficits. Simultaneously, multichannel fNIRS data were represented as functional brain networks to assess interregional connectivity changes associated with cognitive impairment. We proposed MCIformer, a dual-modal fusion model that applies a Transformer to kinematic sequences to capture dynamic motor patterns and a Graph Transformer to fNIRS networks to detect connectivity alterations. Results Experimental results show that the proposed system achieved 90% accuracy, significantly outperforming models using only kinematic data (80%) or only fNIRS data (85%). The findings demonstrate that integrating temporal motor patterns and spatial brain connectivity patterns enhances classification performance by capturing complementary brain–behaviour information. Conclusions The proposed VR-based dual-modal MCI assessment approach demonstrates strong potential for scalable, accurate early diagnosis in community settings, supporting the development of more comprehensive brain–behaviour monitoring systems for cognitive health.