A New Era in Human Factors Engineering: A Survey of the Applications and Prospects of Large Multimodal Models

In recent years, the potential applications of Large Multimodal Models (LMMs) in fields, such as healthcare, social psychology, and industrial design have attracted wide research attention, providing new directions for human factors research. For instance, LMM-based smart systems have become novel research subjects of human factors studies, and LMM introduces new research paradigms and methodologies to this field. Therefore, this article aims to explore the applications, challenges, and future prospects of LMM in the domain of human factors and ergonomics through an expert-LMM collaborated literature and patent review. Specifically, this article proposes a novel review method and introduces research studies and patents related to LMM-based accident analysis, human modeling, and intervention design. Subsequently, the article discusses future trends in research paradigm and challenges of human factors and ergonomics studies in the era of LMMs. It is expected that the review results offer valuable insights and serve as a reference for using LMMs in human factors research.