Nonintrusive multimodal attention detection

Hugo Jiawei Sun, Michael Xuelin Huang, Grace Ngai, Stephen Chi Fai Chan

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

8 Citations (Scopus)


With the increasing deployment of computers in a wide variety of applications, the ability to detect the user's attention, or engagement, is becoming more important as a key piece of contextual information in building effective interactive systems. For instance, one can imagine that a system that is aware of whether the user is attending to it would be able to adapt itself better to the user activities to enhance productivity. The ability to detect attention would also be useful for system analysis in designing and building better systems. However, much previous work in attention detection is either obtrusive or imposes demanding constraints on the context and the participants. In addition, most approaches rely on uni-modal signals, which are often limited in availability and stability. This paper attempts to address these two major limitations through a noninvasive multimodal solution, which allows participants to work naturally without interference. The solution makes use of common off-the-shelf items that could reasonably be expected of any computing environment and does not rely on expensive and tailor-made equipment. Using a three-class attention state setting, it achieves average accuracy rates of 59.63% to 77.81%; the best result being 77.81% for a general searching task, which shows 11.9% improvement over the baseline. We also analyze and discuss the contribution by individual features to different models.
Original languageEnglish
Title of host publicationACHI 2014 - 7th International Conference on Advances in Computer-Human Interactions
PublisherInternational Academy, Research and Industry Association, IARIA
Number of pages8
ISBN (Electronic)9781612083254
Publication statusPublished - 1 Jan 2014
Event7th International Conference on Advances in Computer-Human Interactions, ACHI 2014 - Barcelona, Spain
Duration: 23 Mar 201427 Mar 2014


Conference7th International Conference on Advances in Computer-Human Interactions, ACHI 2014


  • Affective computing
  • Attention detection
  • Facial expression
  • Keystroke dynamics
  • Multimodal recognition

ASJC Scopus subject areas

  • Human-Computer Interaction


Dive into the research topics of 'Nonintrusive multimodal attention detection'. Together they form a unique fingerprint.

Cite this