Nonintrusive multimodal attention detection

With the increasing deployment of computers in a wide variety of applications, the ability to detect the user's attention, or engagement, is becoming more important as a key piece of contextual information in building effective interactive systems. For instance, one can imagine that a system that is aware of whether the user is attending to it would be able to adapt itself better to the user activities to enhance productivity. The ability to detect attention would also be useful for system analysis in designing and building better systems. However, much previous work in attention detection is either obtrusive or imposes demanding constraints on the context and the participants. In addition, most approaches rely on uni-modal signals, which are often limited in availability and stability. This paper attempts to address these two major limitations through a noninvasive multimodal solution, which allows participants to work naturally without interference. The solution makes use of common off-the-shelf items that could reasonably be expected of any computing environment and does not rely on expensive and tailor-made equipment. Using a three-class attention state setting, it achieves average accuracy rates of 59.63% to 77.81%; the best result being 77.81% for a general searching task, which shows 11.9% improvement over the baseline. We also analyze and discuss the contribution by individual features to different models.