Facial Expressions of Sentence Comprehension

Cigdem Turan, Yixin Wang, Shun Cheung Lai, Karl David Neergaard, Kin Man Lam

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

1 Citation (Scopus)

Abstract

Understanding facial expressions allows access to one's intentional and affective states. Using the findings in psychology and neuroscience, in which physical behaviors of the face are linked to emotional states, this paper aims to study sentence comprehension shown by facial expressions. In our experiments, participants took part in a roughly 30-minute computer mediated task, where they were asked to answer either 'true' or 'false' to knowledge-based questions, then immediately given feedback of 'correct' or 'incorrect'. Their faces, which were recorded during the task using the Kinect v2 device, are later used to identify the level of comprehension shown by their expressions. To achieve this, the SVM and Random Forest classifiers with facial appearance information extracted using a spatiotemporal local descriptor, named LPQ-TOP, are employed. Results of online sentence comprehension show that facial dynamics are promising to help understand cognitive states of the mind.

Original languageEnglish
Title of host publication2018 IEEE 23rd International Conference on Digital Signal Processing, DSP 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538668115
DOIs
Publication statusPublished - 19 Nov 2018
Event23rd IEEE International Conference on Digital Signal Processing, DSP 2018 - Shanghai, China
Duration: 19 Nov 201821 Nov 2018

Publication series

NameInternational Conference on Digital Signal Processing, DSP
Volume2018-November

Conference

Conference23rd IEEE International Conference on Digital Signal Processing, DSP 2018
CountryChina
CityShanghai
Period19/11/1821/11/18

Keywords

  • affective computing
  • facial expression recognition
  • human-computer interaction
  • sentence comprehension

ASJC Scopus subject areas

  • Signal Processing

Cite this