ScreenGlint: Practical, in-situ gaze estimation on smartphones

Michael Xuelin Huang; Jiajia Li; Grace Ngai; Hong Va Leong

doi:10.1145/3025453.3025794

ScreenGlint: Practical, in-situ gaze estimation on smartphones

Michael Xuelin Huang, Jiajia Li, Grace Ngai, Hong Va Leong

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

44 Citations (Scopus)

Abstract

Gaze estimation has widespread applications. However, little work has explored gaze estimation on smartphones, even though they are fast becoming ubiquitous. This paper presents ScreenGlint, a novel approach which exploits the glint (reflection) of the screen on the user's cornea for gaze estimation, using only the image captured by the front-facing camera. We first conduct a user study on common postures during smartphone use. We then design an experiment to evaluate the accuracy of ScreenGlint under varying face-to-screen distances. An in-depth evaluation involving multiple users is conducted and the impact of head pose variations is investigated. ScreenGlint achieves an overall angular error of 2.44° without head pose variations, and 2.94° with head pose variations. Our technique compares favorably to state-of-the-art research works, indicating that the glint of the screen is an effective and practical cue to gaze estimation on the smartphone platform. We believe that this work can open up new possibilities for practical and ubiquitous gaze-aware applications.

Original language	English
Title of host publication	CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems
Subtitle of host publication	Explore, Innovate, Inspire
Publisher	Association for Computing Machinery
Pages	2546-2557
Number of pages	12
Volume	2017-May
ISBN (Electronic)	9781450346559
DOIs	https://doi.org/10.1145/3025453.3025794
Publication status	Published - 2 May 2017
Event	2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017 - Denver, United States Duration: 6 May 2017 → 11 May 2017

Conference

Conference	2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017
Country/Territory	United States
City	Denver
Period	6/05/17 → 11/05/17

Keywords

Gaze estimation
Glint
Mobile eye tracker
Screen reflection

ASJC Scopus subject areas

Human-Computer Interaction
Computer Graphics and Computer-Aided Design
Software

More information

10.1145/3025453.3025794

Cite this

@inproceedings{08989be32df04fc4ab52cd7af4f2e78e,

title = "ScreenGlint: Practical, in-situ gaze estimation on smartphones",

abstract = "Gaze estimation has widespread applications. However, little work has explored gaze estimation on smartphones, even though they are fast becoming ubiquitous. This paper presents ScreenGlint, a novel approach which exploits the glint (reflection) of the screen on the user's cornea for gaze estimation, using only the image captured by the front-facing camera. We first conduct a user study on common postures during smartphone use. We then design an experiment to evaluate the accuracy of ScreenGlint under varying face-to-screen distances. An in-depth evaluation involving multiple users is conducted and the impact of head pose variations is investigated. ScreenGlint achieves an overall angular error of 2.44° without head pose variations, and 2.94° with head pose variations. Our technique compares favorably to state-of-the-art research works, indicating that the glint of the screen is an effective and practical cue to gaze estimation on the smartphone platform. We believe that this work can open up new possibilities for practical and ubiquitous gaze-aware applications.",

keywords = "Gaze estimation, Glint, Mobile eye tracker, Screen reflection",

author = "Huang, {Michael Xuelin} and Jiajia Li and Grace Ngai and {Va Leong}, Hong",

year = "2017",

month = may,

day = "2",

doi = "10.1145/3025453.3025794",

language = "English",

volume = "2017-May",

pages = "2546--2557",

booktitle = "CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems",

publisher = "Association for Computing Machinery",

note = "2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017 ; Conference date: 06-05-2017 Through 11-05-2017",

}

Huang, MX, Li, J, Ngai, G & Va Leong, H 2017, ScreenGlint: Practical, in-situ gaze estimation on smartphones. in CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems: Explore, Innovate, Inspire. vol. 2017-May, Association for Computing Machinery, pp. 2546-2557, 2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017, Denver, United States, 6/05/17. https://doi.org/10.1145/3025453.3025794

ScreenGlint: Practical, in-situ gaze estimation on smartphones. / Huang, Michael Xuelin; Li, Jiajia; Ngai, Grace et al.
CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems: Explore, Innovate, Inspire. Vol. 2017-May Association for Computing Machinery, 2017. p. 2546-2557.

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - ScreenGlint: Practical, in-situ gaze estimation on smartphones

AU - Huang, Michael Xuelin

AU - Li, Jiajia

AU - Ngai, Grace

AU - Va Leong, Hong

PY - 2017/5/2

Y1 - 2017/5/2

N2 - Gaze estimation has widespread applications. However, little work has explored gaze estimation on smartphones, even though they are fast becoming ubiquitous. This paper presents ScreenGlint, a novel approach which exploits the glint (reflection) of the screen on the user's cornea for gaze estimation, using only the image captured by the front-facing camera. We first conduct a user study on common postures during smartphone use. We then design an experiment to evaluate the accuracy of ScreenGlint under varying face-to-screen distances. An in-depth evaluation involving multiple users is conducted and the impact of head pose variations is investigated. ScreenGlint achieves an overall angular error of 2.44° without head pose variations, and 2.94° with head pose variations. Our technique compares favorably to state-of-the-art research works, indicating that the glint of the screen is an effective and practical cue to gaze estimation on the smartphone platform. We believe that this work can open up new possibilities for practical and ubiquitous gaze-aware applications.

AB - Gaze estimation has widespread applications. However, little work has explored gaze estimation on smartphones, even though they are fast becoming ubiquitous. This paper presents ScreenGlint, a novel approach which exploits the glint (reflection) of the screen on the user's cornea for gaze estimation, using only the image captured by the front-facing camera. We first conduct a user study on common postures during smartphone use. We then design an experiment to evaluate the accuracy of ScreenGlint under varying face-to-screen distances. An in-depth evaluation involving multiple users is conducted and the impact of head pose variations is investigated. ScreenGlint achieves an overall angular error of 2.44° without head pose variations, and 2.94° with head pose variations. Our technique compares favorably to state-of-the-art research works, indicating that the glint of the screen is an effective and practical cue to gaze estimation on the smartphone platform. We believe that this work can open up new possibilities for practical and ubiquitous gaze-aware applications.

KW - Gaze estimation

KW - Glint

KW - Mobile eye tracker

KW - Screen reflection

UR - http://www.scopus.com/inward/record.url?scp=85041481290&partnerID=8YFLogxK

U2 - 10.1145/3025453.3025794

DO - 10.1145/3025453.3025794

M3 - Conference article published in proceeding or book

VL - 2017-May

SP - 2546

EP - 2557

BT - CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

T2 - 2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017

Y2 - 6 May 2017 through 11 May 2017

ER -

ScreenGlint: Practical, in-situ gaze estimation on smartphones

Abstract

Conference

Keywords

ASJC Scopus subject areas

More information

Other files and links

Fingerprint

Cite this