UbiPoint: towards non-intrusive mid-air interaction for hardware constrained smart glasses

Lik Hang Lee, Tristan Braud, Farshid Hassani Bijarbooneh, Pan Hui

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

15 Citations (Scopus)

Abstract

Throughout the past decade, numerous interaction techniques have been designed for mobile and wearable devices. Among these devices, smartglasses mostly rely on hardware interfaces such as touchpad and buttons, which are often cumbersome and counterintuitive to use. Furthermore, smartglasses feature cheap and low-power hardware preventing the use of advanced pointing techniques. To overcome these issues, we introduce UbiPoint, a freehand mid-air interaction technique. UbiPoint uses the monocular camera embedded in smartglasses to detect the user's hand without relying on gloves, markers, or sensors, enabling intuitive and non-intrusive interaction. We introduce a computationally fast and light-weight algorithm for fingertip detection, which is especially suited for the limited hardware specifications and the short battery life of smartglasses. UbiPoint processes pictures at a rate of 20 frames per second with high detection accuracy - no more than 6 pixels deviation. Our evaluation shows that UbiPoint, as a mid-air non-intrusive interface, delivers a better experience for users and smart glasses interactions, with users completing typical tasks 1.82 times faster than when using the original hardware.

Original languageEnglish
Title of host publicationMMSys 2020 - Proceedings of the 2020 Multimedia Systems Conference
PublisherAssociation for Computing Machinery, Inc
Pages190-201
Number of pages12
ISBN (Electronic)9781450368452
DOIs
Publication statusPublished - 27 May 2020
Externally publishedYes
Event11th ACM Multimedia Systems Online Conference, MMSys 2020 - Istanbul, Turkey
Duration: 8 Jun 202011 Jun 2020

Publication series

NameMMSys 2020 - Proceedings of the 2020 Multimedia Systems Conference

Conference

Conference11th ACM Multimedia Systems Online Conference, MMSys 2020
Country/TerritoryTurkey
CityIstanbul
Period8/06/2011/06/20

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software

Fingerprint

Dive into the research topics of 'UbiPoint: towards non-intrusive mid-air interaction for hardware constrained smart glasses'. Together they form a unique fingerprint.

Cite this