A Real-Time ASL Recognition System Using Leap Motion Sensors

Kai Yin Fok, Nuwan Ganganath, Chi Tsun Cheng, Chi Kong Tse

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

26 Citations (Scopus)

Abstract

It is always challenging for deaf and speech-impaired people to communicate with non-sign language users. A real-Time sign language recognition system using 3D motion sensors could lower the aforementioned communication barrier. However, most existing gesture recognition systems are adopting a single sensor framework, whose performance is susceptible to occlusions. In this paper, we proposed a real-Time multi-sensor recognition system for American sign language (ASL). Data collected from Leap Motion sensors are fused using multiple sensors data fusion (MSDF) and the recognition is performed using hidden Markov models (HMM). Experimental results demonstrate that the proposed system can deliver higher recognition accuracy over single-sensor systems. Due to its low implementation cost and higher accuracy, the proposed system can be widely deployed and bring conveniences to sign language users.
Original languageEnglish
Title of host publicationProceedings - 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC 2015
PublisherIEEE
Pages411-414
Number of pages4
ISBN (Electronic)9781467391993
DOIs
Publication statusPublished - 26 Oct 2015
Event7th International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC 2015 - Xi'an, China
Duration: 17 Sep 201519 Sep 2015

Conference

Conference7th International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC 2015
Country/TerritoryChina
CityXi'an
Period17/09/1519/09/15

Keywords

  • American sign language
  • depth sensors
  • hidden Markov models
  • Leap Motion sensor
  • sensor fusion
  • Sign language recognition

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this