Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation

Shenlu Jiang, Wei Yao, Man Sing Wong, Meng Hang, Zhonghua Hong, Eun Jin Kim, Sung Hyeon Joo, Tae Yong Kuc

Research output: Journal article publicationJournal articleAcademic researchpeer-review

9 Citations (Scopus)


Simultaneous localization and mapping (SLAM) is an important function for service robots to self-navigate modernized buildings. However, only a few existing applications allow them to automatically move between stories through elevator. Some approaches have accomplished with the aid of hardware; however, this study shows that computer vision can be a promising alternative for button localization. In this paper, we proposed a real-Time multi-story SLAM system which overcomes the problem of detecting elevator buttons using a localization framework that combines tracking and detecting approaches. A two-stage deep neural network initially locates the original positions of the target buttons, and a part-based tracker follows the target buttons in real-Time. A positive-negative classifier and deep learning neural network (particular for button shape detection) modify the tracker's output in every frame. To allow the robot to self-navigate, a 2D grid mapping approach was used for the localization and mapping. Then, when the robot navigates a floor, the A∗ algorithm generates the shortest path. In the experiment, two dynamic scenes (which include common elevator button localization challenges) were used to evaluate the efficiency of our approach, and compared it with other state-of-The-Art methods. Our approach was also tested on a prototype robot system to assesses how well it can navigate a multi-story building. The results show that our method could overcome the common background challenges that occur inside an elevator, and in doing so, it enables the mobile robot to autonomously navigate a multi-story building.

Original languageEnglish
Article number8926334
Pages (from-to)1118-1134
Number of pages17
JournalIEEE Access
Publication statusPublished - 1 Jan 2020


  • deep learning
  • Elevator button localization
  • multi-story navigation
  • object detection
  • visual tracking

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation'. Together they form a unique fingerprint.

Cite this