A feasibility study on smartphone localization using image registration with segmented 3D building models based on multi-material classes

Max Jwo Lem Lee, Li Ta Hsu

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review


Accurate smartphone-based outdoor localization system in deep urban canyons are increasingly needed for various IoT applications such as augmented reality, intelligent transportation, etc. This article proposes a multi-material image registration solution for accurate pose estimation in urban canyons where global navigation satellite system (GNSS) tends to fail. In the offline stage, a material segmented city model is used to generate segmented images at each pose (six degrees of freedom of position and rotation). In the online stage, an image is taken with a smartphone camera that provides textual information about the surrounding environment. The approach utilizes computer vision algorithms to rectify and manually segment between the different types of material identified in the smartphone image. The hypothesized poses (candidate) images are then matched with the segmented smartphone image. The candidate image with the maximum likelihood is regarded as the estimated pose of the user. The positioning results achieves 2.0m level accuracy in common high rise along street, 5.5m in foliage dense environment and 15.7m in alleyway. A 45% positioning improvement to current state-of-the-art method. The estimation of yaw achieves 2.3° level accuracy, 8 times the improvement to smartphone IMU.

Original languageEnglish
Title of host publicationION 2021 International Technical Meeting Proceedings
PublisherInstitute of Navigation
Number of pages7
ISBN (Electronic)9780936406275
Publication statusPublished - Jan 2021

Publication series

NameION 2021 International Technical Meeting Proceedings

ASJC Scopus subject areas

  • Computer Science Applications
  • Aerospace Engineering
  • Electrical and Electronic Engineering
  • Ocean Engineering
  • Transportation

Cite this