Towards Neural Ar: Unsupervised Object Segmentation with 3d Scanned Model Through Relative

Zackary P.T. Sin, Peter H.F. Ng, Hong Va Leong

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

Neural AR augments the reality with a neural model to enable direct manipulation of the reality. This demands object tracking for augmentation, but there is not much work as in typical AR setting. Since most computer vision models manipulate on an image level, effectively tracking a target object for manipulation manifests as a segmentation problem. To the best of our knowledge, there is a gap for a model to perform unsupervised object segmentation with a 3D scanned model useful for Neural AR in reducing the labor-intensive data annotation. We propose in this paper a CycleGAN-inspired model, Realistic Layered Training Image from Virtual Environment (ReLaTIVE), which only requires a user to 3D scan a target object as with typical AR. The ReLaTIVE generator then outputs the object's mask for Neural AR. Without any annotated segmentation mask, it enables the generation of training samples with the 3D scanned model for learning to separate the foreground and background.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1-6
ISBN (Electronic)9781728114859
DOIs
Publication statusPublished - Jul 2020
Event2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020 - London, United Kingdom
Duration: 6 Jul 202010 Jul 2020

Publication series

Name2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020

Conference

Conference2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020
Country/TerritoryUnited Kingdom
CityLondon
Period6/07/2010/07/20

Keywords

  • 3D Scanned Model
  • Neural AR
  • Training Sample Generation
  • Unsupervised Segmentation

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Media Technology

Fingerprint

Dive into the research topics of 'Towards Neural Ar: Unsupervised Object Segmentation with 3d Scanned Model Through Relative'. Together they form a unique fingerprint.

Cite this