Progressive multi-scale reconstruction for guided depth map super-resolution via deep residual gate fusion network

Yang Wen, Jihong Wang, Zhen Li, Bin Sheng, Ping Li, Xiaoyu Chi, Lijuan Mao

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

Depth maps obtained by consumer depth sensors are often accompanied by two challenging problems: low spatial resolution and insufficient quality, which greatly limit the potential applications of depth images. To overcome these shortcomings, some depth map super-resolution (DSR) methods tend to extrapolate a high-resolution depth map from a low-resolution depth map with the additional guidance of the corresponding high-resolution intensity image. However, these methods are still prone to texture copying and boundary discontinuities due to improper guidance. In this paper, we propose a deep residual gate fusion network (DRGFN) for guided depth map super-resolution with progressive multi-scale reconstruction. To alleviate the misalignment between color images and depth maps, DRGFN applies a color-guided gate fusion module to acquire content-adaptive attention for better fusing the color and depth features. To focus on restoring details such as boundaries, DRGFN applies a residual attention module to highlight the different importance of different channels. Furthermore, DRGFN applies a multi-scale fusion reconstruction module to make use of multi-scale information for better image reconstruction. Quantitative and qualitative experiments on several benchmarks fully show that DRGFN obtains the state-of-the-art performance for depth map super-resolution.

Original languageEnglish
Title of host publicationAdvances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings
EditorsNadia Magnenat-Thalmann, Nadia Magnenat-Thalmann, Victoria Interrante, Daniel Thalmann, George Papagiannakis, Bin Sheng, Jinman Kim, Marina Gavrilova
PublisherSpringer Science and Business Media Deutschland GmbH
Pages67-79
Number of pages13
ISBN (Print)9783030890285
DOIs
Publication statusPublished - Sep 2021
Event38th Computer Graphics International Conference, CGI 2021 - Virtual, Online
Duration: 6 Sep 202110 Sep 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13002 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference38th Computer Graphics International Conference, CGI 2021
CityVirtual, Online
Period6/09/2110/09/21

Keywords

  • Attention mechanism
  • Color image guidance
  • Depth map super-resolution
  • Gate fusion network
  • Multi-scale

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Progressive multi-scale reconstruction for guided depth map super-resolution via deep residual gate fusion network'. Together they form a unique fingerprint.

Cite this