DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

Chu Tak Li, Wan Chi Siu, Zhi Song Liu, Li Wen Wang, Daniel Pak Kong Lun

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

The degree of difficulty in image inpainting depends on the types and sizes of the missing parts. Existing image inpainting approaches usually encounter difficulties in completing the missing parts in the wild with pleasing visual and contextual results as they are trained for either dealing with one specific type of missing patterns (mask) or unilaterally assuming the shapes and/or sizes of the masked areas. We propose a deep generative inpainting network, named DeepGIN, to handle various types of masked images. We design a Spatial Pyramid Dilation (SPD) ResNet block to enable the use of distant features for reconstruction. We also employ Multi-Scale Self-Attention (MSSA) mechanism and Back Projection (BP) technique to enhance our inpainting results. Our DeepGIN outperforms the state-of-the-art approaches generally, including two publicly available datasets (FFHQ and Oxford Buildings), both quantitatively and qualitatively. We also demonstrate that our model is capable of completing masked images in the wild.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2020 Workshops, Proceedings
EditorsAdrien Bartoli, Andrea Fusiello
PublisherSpringer Science and Business Media Deutschland GmbH
Pages5-22
Number of pages18
ISBN (Print)9783030668228
DOIs
Publication statusPublished - Jan 2021
EventWorkshops held at the 16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom
Duration: 23 Aug 202028 Aug 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12538 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceWorkshops held at the 16th European Conference on Computer Vision, ECCV 2020
Country/TerritoryUnited Kingdom
CityGlasgow
Period23/08/2028/08/20

Keywords

  • Attention
  • Back projection
  • Image inpainting

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this