Multi-stream fusion network for multi-distortion image super-resolution

Yang Wen, Yupeng Xu, Bin Sheng, Ping Li, Lei Bi, Jinman Kim, Xiangui He, Xun Xu

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

Deblurring, denoising and super-resolution (SR) are important image recovery tasks that are committed to improving image quality. Despite the rapid development of deep learning and vast studies on improving image quality have been proposed, the most existing recovery solutions simply deal with quality degradation caused by a single distortion factor, such as SR focusing on improving spatial resolution. Since very little work has been done to analyze the interaction and characteristics of the deblurring, denoising and SR mixing problems, this paper considers the multi-distortion image recovery problem from a holistic perspective and introduces an end-to-end multi-stream fusion network (MSFN) to restore a multi-distortion image (low-resolution image with noise and blur) into a clear high-resolution (HR) image. Firstly, MSFN adopts multiple reconstruction branches to extract deblurring, denoise and SR features with respect to different degradations. Then, MSFN gradually fuses these multi-stream recovery features in a determined order and obtains an enhanced restoration feature by using two fusion modules. In addition, MSFN uses fusion modules and residual attention modules to facilitate the fusion of different recovery features from the denoising branch and the deblurring branch for the trunk SR branch. Experiments on several benchmarks fully demonstrate the superiority of our MSFN in solving the multi-distortion image recovery problem.

Original languageEnglish
Title of host publicationAdvances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings
EditorsNadia Magnenat-Thalmann, Nadia Magnenat-Thalmann, Victoria Interrante, Daniel Thalmann, George Papagiannakis, Bin Sheng, Jinman Kim, Marina Gavrilova
PublisherSpringer Science and Business Media Deutschland GmbH
Pages242-251
Number of pages10
ISBN (Print)9783030890285
DOIs
Publication statusPublished - Sep 2021
Event38th Computer Graphics International Conference, CGI 2021 - Virtual, Online
Duration: 6 Sep 202110 Sep 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13002 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference38th Computer Graphics International Conference, CGI 2021
CityVirtual, Online
Period6/09/2110/09/21

Keywords

  • Multi-distortion
  • Multi-stream
  • Super-resolution

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Multi-stream fusion network for multi-distortion image super-resolution'. Together they form a unique fingerprint.

Cite this