TY - GEN
T1 - Multi-stream fusion network for multi-distortion image super-resolution
AU - Wen, Yang
AU - Xu, Yupeng
AU - Sheng, Bin
AU - Li, Ping
AU - Bi, Lei
AU - Kim, Jinman
AU - He, Xiangui
AU - Xu, Xun
N1 - Funding Information:
Acknowledgement. This work was supported in part by the National Natural Science Foundation of China under Grants 62077037 and 61872241, in part by Shanghai Municipal Science and Technology Major Project under Grant 2021SHZDZX0102, in part by the Science and Technology Commission of Shanghai Municipality under Grants 18410750700 and 17411952600, in part by Shanghai Lin-Gang Area Smart Manufacturing Special Project under Grant ZN2018020202-3, and in part by Project of Shanghai Municipal Health Commission(2018ZHYL0230).
Publisher Copyright:
© 2021, Springer Nature Switzerland AG.
PY - 2021/9
Y1 - 2021/9
N2 - Deblurring, denoising and super-resolution (SR) are important image recovery tasks that are committed to improving image quality. Despite the rapid development of deep learning and vast studies on improving image quality have been proposed, the most existing recovery solutions simply deal with quality degradation caused by a single distortion factor, such as SR focusing on improving spatial resolution. Since very little work has been done to analyze the interaction and characteristics of the deblurring, denoising and SR mixing problems, this paper considers the multi-distortion image recovery problem from a holistic perspective and introduces an end-to-end multi-stream fusion network (MSFN) to restore a multi-distortion image (low-resolution image with noise and blur) into a clear high-resolution (HR) image. Firstly, MSFN adopts multiple reconstruction branches to extract deblurring, denoise and SR features with respect to different degradations. Then, MSFN gradually fuses these multi-stream recovery features in a determined order and obtains an enhanced restoration feature by using two fusion modules. In addition, MSFN uses fusion modules and residual attention modules to facilitate the fusion of different recovery features from the denoising branch and the deblurring branch for the trunk SR branch. Experiments on several benchmarks fully demonstrate the superiority of our MSFN in solving the multi-distortion image recovery problem.
AB - Deblurring, denoising and super-resolution (SR) are important image recovery tasks that are committed to improving image quality. Despite the rapid development of deep learning and vast studies on improving image quality have been proposed, the most existing recovery solutions simply deal with quality degradation caused by a single distortion factor, such as SR focusing on improving spatial resolution. Since very little work has been done to analyze the interaction and characteristics of the deblurring, denoising and SR mixing problems, this paper considers the multi-distortion image recovery problem from a holistic perspective and introduces an end-to-end multi-stream fusion network (MSFN) to restore a multi-distortion image (low-resolution image with noise and blur) into a clear high-resolution (HR) image. Firstly, MSFN adopts multiple reconstruction branches to extract deblurring, denoise and SR features with respect to different degradations. Then, MSFN gradually fuses these multi-stream recovery features in a determined order and obtains an enhanced restoration feature by using two fusion modules. In addition, MSFN uses fusion modules and residual attention modules to facilitate the fusion of different recovery features from the denoising branch and the deblurring branch for the trunk SR branch. Experiments on several benchmarks fully demonstrate the superiority of our MSFN in solving the multi-distortion image recovery problem.
KW - Multi-distortion
KW - Multi-stream
KW - Super-resolution
UR - http://www.scopus.com/inward/record.url?scp=85118354610&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-89029-2_19
DO - 10.1007/978-3-030-89029-2_19
M3 - Conference article published in proceeding or book
AN - SCOPUS:85118354610
SN - 9783030890285
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 242
EP - 251
BT - Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings
A2 - Magnenat-Thalmann, Nadia
A2 - Magnenat-Thalmann, Nadia
A2 - Interrante, Victoria
A2 - Thalmann, Daniel
A2 - Papagiannakis, George
A2 - Sheng, Bin
A2 - Kim, Jinman
A2 - Gavrilova, Marina
PB - Springer Science and Business Media Deutschland GmbH
T2 - 38th Computer Graphics International Conference, CGI 2021
Y2 - 6 September 2021 through 10 September 2021
ER -