Pixel-Level and Perceptual-Level Regularized Adversarial Learning for Joint Motion Deblurring and Super-Resolution

This paper aims to restore a clear image at high resolution from a low-resolution and motion-blurred image. To this end, we propose an end-to-end neural network named P²GAN containing deblurring and super-resolution modules with pixel-level and perceptual-level regularization terms. In the deblurring module, we propose an Asymmetric Residual Encoder-Decoder architecture, which enlarges the receptive field by stacking the Residual In Residual (RIR) structure with different numbers of convolutional filters to address different sizes of motion blur kernels. In particular, we extend RIR with a Channel Attention Block (CAB) to reweight the deblurring channel features. Similarly, CAB is integrated into the Residual In Residual Dense (RIRD) structure, denoted as RIRD-CA, to reweight channelwise features from input images and the deblurring module in the super-resolution module. RIRD-CA establishes a continuous-memory mechanism that preserves the features extracted from earlier layers along with the network. In particular, pixelwise loss, adversarial loss, and contextual loss have been incorporated into P²GAN from pixel-level and perceptual-level perspectives. Adversarial loss is used to generate rich textures and high-frequency details, while contextual loss eliminates unrealistic textures. Extensive experiments conducted on a publicly available dataset show the effectiveness of the proposed model which outperforms the state-of-the-art approaches.