Noisy practical facial super-resolution method via deformable constrained model with small dataset

Face Super-Resolution (FSR) is to infer high resolution facial image(s) from given low resolution one(s). But when large-scale training samples are absent, FSR may fail in inferring high resolution image for practical low resolution facial image with complex degradation. To solve this problem, we present a novel position patch-based FSR method via latent Deformable Constrained Model (FSR-DCM). Different from conventional FSR methods that view an image patch as a fixed-length vector, we train the target image patch as a matrix in a flexible deformation flow form. This enables the dictionary to cover patterns that do not appear in training examples, resulting in our FSR method to be more expressive and able to solve the outlier (heterogeneous) problem. Besides, instead of explicit Euclidean distance, we use latent deformable similarity as distance criteria to measure the patch similarity, which facilitates our FSR method to work in low-quality scene by emphasizing neighbor relationship and enlarging the distance difference. Through enforcing such a constraint, the expressive capability of the FSR method can be improved, and the restoration failure caused by the coefficients of wrongly emphasized candidates can be overcome. Experiments on the public face datasets CAS-PEAL-R1 [9] and FEI [25] demonstrate the superiority of the proposed algorithm against the existing solutions to the problem of enhancing facial images of very low resolution both quantitatively (Peak-Singal-to-Noise Ratio, i.e., PSNR and Structural Similarity, i.e., the SSIM [29]) and qualitatively (subjective performance).