Photo-Realistic Image Super-Resolution via Variational Autoencoders

Zhi Song Liu, Wan Chi Siu, Yui Lam Chan

Research output: Journal article publicationJournal articleAcademic researchpeer-review

26 Citations (Scopus)


There is a great leap in objective accuracy on image super-resolution, which recently brings a new challenge on image super-resolution with larger up-scaling (e.g. 4×) using pixel based distortion for measurement. This causes over-smooth effect which cannot grasp well the perceptual similarity. The advent of generative adversarial networks makes it possible super-resolve a low-resolution image to generate photo-realistic images sharing distribution with the high-resolution images. However, generative networks suffer from problems of mode-collapse and unrealistic sample generation. We propose to perform Image Super- Resolution via Variational AutoEncoders (SR-VAE) learning according to the conditional distribution of the high-resolution images induced by the low-resolution images. Given that the Conditional Variational Autoencoders tend to generate blur images, we add the conditional sampling mechanism to narrow down the latent subspace for reconstruction. To evaluate the model generalization, we use KL loss to measure the divergence between latent vectors and standard Gaussian distribution. Eventually, in order to balance the trade-off between super-resolution distortion and perception, not only that we use pixel based loss, we also use the modified deep feature loss between SR and HR images to estimate the reconstruction. In experiments, we evaluated a large number of datasets to make comparison with other stateof- the-art super-resolution approaches. Results on both objective and subjective measurements show that our proposed SR-VAE can achieve good photo-realistic perceptual quality closer to the natural image manifold while maintain low distortion.

Original languageEnglish
Article number9121327
Pages (from-to)1-15
JournalIEEE Transactions on Circuits and Systems for Video Technology
Publication statusPublished - Apr 2021


  • distortion
  • Distortion
  • Distortion measurement
  • divergence
  • Feature extraction
  • Gallium nitride
  • Generative adversarial networks
  • Generators
  • Image reconstruction
  • Image super-resolution
  • variational autoencoders

ASJC Scopus subject areas

  • Media Technology
  • Electrical and Electronic Engineering


Dive into the research topics of 'Photo-Realistic Image Super-Resolution via Variational Autoencoders'. Together they form a unique fingerprint.

Cite this