Multi-Constraint Adversarial Networks for Unsupervised Image-to-Image Translation

Divya Saxena, Tarun Kulshrestha, Jiannong Cao, Shing Chi Cheung

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Unsupervised image-to-image translation aims to learn the mapping from an input image in a source domain to an output image in a target domain without paired training dataset. Recently, remarkable progress has been made in translation due to the development of generative adversarial networks (GANs). However, existing methods suffer from the training instability as gradients passing from discriminator to generator become less informative when the source and target domains exhibit sufficiently large discrepancies in appearance or shape. To handle this challenging problem, in this paper, we propose a novel multi-constraint adversarial model (MCGAN) for image translation in which multiple adversarial constraints are applied at generator's multi-scale outputs by a single discriminator to pass gradients to all the scales simultaneously and assist generator training for capturing large discrepancies in appearance between two domains. We further notice that the solution to regularize generator is helpful in stabilizing adversarial training, but results may have unreasonable structure or blurriness due to less context information flow from discriminator to generator. Therefore, we adopt dense combinations of the dilated convolutions at discriminator for supporting more information flow to generator. With extensive experiments on three public datasets, cat-to-dog, horse-to-zebra, and apple-to-orange, our method significantly improves state-of-the-arts on all datasets.

Original languageEnglish
Pages (from-to)1601-1612
Number of pages12
JournalIEEE Transactions on Image Processing
Volume31
DOIs
Publication statusPublished - 2022

Keywords

  • GANs
  • Generative adversarial networks
  • generative modeling
  • image synthesis
  • unsupervised image-to-image translation

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Multi-Constraint Adversarial Networks for Unsupervised Image-to-Image Translation'. Together they form a unique fingerprint.

Cite this