Sketch-then-Edit Generative Adversarial Network

Wei Li, Linchuan Xu, Zhixuan Liang, Senzhang Wang, Jiannong Cao, Chao Ma, Xiaohui Cui

Research output: Journal article publicationJournal articleAcademic researchpeer-review

16 Citations (Scopus)


Generative Adversarial Network (GAN) has been widely used to generate impressively plausible data. However, it is a non-trivial task to train the original GAN model in practice due to the vanishing gradient problem. This is because the JS divergence could be a constant (i.e., log2) when original data distribution and generated data distribution hold a negligible overlapping area. Under such a scenario, the gradient of generator is 0. Most efforts have been devoted to designing a more proper difference measure while few attentions have been paid to the former aspect of the issue. In this paper, we propose a new method to design a noise distribution having a guaranteed non-negligible overlapping area with raw data distribution. The key idea is to transform the noise from the randomized space into the raw data space. We propose to obtain the transformation as the basis matrix in non-negative matrix factorization because the basis matrix has the underlying features of the raw data. The proposed idea is instantiated as Sketch-then-Edit GAN (SEGAN) where sketches are the noises after transformation and are adopted as the name since they contains basic features of the raw data. Moreover, a new generator for editing the sketches into realistic-like data is designed. We mathematically prove that SEGAN solves the gradient vanishing problem, and conduct extensive experiments on the MNIST, CIFAR10, SVHN and Celeba datasets to demonstrate the effectiveness of SEGAN.

Original languageEnglish
Article number106102
Pages (from-to)1-11
Number of pages11
JournalKnowledge-Based Systems
Publication statusPublished - 5 Sept 2020


  • Generative adversarial network
  • Non-negative matrix factorization
  • Vanishing gradient problem

ASJC Scopus subject areas

  • Management Information Systems
  • Software
  • Information Systems and Management
  • Artificial Intelligence


Dive into the research topics of 'Sketch-then-Edit Generative Adversarial Network'. Together they form a unique fingerprint.

Cite this