Self-attention starGAN for multi-domain image-to-image translation

Ziliang He, Zhenguo Yang, Xudong Mao, Jianming Lv, Qing Li, Wenyin Liu

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

In this paper, we propose a Self-attention StarGAN by introducing the self-attention mechanism into StarGAN to deal with multi-domain image-to-image translation, aiming to generate images with high-quality details and obtain consistent backgrounds. The self-attention mechanism models the long-range dependencies among the feature maps at all positions, which is not limited to the local image regions. Simultaneously, we take the advantage of batch normalization to reduce reconstruction error and generate fine-grained texture details. We adopt spectral normalization in the network to stabilize the training of Self-attention StarGAN. Both quantitative and qualitative experiments on a public dataset have been conducted. The experimental results demonstrate that the proposed model achieves lower reconstruction error and generates images in higher quality compared to StarGAN. We exploit Amazon Mechanical Turk (AMT) for perceptual evaluation, and 68.1% of all 1,000 AMT Turkers agree that the backgrounds of the images generated by Self-attention StarGAN are more consistent with the original images.

Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning – ICANN 2019
Subtitle of host publicationImage Processing - 28th International Conference on Artificial Neural Networks, 2019, Proceedings
EditorsIgor V. Tetko, Pavel Karpov, Fabian Theis, Vera Kurková
PublisherSpringer Verlag
Pages537-549
Number of pages13
ISBN (Print)9783030305079
DOIs
Publication statusPublished - 2019
Event28th International Conference on Artificial Neural Networks, ICANN 2019 - Munich, Germany
Duration: 17 Sep 201919 Sep 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11729 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference28th International Conference on Artificial Neural Networks, ICANN 2019
CountryGermany
CityMunich
Period17/09/1919/09/19

Keywords

  • GANs
  • Image-to-image translation
  • Multi-domain adaptation
  • Self-attention mechanism

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this