End-to-end photo-sketch generation via fully convolutional representation learning

Liliang Zhang, Liang Lin, Xian Wu, Shengyong Ding, Lei Zhang

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

77 Citations (Scopus)

Abstract

Sketch-based face recognition is an interesting task in vision and multimedia research, yet it is quite challenging due to the great difference between face photos and sketches. In this paper, we propose a novel approach for photo-sketch generation, aiming to automatically transform face photos into detail-preserving personal sketches. Unlike the traditional models synthesizing sketches based on a dictionary of exemplars, we develop a fully convolutional network to learn the end-to-end photo-sketch mapping. Our approach takes whole face photos as inputs and directly generates the corresponding sketch images with efficient inference and learning, in which the architecture is stacked by only convolutional kernels of very small sizes. To well capture the person identity during the photo-sketch transformation, we define our optimization objective in the form of joint generativediscriminative minimization. In particular, a discriminative regularization term is incorporated into the photo-sketch generation, enhancing the discriminability of the generated person sketches against other individuals. Extensive experiments on several standard benchmarks suggest that our approach outperforms other state-of-the-arts in both photosketch generation and face sketch verification.

Original languageEnglish
Title of host publicationICMR 2015 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages627-634
Number of pages8
ISBN (Electronic)9781450332743
DOIs
Publication statusPublished - 22 Jun 2015
Event5th ACM International Conference on Multimedia Retrieval, ICMR 2015 - Shanghai, China
Duration: 23 Jun 201526 Jun 2015

Publication series

NameICMR 2015 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval

Conference

Conference5th ACM International Conference on Multimedia Retrieval, ICMR 2015
CountryChina
CityShanghai
Period23/06/1526/06/15

Keywords

  • Face verification
  • Neural nets
  • Sketch-photo generation

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software
  • Computer Vision and Pattern Recognition

Cite this