Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network

Shumin Zhu; Xingxing Zou; Wai Keung Wong

Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network

Shumin Zhu
, Xingxing Zou
, Wai Keung Wong

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

Abstract

Studying converting sketch images into corresponding real pictures is challenging. The rapid development of the generative adversarial network (GAN) and its excellent performance in image-to-image translation tasks make this research possible. This paper proposes a fashion sketch image to real image (FSRI) system based on the StyleGAN network. The system leverages the pSp encoder to extract the styles of the sketch image, inputs the pre-trained StyleGAN to obtain the corresponding real picture, and implements the image warping algorithm to automatically adjust the head-to-body ratio of the model in the output picture. Extensive experimentation and visualization results derived from a self-established sketch dataset show that the system has excellent capabilities to convert complex sketch images to real images.

Original language	English
Title of host publication	International Conference on Design and Semantics of Form and Movement
Editors	Miguel Bruns, Lin-Lin Chen, Jun Hu, Sara Colombo, Yihyun Lim, Steven Kyffin, Ozcan Vieira, E. Jeroen Raijmakers, Lucia Rampino, Edgar Rodriguez Ramirez, Dagmar Johanna Steffen, Calvin Wong
Publisher	The Hong Kong Polytechnic University
Pages	105-115
Number of pages	11
ISBN (Print)	9789623678704
Publication status	Published - 2023
Event	12th International Conference on Design and Semantics of Form and Movement, DeSForM 2023 - Hong Kong, Hong Kong Duration: 5 Jul 2023 → 7 Jul 2023

Publication series

Name	International Conference on Design and Semantics of Form and Movement
ISSN (Electronic)	2706-6150

Conference

Conference	12th International Conference on Design and Semantics of Form and Movement, DeSForM 2023
Country/Territory	Hong Kong
City	Hong Kong
Period	5/07/23 → 7/07/23

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

fashion sketch
image warping
pSp encoder
real image
StyleGAN

ASJC Scopus subject areas

Arts and Humanities (miscellaneous)
Human-Computer Interaction
Mechanics of Materials

Cite this

Zhu, S., Zou, X., & Wong, W. K. (2023). Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network. In M. Bruns, L.-L. Chen, J. Hu, S. Colombo, Y. Lim, S. Kyffin, O. Vieira, E. J. Raijmakers, L. Rampino, E. R. Ramirez, D. J. Steffen, & C. Wong (Eds.), International Conference on Design and Semantics of Form and Movement (pp. 105-115). (International Conference on Design and Semantics of Form and Movement). The Hong Kong Polytechnic University.

Zhu, Shumin ; Zou, Xingxing ; Wong, Wai Keung. / Fashion Sketch to Real Image System : A Designer Aids based on Generative Adversarial Network. International Conference on Design and Semantics of Form and Movement. editor / Miguel Bruns ; Lin-Lin Chen ; Jun Hu ; Sara Colombo ; Yihyun Lim ; Steven Kyffin ; Ozcan Vieira ; E. Jeroen Raijmakers ; Lucia Rampino ; Edgar Rodriguez Ramirez ; Dagmar Johanna Steffen ; Calvin Wong. The Hong Kong Polytechnic University, 2023. pp. 105-115 (International Conference on Design and Semantics of Form and Movement).

@inproceedings{43b35964bbe04742bd949e5aad0f491e,

title = "Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network",

abstract = "Studying converting sketch images into corresponding real pictures is challenging. The rapid development of the generative adversarial network (GAN) and its excellent performance in image-to-image translation tasks make this research possible. This paper proposes a fashion sketch image to real image (FSRI) system based on the StyleGAN network. The system leverages the pSp encoder to extract the styles of the sketch image, inputs the pre-trained StyleGAN to obtain the corresponding real picture, and implements the image warping algorithm to automatically adjust the head-to-body ratio of the model in the output picture. Extensive experimentation and visualization results derived from a self-established sketch dataset show that the system has excellent capabilities to convert complex sketch images to real images.",

keywords = "fashion sketch, image warping, pSp encoder, real image, StyleGAN",

author = "Shumin Zhu and Xingxing Zou and Wong, \{Wai Keung\}",

note = "Publisher Copyright: {\textcopyright} 2023, The Hong Kong Polytechnic University. All rights reserved.; 12th International Conference on Design and Semantics of Form and Movement, DeSForM 2023 ; Conference date: 05-07-2023 Through 07-07-2023",

year = "2023",

language = "English",

isbn = "9789623678704",

series = "International Conference on Design and Semantics of Form and Movement",

publisher = "The Hong Kong Polytechnic University",

pages = "105--115",

editor = "Miguel Bruns and Lin-Lin Chen and Jun Hu and Sara Colombo and Yihyun Lim and Steven Kyffin and Ozcan Vieira and Raijmakers, \{E. Jeroen\} and Lucia Rampino and Ramirez, \{Edgar Rodriguez\} and Steffen, \{Dagmar Johanna\} and Calvin Wong",

booktitle = "International Conference on Design and Semantics of Form and Movement",

address = "Hong Kong",

}

Zhu, S, Zou, X & Wong, WK 2023, Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network. in M Bruns, L-L Chen, J Hu, S Colombo, Y Lim, S Kyffin, O Vieira, EJ Raijmakers, L Rampino, ER Ramirez, DJ Steffen & C Wong (eds), International Conference on Design and Semantics of Form and Movement. International Conference on Design and Semantics of Form and Movement, The Hong Kong Polytechnic University, pp. 105-115, 12th International Conference on Design and Semantics of Form and Movement, DeSForM 2023, Hong Kong, Hong Kong, 5/07/23.

Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network. / Zhu, Shumin; Zou, Xingxing; Wong, Wai Keung.
International Conference on Design and Semantics of Form and Movement. ed. / Miguel Bruns; Lin-Lin Chen; Jun Hu; Sara Colombo; Yihyun Lim; Steven Kyffin; Ozcan Vieira; E. Jeroen Raijmakers; Lucia Rampino; Edgar Rodriguez Ramirez; Dagmar Johanna Steffen; Calvin Wong. The Hong Kong Polytechnic University, 2023. p. 105-115 (International Conference on Design and Semantics of Form and Movement).

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - Fashion Sketch to Real Image System

T2 - 12th International Conference on Design and Semantics of Form and Movement, DeSForM 2023

AU - Zhu, Shumin

AU - Zou, Xingxing

AU - Wong, Wai Keung

PY - 2023

Y1 - 2023

N2 - Studying converting sketch images into corresponding real pictures is challenging. The rapid development of the generative adversarial network (GAN) and its excellent performance in image-to-image translation tasks make this research possible. This paper proposes a fashion sketch image to real image (FSRI) system based on the StyleGAN network. The system leverages the pSp encoder to extract the styles of the sketch image, inputs the pre-trained StyleGAN to obtain the corresponding real picture, and implements the image warping algorithm to automatically adjust the head-to-body ratio of the model in the output picture. Extensive experimentation and visualization results derived from a self-established sketch dataset show that the system has excellent capabilities to convert complex sketch images to real images.

AB - Studying converting sketch images into corresponding real pictures is challenging. The rapid development of the generative adversarial network (GAN) and its excellent performance in image-to-image translation tasks make this research possible. This paper proposes a fashion sketch image to real image (FSRI) system based on the StyleGAN network. The system leverages the pSp encoder to extract the styles of the sketch image, inputs the pre-trained StyleGAN to obtain the corresponding real picture, and implements the image warping algorithm to automatically adjust the head-to-body ratio of the model in the output picture. Extensive experimentation and visualization results derived from a self-established sketch dataset show that the system has excellent capabilities to convert complex sketch images to real images.

KW - fashion sketch

KW - image warping

KW - pSp encoder

KW - real image

KW - StyleGAN

UR - https://www.scopus.com/pages/publications/85190668508

M3 - Conference article published in proceeding or book

AN - SCOPUS:85190668508

SN - 9789623678704

T3 - International Conference on Design and Semantics of Form and Movement

SP - 105

EP - 115

BT - International Conference on Design and Semantics of Form and Movement

A2 - Bruns, Miguel

A2 - Chen, Lin-Lin

A2 - Hu, Jun

A2 - Colombo, Sara

A2 - Lim, Yihyun

A2 - Kyffin, Steven

A2 - Vieira, Ozcan

A2 - Raijmakers, E. Jeroen

A2 - Rampino, Lucia

A2 - Ramirez, Edgar Rodriguez

A2 - Steffen, Dagmar Johanna

A2 - Wong, Calvin

PB - The Hong Kong Polytechnic University

Y2 - 5 July 2023 through 7 July 2023

ER -

Zhu S, Zou X, Wong WK. Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network. In Bruns M, Chen LL, Hu J, Colombo S, Lim Y, Kyffin S, Vieira O, Raijmakers EJ, Rampino L, Ramirez ER, Steffen DJ, Wong C, editors, International Conference on Design and Semantics of Form and Movement. The Hong Kong Polytechnic University. 2023. p. 105-115. (International Conference on Design and Semantics of Form and Movement).

Fashion Sketch to Real Image System: A Designer Aids based on Generative Adversarial Network

Abstract

Publication series

Conference

UN SDGs

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this