From writing to painting: A kinect-based cross-modal Chinese painting generation system

Jiajia Li, Grace Ngai, Stephen C.F. Chan, Kien A. Hua, Hong Va Leong, Alvin Chan

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

3 Citations (Scopus)


As computer and interaction technologies mature, a much broader range of media is now used for input and output, each of which has its own rich repertoire of techniques, instruments, and cultural heritage. The combination of multiple media can produce novel multimedia human-computer interaction approaches which are more efficient and interesting than traditional single media methods. This paper presents CalliPaint, a system for cross-modal art generation that links together Chinese ink brush calligraphy writing and Chinese landscape painting. We investigate the mapping between the two modalities based on concepts of metaphoric congruence, and implement our findings into a prototype system. A multi-step evaluation experiment with real users suggests that CalliPaint provides a realistic and intuitive experience that allows even novice users to create attractive landscape paintings from writing. Comparison with a generalpurpose digital painting software suggests that CalliPaint provides users with a more enjoyable experience. Finally, exhibiting CalliPaint in an open-access location for use by casual users without any training shows that the system is easy to learn.
Original languageEnglish
Title of host publicationMM 2014 - Proceedings of the 2014 ACM Conference on Multimedia
PublisherAssociation for Computing Machinery, Inc
Number of pages10
ISBN (Electronic)9781450330633
Publication statusPublished - 1 Jan 2014
Event2014 ACM Conference on Multimedia, MM 2014 - Orlando, United States
Duration: 3 Nov 20147 Nov 2014


Conference2014 ACM Conference on Multimedia, MM 2014
Country/TerritoryUnited States


  • Creativity
  • Cross-modal interaction
  • Paintings
  • Writing

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Media Technology
  • Software


Dive into the research topics of 'From writing to painting: A kinect-based cross-modal Chinese painting generation system'. Together they form a unique fingerprint.

Cite this