Deep semantic space with intra-class low-rank constraint for cross-modal retrieval

Peipei Kang, Zehang Lin, Zhenguo Yang, Xiaozhao Fang, Qing Li, Wenyin Liu

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

7 Citations (Scopus)

Abstract

In this paper, a novel Deep Semantic Space learning model with Intra-class Low-rank constraint (DSSIL) is proposed for crossmodal retrieval, which is composed of two subnetworks for modality-specific representation learning, followed by projection layers for common space mapping. In particular, DSSIL takes into account semantic consistency to fuse the cross-modal data in a high-level common space, and constrains the common representation matrix within the same class to be low-rank, in order to induce the intra-class representations more relevant. More formally, two regularization terms are devised for the two aspects, which have been incorporated into the objective of DSSIL. To optimize the modality-specific subnetworks and the projection layers simultaneously by exploiting the gradient decent directly, we approximate the nonconvex low-rank constraint by minimizing a few smallest singular values of the intra-class matrix with theoretical analysis. Extensive experiments conducted on three public datasets demonstrate the competitive superiority of DSSIL for cross-modal retrieval compared with the state-of-theart methods.

Original languageEnglish
Title of host publicationICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages226-234
Number of pages9
ISBN (Electronic)9781450367653
DOIs
Publication statusPublished - 5 Jun 2019
Event2019 ACM International Conference on Multimedia Retrieval, ICMR 2019 - Ottawa, Canada
Duration: 10 Jun 201913 Jun 2019

Publication series

NameICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval

Conference

Conference2019 ACM International Conference on Multimedia Retrieval, ICMR 2019
Country/TerritoryCanada
CityOttawa
Period10/06/1913/06/19

Keywords

  • Cross-modal retrieval
  • Deep neural networks
  • Intra-class low-rank
  • Semantic space

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition

Cite this