Content-based image retrieval based on eye-tracking

Ying Zhou, Jiajun Wang, Zheru Chi

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

1 Citation (Scopus)


To improve the performance of an image retrieval system, a novel content-based image retrieval (CBIR) framework with eye tracking data based on an implicit relevance feedback mechanism is proposed in this paper. Our proposed framework consists of three components: feature extraction and selection, visual retrieval, and relevance feedback. First, by using the quantum genetic algorithm and the principle component analysis algorithm, optimal image features with 70 components are extracted. Second, a finer retrieving procedure based on multiclass support vector machine (SVM) and fuzzy c-mean (FCM) algorithm is implemented for retrieving most relevant images. Finally, a deep neural network is trained to exploit the information of the user regarding the relevance of the returned images. This information is then employed to update the retrieving point for a new round retrieval. Experiments on two databases (Corel and Caltech) show that the performance of CBIR can be significantly improved by using our proposed framework.

Original languageEnglish
Title of host publicationProceedings - COGAIN 2018
Subtitle of host publicationCommunication by Gaze Interaction
EditorsStephen N. Spencer
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450357906
Publication statusPublished - 15 Jun 2018
Event2018 Workshop on Communication by Gaze Interaction, COGAIN 2018 - Warsaw, Poland
Duration: 15 Jun 2018 → …

Publication series

NameProceedings - COGAIN 2018: Communication by Gaze Interaction


Conference2018 Workshop on Communication by Gaze Interaction, COGAIN 2018
Period15/06/18 → …


  • CBIR
  • Deep neural network
  • Eye tracking

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Content-based image retrieval based on eye-tracking'. Together they form a unique fingerprint.

Cite this