Content-based image retrieval using a combination of visual features and eye tracking data

Zhen Liang, Hong Fu, Yun Zhang, Zheru Chi, Dagan Feng

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

16 Citations (Scopus)


Image retrieval technology has been developed for more than twenty years. However, the current image retrieval techniques cannot achieve a satisfactory recall and precision. To improve the effectiveness and efficiency of an image retrieval system, a novel content-based image retrieval method with a combination of image segmentation and eye tracking data is proposed in this paper. In the method, eye tracking data is collected by a non-intrusive table mounted eye tracker at a sampling rate of 120 Hz, and the corresponding fixation data is used to locate the human's Regions of Interest (hROIs) on the segmentation result from the JSEG algorithm. The hROIs are treated as important informative segments/objects and used in the image matching. In addition, the relative gaze duration of each hROI is used to weigh the similarity measure for image retrieval. The similarity measure proposed in this paper is based on a retrieval strategy emphasizing the most important regions. Experiments on 7346 Hemera color images annotated manually show that the retrieval results from our proposed approach compare favorably with conventional content-based image retrieval methods, especially when the important regions are difficult to be located based on visual features.
Original languageEnglish
Title of host publicationProceedings of ETRA 2010
Subtitle of host publicationACM Symposium on Eye-Tracking Research and Applications
Number of pages4
Publication statusPublished - 21 May 2010
EventACM Symposium on Eye-Tracking Research and Applications, ETRA 2010 - Austin, TX, United States
Duration: 22 Mar 201024 Mar 2010


ConferenceACM Symposium on Eye-Tracking Research and Applications, ETRA 2010
Country/TerritoryUnited States
CityAustin, TX


  • Content-based image retrieval (CBIR)
  • Eye tracking
  • Fixation
  • Similarity measure
  • Visual perception

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Ophthalmology
  • Sensory Systems


Dive into the research topics of 'Content-based image retrieval using a combination of visual features and eye tracking data'. Together they form a unique fingerprint.

Cite this