Eye movement as an interaction mechanism for relevance feedback in a content-based image retrieval system

Yun Zhang, Hong Fu, Zhen Liang, Zheru Chi, Dagan Feng

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

21 Citations (Scopus)


Relevance feedback (RF) mechanisms are widely adopted in Content-Based Image Retrieval (CBIR) systems to improve image retrieval performance. However, there exist some intrinsic problems: (1) the semantic gap between high-level concepts and low-level features and (2) the subjectivity of human perception of visual contents. The primary focus of this paper is to evaluate the possibility of inferring the relevance of images based on eye movement data. In total, 882 images from 101 categories are viewed by 10 subjects to test the usefulness of implicit RF, where the relevance of each image is known beforehand. A set of measures based on fixations are thoroughly evaluated which include fixation duration, fixation count, and the number of revisits. Finally, the paper proposes a decision tree to predict the user's input during the image searching tasks. The prediction precision of the decision tree is over 87%, which spreads light on a promising integration of natural eye movement into CBIR systems in the future.
Original languageEnglish
Title of host publicationProceedings of ETRA 2010
Subtitle of host publicationACM Symposium on Eye-Tracking Research and Applications
Number of pages4
Publication statusPublished - 21 May 2010
EventACM Symposium on Eye-Tracking Research and Applications, ETRA 2010 - Austin, TX, United States
Duration: 22 Mar 201024 Mar 2010


ConferenceACM Symposium on Eye-Tracking Research and Applications, ETRA 2010
Country/TerritoryUnited States
CityAustin, TX


  • Content-based image retrieval (CBIR)
  • Eye tracking
  • Relevance feedback (RF)
  • Visual perception

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Ophthalmology
  • Sensory Systems

Cite this