Eye tracking data guided feature selection for image classification

Xuan Zhou, Xin Gao, Jiajun Wang, Hui Yu, Zhiyong Wang, Zheru Chi

Research output: Journal article publicationJournal articleAcademic researchpeer-review

24 Citations (Scopus)


Although existing feature selection methods have achieved promising progress, human factors have seldom been taken into account. To tackle such a problem, a novel two-stage feature selection method is proposed for image classification by taking human factors into account and leveraging the value of eye tracking data. In the coarse selection stage, with the help of eye tracking data, Regions of Interests (ROIs) from the human perspective are first identified to represent an image with visual features. Then, with an improved quantum genetic algorithm (IQGA) that incorporates a novel mutation strategy for alleviating the premature convergence, a subset of features is obtained for the subsequent fine selection. In the fine selection stage, a hybrid method is proposed to integrate the efficiency of the minimal-Redundancy-Maximal-Relevance (mRMR) and the effectiveness of the Support Vector Machine based Recursive Feature Elimination (SVM-RFE). In particular, the ranking criterion of the SVM-RFE is improved by incorporating the ranking information obtained from the mRMR. Comprehensive experimental results in two benchmark datasets demonstrate that eye tracking data are of great importance to improve the performance of feature selection for image classification.
Original languageEnglish
Pages (from-to)56-70
Number of pages15
JournalPattern Recognition
Publication statusPublished - 1 Mar 2017


  • Eye tracking
  • Feature selection
  • mRMR
  • Quantum genetic algorithm (QGA)

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this