Maximum entropy relief feature weighting

Xiang Zhang, Zhaohong Deng, Shitong Wang, Kup Sze Choi

Research output: Journal article publicationJournal articleAcademic researchpeer-review

11 Citations (Scopus)


A latest advance in Relief feature weighting techniques is that it can be approximately expressed as a margin maximization problem and therefore its distinctive properties can be investigated with the help of the optimization theory. Although Relief feature has been widely used, it lacks a mechanism to deal with outlier data and how to enhance the robustness and the adjustability of the algorithm in noisy environments is still not very obvious. In order to enhance Relief's adjustability and robustness, by integrating maximum entropy technique into Relief feature weighting techniques, the more robust and adaptive Relief feature weighting new algorithms are investigated. First, a new margin-based objective function integrating maximum entropy is proposed within the optimization framework, where two maximum entropy terms are adopted to control the feature weights and sample force coefficients respectively. Then by applying optimization theory, some of useful theoretical results are derived from the proposed objective function and then a set of robust Relief feature weighting algorithms are developed for two-class data, multi-class data and online data. As demonstrated by extensive experiments in UCI benchmark datasets and gene expression datasets, the proposed new algorithms show the competitive performance to the state-of-the-art algorithms and much better robustness to datasets with noise and/or outliers.
Original languageEnglish
Pages (from-to)1038-1048
Number of pages11
JournalJisuanji Yanjiu yu Fazhan/Computer Research and Development
Issue number6
Publication statusPublished - 1 Jun 2011


  • Feature selection
  • Feature weighting
  • Margin maximization principle
  • Maximum entropy
  • Relief algorithm

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Maximum entropy relief feature weighting'. Together they form a unique fingerprint.

Cite this