Feature evaluation and selection based on neighborhood soft margin

Feature selection is considered to be an important preprocessing step in machine learning and pattern recognition, and feature evaluation is the key issue for constructing a feature selection algorithm. In this work, we propose a new concept of neighborhood margin and neighborhood soft margin to measure the minimal distance between different classes. We use the criterion of neighborhood soft margin to evaluate the quality of candidate features and construct a forward greedy algorithm for feature selection. We conduct this technique on eight classification learning tasks and some cancer recognition tasks. Compared with the raw data and other feature selection algorithms, the proposed technique is effective in most of the cases.