A novel evolutionary preprocessing method based on over-sampling and under-sampling for imbalanced datasets

Ginny Y. Wong, Hung Fat Frank Leung, Sai Ho Ling

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

8 Citations (Scopus)

Abstract

Imbalanced datasets are commonly encountered in real-world classification problems. However, many machine learning algorithms are originally designed for well-balanced datasets. Re-sampling has become an important step to preprocess imbalanced dataset. It aims at balancing the datasets by increasing the sample size of the smaller class or decreasing the sample size of the larger class, which are known as over-sampling and under-sampling respectively. In this paper, a novel sampling strategy based on both over-sampling and under-sampling is proposed, in which the new samples of the smaller class are created by the Synthetic Minority Over-sampling Technique (SMOTE). The improvement of the datasets is done by the evolutionary computational method of CHC that works on both the minority class and majority class samples. The result is a hybrid data preprocessing method that combines both over-sampling and under-sampling techniques to re-sample datasets. The evaluation is done by applying the learning algorithm C4.5 to obtain a classification model from the re-sampled datasets. Experimental results reported that the proposed approach can decrease the over-sampling rate about 50% with only around 3% discrepancy on the accuracy.
Original languageEnglish
Title of host publicationProceedings, IECON 2013 - 39th Annual Conference of the IEEE Industrial Electronics Society
Pages2354-2359
Number of pages6
DOIs
Publication statusPublished - 1 Dec 2013
Event39th Annual Conference of the IEEE Industrial Electronics Society, IECON 2013 - Vienna, Austria
Duration: 10 Nov 201314 Nov 2013

Conference

Conference39th Annual Conference of the IEEE Industrial Electronics Society, IECON 2013
CountryAustria
CityVienna
Period10/11/1314/11/13

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this