Multilayer ensemble pruning via novel multi-sub-swarm particle swarm optimization

Jun Zhang, Kwok Wing Chau

Research output: Journal article publicationJournal articleAcademic researchpeer-review

149 Citations (Scopus)

Abstract

Recently, classifier ensemble methods are gaining more and more attention in the machine-learning and data-mining communities. In most cases, the performance of an ensemble is better than a single classifier. Many methods for creating diverse classifiers were developed during the past decade. When these diverse classifiers are generated, it is important to select the proper base classifier to join the ensemble. Usually, this selection process is called pruning the ensemble. In general, the ensemble pruning is a selection process in which an optimal combination will be selected from many existing base classifiers. Some base classifiers containing useful information may be excluded in this pruning process. To avoid this problem, the multilayer ensemble pruning model is used in this paper. In this model, the pruning of one layer can be seen as a multimodal optimization problem. A novel multi-sub-swarm particle swarm optimization (MSSPSO) is used here to find multi-solutions for this multilayer ensemble pruning model. In this model, each base classifier will generate an oracle output. Each layer will use MSSPSO algorithm to generate a different pruning based on previous oracle output. A series of experiments using UCI dataset is conducted, the experimental results show that the multilayer ensemble pruning via MSSPSO algorithm can improve the generalization performance of the multi-classifiers ensemble system. Besides, the experimental results show a relationship between the diversity and the pruning technique.
Original languageEnglish
Pages (from-to)840-858
Number of pages19
JournalJournal of Universal Computer Science
Volume15
Issue number4
Publication statusPublished - 15 Jul 2009

Keywords

  • Classifier ensemble
  • Ensemble pruning
  • Multi-layer ensemble model
  • Particle swarm optimization

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this