Deep learning regularized fisher mappings

Wai Keung Wong, Mingming Sun

Research output: Journal article publicationJournal articleAcademic researchpeer-review

33 Citations (Scopus)

Abstract

For classification tasks, it is always desirable to extract features that are most effective for preserving class separability. In this brief, we propose a new feature extraction method called regularized deep Fisher mapping (RDFM), which learns an explicit mapping from the sample space to the feature space using a deep neural network to enhance the separability of features according to the Fisher criterion. Compared to kernel methods, the deep neural network is a deep and nonlocal learning architecture, and therefore exhibits more powerful ability to learn the nature of highly variable datasets from fewer samples. To eliminate the side effects of overfitting brought about by the large capacity of powerful learners, regularizers are applied in the learning procedure of RDFM. RDFM is evaluated in various types of datasets, and the results reveal that it is necessary to apply unsupervised regularization in the fine-tuning phase of deep learning. Thus, for very flexible models, the optimal Fisher feature extractor may be a balance between discriminative ability and descriptive ability.
Original languageEnglish
Article number5982410
Pages (from-to)1668-1675
Number of pages8
JournalIEEE Transactions on Neural Networks
Volume22
Issue number10
DOIs
Publication statusPublished - 1 Oct 2011

Keywords

  • Deep learning architecture
  • feature extraction
  • Fisher criterion
  • regularization

ASJC Scopus subject areas

  • Software
  • Medicine(all)
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this