From Gaussian kernel density estimation to kernel methods

Shitong Wang, Zhaohong Deng, Fu Lai Korris Chung, Wenjun Hu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

10 Citations (Scopus)

Abstract

This paper explores how a kind of probabilistic systems, namely, Gaussian kernel density estimation (GKDE), can be used to interpret several classical kernel methods, including the well-known support vector machine (SVM), support vector regression (SVR), one-class kernel classifier, i. e., support vector data description (SVDD) or equivalently minimal enclosing ball (MEB), and the fuzzy systems (FS). For the SVM, we reveal that the classical SVM with Gaussian density kernel attempts to find a noisy GKDE based Bayesian classifier with equal prior probabilities for each class. For the SVR, the classification based ε-SVR attempts to obtain two noisy GKDEs for each class in the constructed binary classification dataset, and the decision boundary just corresponds to the mapping function of the original regression problem. For the MEB or SVDD, we reveal the equivalence between it and the integrated-squared-errors (ISE) criterion based GKDE and by using this equivalence a MEB based classifier with privacy-preserving function is proposed for one kind of classification tasks where the datasets contain privacy-preserving clouds. For the FS, we show that the GKDE for a regression dataset is equivalent to the construction of a zero-order Takagi-Sugeno-Kang (TSK) fuzzy system based on the same dataset. Our extensive experiments confirm the obtained conclusions and demonstrated the effectiveness of the proposed new machine learning and modeling methods.
Original languageEnglish
Pages (from-to)119-137
Number of pages19
JournalInternational Journal of Machine Learning and Cybernetics
Volume4
Issue number2
DOIs
Publication statusPublished - 1 Apr 2013

Keywords

  • Bayesian classifier
  • Fuzzy system
  • Gaussian kernel density function
  • Kernel methods
  • Minimal enclosing ball
  • Privacy-preserving classifier
  • Support vector machine

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this