Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection

Jing Xiao Liao, Bo Jian Hou, Hang Cheng Dong, Hao Zhang, Xiaoge Zhang, Jinwei Sun, Shiping Zhang, Feng Lei Fan

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Inspired by the complexity and diversity of biological neurons, a quadratic neuron is proposed to replace the inner product in the current neuron with a simplified quadratic function. Employing such a novel type of neurons offers a new perspective on developing deep learning. When analyzing quadratic neurons, we find that there exists a function such that a heterogeneous network can approximate it well with a polynomial number of neurons but a purely conventional or quadratic network needs an exponential number of neurons to achieve the same level of error. Encouraged by this inspiring theoretical result on heterogeneous networks, we directly integrate conventional and quadratic neurons in an autoencoder to make a new type of heterogeneous autoencoders. To our best knowledge, it is the first heterogeneous autoencoder that is made of different types of neurons. Next, we apply the proposed heterogeneous autoencoder to unsupervised anomaly detection for tabular data and bearing fault signals. The anomaly detection faces difficulties such as data unknownness, anomaly feature heterogeneity, and feature unnoticeability, which is suitable for the proposed heterogeneous autoencoder. Its high feature representation ability can characterize a variety of anomaly data (heterogeneity), discriminate the anomaly from the normal (unnoticeability), and accurately learn the distribution of normal samples (unknownness). Experiments show that heterogeneous autoencoders perform competitively compared to other state-of-the-art models.

Original languageEnglish
Pages (from-to)1-15
Number of pages15
JournalIEEE Transactions on Artificial Intelligence
DOIs
Publication statusPublished - 29 Apr 2024

Keywords

  • anomaly detection
  • Anomaly detection
  • Biological neural networks
  • Biological system modeling
  • Deep learning
  • Deep learning theory
  • heterogeneous autoencoder
  • Heterogeneous networks
  • Neurons
  • quadratic neuron
  • Task analysis

ASJC Scopus subject areas

  • Computer Science Applications
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection'. Together they form a unique fingerprint.

Cite this