Investigation on Performance of Neural Networks Using Quadratic Relative Error Cost Function

Ning Zhang, Shui Long Shen, Annan Zhou, Ye Shuang Xu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

60 Citations (Scopus)

Abstract

The performance of neural networks with quadratic cost function (MSE cost function) is analyzed in terms of the adjustment rate of weights and its performance in multi-magnitude data processing using a qualitative mathematical method based on mean squared error. However, neural networks using quadratic cost functions exhibit low-weight updating rates and variations in performances in multi-magnitude data processing. This paper investigates the performance of neural networks using a quadratic relative error cost function (REMSE cost function). Two-node-to-one-node models are built to investigate the performance of the REMSE and MSE cost functions in adjustment rate of weights and multi-magnitude data processing. A three-layer neural network is employed to compare the training and prediction performances of the REMSE cost function and MSE cost function. Three LSTM networks are used to evaluate the differences between REMSE, MSE, and Logcosh in actual applications by learning stress and strain of soil. The results indicate that the REMSE cost function can notably accelerate the adjustment rate of weights and improve the performance of the neural network in small magnitude data regression. The applications of the REMSE cost function are also discussed.

Original languageEnglish
Article number8769865
Pages (from-to)106642-106652
Number of pages11
JournalIEEE Access
Volume7
DOIs
Publication statusPublished - 2019
Externally publishedYes

Keywords

  • cost function
  • mean square error methods
  • neural networks
  • Optimization
  • soil
  • strain
  • stress

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering

Fingerprint

Dive into the research topics of 'Investigation on Performance of Neural Networks Using Quadratic Relative Error Cost Function'. Together they form a unique fingerprint.

Cite this