Magnified gradient function with deterministic weight modification in adaptive learning

Sin Chun Ng, Chi Chung Cheung, Shu Hung Leung

Research output: Journal article publicationJournal articleAcademic researchpeer-review

69 Citations (Scopus)

Abstract

This paper presents two novel approaches, backpropagation (BP) with magnified gradient function (MGFPROP) and deterministic weight modification (DWM), to speed up the convergence rate and improve the global convergence capability of the standard BP learning algorithm. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function, while the main objective of DWM is to reduce the system error by changing the weights of a multilayered feedforward neural network in a deterministic way. Simulation results show that the performance of the above two approaches is better than BP and other modified BP algorithms for a number of learning problems. Moreover, the integration of the above two approaches forming a new algorithm called MDPROP, can further improve the performance of MGFPROP and DWM. From our simulation results, the MDPROP algorithm always outperforms BP and other modified BP algorithms in terms of convergence rate and global convergence capability.

Original languageEnglish
Pages (from-to)1411-1423
Number of pages13
JournalIEEE Transactions on Neural Networks
Volume15
Issue number6
DOIs
Publication statusPublished - 1 Nov 2004
Externally publishedYes

Keywords

  • Backpropagation (BP)
  • Neural networks

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this