Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks

Chi Chung Cheung, Sin Chun Ng, Andrew Kwok Fai Lui

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

1 Citation (Scopus)

Abstract

Backpropagation (BP) learning algorithm is the most widely supervised learning technique which is extensively applied in the training of multi-layer feed-forward neural networks. Many modifications of BP have been proposed to speed up the learning of the original BP. However, they all have different drawbacks and they cannot perform very well in all kinds of applications. This paper proposes a new algorithm, which provides a systematic approach to make use of the characteristics of different fast learning algorithms so that the learning process can converge to the global minimum. During the training, different fast learning algorithms will be used in different phases to improve the global convergence capability. Our performance investigation shows that the proposed algorithm always converges in different benchmarking problems (applications) whereas other popular fast learning algorithms sometimes give very poor global convergence capabilities.

Original languageEnglish
Title of host publicationAdvances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings
Pages580-589
Number of pages10
EditionPART 1
DOIs
Publication statusPublished - 11 Jul 2012
Event9th International Symposium on Neural Networks, ISNN 2012 - Shenyang, China
Duration: 11 Jul 201214 Jul 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume7367 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference9th International Symposium on Neural Networks, ISNN 2012
Country/TerritoryChina
CityShenyang
Period11/07/1214/07/12

Keywords

  • backpropagation
  • local minimum
  • multi-phase learning algorithms

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this