Further enhancements in WOM algorithm to solve the local minimum and flat-spot problem in feed-forward neural networks

Chi Chung Cheung, Sin Chun Ng, Andrew Lui, Sean Shensheng Xu

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

4 Citations (Scopus)

Abstract

Backpropagation (BP) algorithm is very popular in supervised learning for feed-forward neural networks. However, it is sometimes slow and easily trapped into a local minimum or a flat-spot area (known as the local minimum and flat-spot area problem respectively). Many modifications have been proposed to speed up its convergence rate but they seldom improve the global convergence capability. Some fast learning algorithms have been proposed recently to solve these two problems: Wrong Output Modification (WOM) is one new algorithm that can improve the global convergence capability significantly. However, some limitations exist in WOM so that it cannot solve the local minimum and flat-spot problem effectively. In this paper, some enhancements are proposed to further improve the performance of WOM by (a) changing the mechanism to escape from a local minimum or a flat-spot area and (b) adding a fast checking procedure to identify the existence of a local minimum or a flat-spot area. The performance investigation shows that the proposed enhancements can improve the performance of WOM significantly when it is applied into different fast learning algorithms. Moreover, WOM with these enhancements is also applied to a very popular second-order gradient descent learning algorithm, Levenberg-Marquardt (LM) algorithm. The performance investigation shows that it can significantly improve the performance of LM.

Original languageEnglish
Title of host publicationProceedings of the International Joint Conference on Neural Networks
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1225-1230
Number of pages6
ISBN (Electronic)9781479914845
DOIs
Publication statusPublished - 6 Jul 2014
Event2014 International Joint Conference on Neural Networks, IJCNN 2014 - Beijing, China
Duration: 6 Jul 201411 Jul 2014

Publication series

NameProceedings of the International Joint Conference on Neural Networks

Conference

Conference2014 International Joint Conference on Neural Networks, IJCNN 2014
Country/TerritoryChina
CityBeijing
Period6/07/1411/07/14

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this