Solving the local minimum and flat-spot problem by modifying wrong outputs for feed-forward neural networks

Chi Chung Cheung, Andrew K. Lui, Sean Shensheng Xu

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

4 Citations (Scopus)

Abstract

Backpropagation (BP) algorithm, which is very popular in supervised learning, is extensively applied in training feed-forward neural networks. Many modifications have been proposed to speed up the convergence process of the standard BP algorithm. However, they seldom focus on improving the global convergence capability. This paper proposes a new algorithm called Wrong Output Modification (WOM) to improve the global convergence capability of a fast learning algorithm. When a learning process is trapped by a local minimum or a flat-spot area, this algorithm looks for some outputs that go to other extremes when compared with their target outputs, and then it modifies such outputs systemically so that they can get close to their target outputs and hence some weights of neurons are changed accordingly. It is hoped that these changes make the learning process escape from such local minima or flat-spot areas and then converge. The performance investigation shows that the proposed algorithm can be applied into different fast learning algorithms, and their global convergence capabilities are improved significantly compared with their original algorithms. Moreover, some statistical data obtained from this algorithm can be used to identify the difficulty of a learning problem.

Original languageEnglish
Title of host publication2013 International Joint Conference on Neural Networks, IJCNN 2013
DOIs
Publication statusPublished - 4 Aug 2013
Event2013 International Joint Conference on Neural Networks, IJCNN 2013 - Dallas, TX, United States
Duration: 4 Aug 20139 Aug 2013

Publication series

NameProceedings of the International Joint Conference on Neural Networks

Conference

Conference2013 International Joint Conference on Neural Networks, IJCNN 2013
Country/TerritoryUnited States
CityDallas, TX
Period4/08/139/08/13

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this