Fast Convergence for Back-Propagation Network with Magnified Gradient Function

S. C. Ng; C. C. Cheung; S. H. Leung; A. Luk

Fast Convergence for Back-Propagation Network with Magnified Gradient Function

S. C. Ng, C. C. Cheung, S. H. Leung, A. Luk

The Hong Kong Polytechnic University

Research output: Unpublished conference presentation (presented paper, abstract, poster) › Conference presentation (not published in journal/proceeding/book) › Academic research › peer-review

33 Citations (Scopus)

Abstract

This paper presents a modified back-propagation algorithm using magnified gradient function (MGFPROP), which can effectively speed up the convergence rate and improve the global convergence capability of back-propagation. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. From the convergence analysis, it is shown that the new algorithm retains the gradient descent property but gives faster convergence than that of the back-propagation algorithm. Simulation results show that, in terms of the convergence rate and the percentage of global convergence, the new algorithm always outperforms the standard back-propagation algorithm and other competing techniques.

Original language	English
Pages	1903-1908
Number of pages	6
Publication status	Published - 20 Jul 2003
Event	International Joint Conference on Neural Networks 2003 - Portland, OR, United States Duration: 20 Jul 2003 → 24 Jul 2003

Conference

Conference	International Joint Conference on Neural Networks 2003
Country/Territory	United States
City	Portland, OR
Period	20/07/03 → 24/07/03

ASJC Scopus subject areas

Software
Artificial Intelligence

Cite this

@conference{7bc14c7e41d54b3a8598fe1e3217d5ba,

title = "Fast Convergence for Back-Propagation Network with Magnified Gradient Function",

abstract = "This paper presents a modified back-propagation algorithm using magnified gradient function (MGFPROP), which can effectively speed up the convergence rate and improve the global convergence capability of back-propagation. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. From the convergence analysis, it is shown that the new algorithm retains the gradient descent property but gives faster convergence than that of the back-propagation algorithm. Simulation results show that, in terms of the convergence rate and the percentage of global convergence, the new algorithm always outperforms the standard back-propagation algorithm and other competing techniques.",

author = "Ng, {S. C.} and Cheung, {C. C.} and Leung, {S. H.} and A. Luk",

year = "2003",

month = jul,

day = "20",

language = "English",

pages = "1903--1908",

note = "International Joint Conference on Neural Networks 2003 ; Conference date: 20-07-2003 Through 24-07-2003",

}

Fast Convergence for Back-Propagation Network with Magnified Gradient Function. / Ng, S. C.; Cheung, C. C.; Leung, S. H. et al.
2003. 1903-1908 Paper presented at International Joint Conference on Neural Networks 2003, Portland, OR, United States.

Research output: Unpublished conference presentation (presented paper, abstract, poster) › Conference presentation (not published in journal/proceeding/book) › Academic research › peer-review

TY - CONF

T1 - Fast Convergence for Back-Propagation Network with Magnified Gradient Function

AU - Ng, S. C.

AU - Cheung, C. C.

AU - Leung, S. H.

AU - Luk, A.

PY - 2003/7/20

Y1 - 2003/7/20

N2 - This paper presents a modified back-propagation algorithm using magnified gradient function (MGFPROP), which can effectively speed up the convergence rate and improve the global convergence capability of back-propagation. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. From the convergence analysis, it is shown that the new algorithm retains the gradient descent property but gives faster convergence than that of the back-propagation algorithm. Simulation results show that, in terms of the convergence rate and the percentage of global convergence, the new algorithm always outperforms the standard back-propagation algorithm and other competing techniques.

AB - This paper presents a modified back-propagation algorithm using magnified gradient function (MGFPROP), which can effectively speed up the convergence rate and improve the global convergence capability of back-propagation. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. From the convergence analysis, it is shown that the new algorithm retains the gradient descent property but gives faster convergence than that of the back-propagation algorithm. Simulation results show that, in terms of the convergence rate and the percentage of global convergence, the new algorithm always outperforms the standard back-propagation algorithm and other competing techniques.

UR - http://www.scopus.com/inward/record.url?scp=0141459829&partnerID=8YFLogxK

M3 - Conference presentation (not published in journal/proceeding/book)

AN - SCOPUS:0141459829

SP - 1903

EP - 1908

T2 - International Joint Conference on Neural Networks 2003

Y2 - 20 July 2003 through 24 July 2003

ER -

Fast Convergence for Back-Propagation Network with Magnified Gradient Function

Abstract

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this