On the improvement of the real time recurrent learning algorithm for recurrent neural networks

Man Wai Mak, K. W. Ku, Y. L. Lu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

35 Citations (Scopus)

Abstract

This paper reviews different approaches to improving the real time recurrent learning (RTRL) algorithm and attempts to group them into common frameworks. The characteristics of sub-grouping strategy, mode exchange RTRL, and cellular genetic algorithms are discussed. The relationships between these algorithms are highlighted and their time complexities and convergence capability are compared. The learning algorithms are applied to train recurrent neural networks in an attempt to solve a long-term dependency problem, to model the Henon map, and to predict the chaotic intensity pulsations of an NH3laser. The results show that the original RTRL algorithm achieves the lowest error among the gradient-based algorithms, but it requires the longest training time; whereas the sub-grouping strategy uses the shortest training time but its convergence capability is the poorest. The results also demonstrate that the cellular genetic algorithm is an alternative means of training recurrent neural networks when the gradient- based methods fail to find an acceptable solution.
Original languageEnglish
Pages (from-to)13-36
Number of pages24
JournalNeurocomputing
Volume24
Issue number1-3
DOIs
Publication statusPublished - 1 Feb 1999

Keywords

  • Cellular genetic algorithms
  • Chaotic series
  • Real-time recurrent learning
  • Recurrent neural networks

ASJC Scopus subject areas

  • Artificial Intelligence
  • Cellular and Molecular Neuroscience

Cite this