A study of the lamarckian evolution of recurrent neural networks

Kim W.C. Ku, Man Wai Mak, Wan Chi Siu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

34 Citations (Scopus)

Abstract

Many frustrating experiences have been encountered when the training of neural networks by local search methods becomes stagnant at local optima. This calls for the development of more satisfactory search methods such as evolutionary search. However, training by evolutionary search can require a long computation time. In certain situations, using Lamarckian evolution, local search and evolutionary search can complement each other to yield a better training algorithm. This paper demonstrates the potential of this evolutionary-learning synergy by applying it to train recurrent neural networks in an attempt to resolve a long-term dependency problem and the inverted pendulum problem. This work also aims at investigating the interaction between local search and evolutionary search when they are combined. It is found that the combinations are particularly efficient when the local search is simple. In the case where no teacher signal is available for the local search to learn the desired task directly, the paper proposes introducing a related local task for the local search to learn, and finds that this approach is able to reduce the training time considerably.
Original languageEnglish
Pages (from-to)31-41
Number of pages11
JournalIEEE Transactions on Evolutionary Computation
Volume4
Issue number1
DOIs
Publication statusPublished - 1 Dec 2000

Keywords

  • Evolutionary computation
  • Lamarckian evolution
  • Recurrent neural networks

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Software
  • Computational Theory and Mathematics

Cite this