TY - GEN
T1 - A systematic algorithm to escape from local minima in training feed-forward neural networks
AU - Cheung, Chi Chung
AU - Xu, Sean Shensheng
AU - Ng, Sin Chun
PY - 2016/7/24
Y1 - 2016/7/24
N2 - A learning process is easily trapped into a local minimum when training multi-layer feed-forward neural networks. An algorithm called Wrong Output Modification (WOM) was proposed to help a learning process escape from local minima, but WOM still cannot totally solve the local minimum problem. Moreover, there is no performance analysis to show that the learning has a higher probability of converging to a global solution by using this algorithm. Additionally, the generalization performance of this algorithm was not investigated when the early stopping method of training is applied. Based on these limitations of WOM, we propose a new algorithm to ensure the learning can escape from local minima, and its performance is analyzed. We also test the generalization performance of this new algorithm when the early stopping method of training is applied.
AB - A learning process is easily trapped into a local minimum when training multi-layer feed-forward neural networks. An algorithm called Wrong Output Modification (WOM) was proposed to help a learning process escape from local minima, but WOM still cannot totally solve the local minimum problem. Moreover, there is no performance analysis to show that the learning has a higher probability of converging to a global solution by using this algorithm. Additionally, the generalization performance of this algorithm was not investigated when the early stopping method of training is applied. Based on these limitations of WOM, we propose a new algorithm to ensure the learning can escape from local minima, and its performance is analyzed. We also test the generalization performance of this new algorithm when the early stopping method of training is applied.
UR - http://www.scopus.com/inward/record.url?scp=85007207577&partnerID=8YFLogxK
U2 - 10.1109/IJCNN.2016.7727226
DO - 10.1109/IJCNN.2016.7727226
M3 - Conference article published in proceeding or book
AN - SCOPUS:85007207577
T3 - Proceedings of the International Joint Conference on Neural Networks
SP - 396
EP - 402
BT - 2016 International Joint Conference on Neural Networks, IJCNN 2016
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2016 International Joint Conference on Neural Networks, IJCNN 2016
Y2 - 24 July 2016 through 29 July 2016
ER -