Accelerating convolutional neural networks using fine-tuned backpropagation progress

Yulong Li, Zhenhong Chen, Yi Cai, Dongping Huang, Qing Li

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

1 Citation (Scopus)

Abstract

In computer vision many tasks have achieved state-of-the-art performance using convolutional neural networks (CNNs) [11], typically at the cost of massive computational complexity. A key problem of the training is the low speed of the progress. It may cost much time especially when computational resources are limited. The focus of this paper is speeding up the training progress based on fine-tuned backpropagation progress. More specifically, we train the CNNs with standard backpropagation firstly. When the feature extraction layers got better features, then we start to block the standard backpropagation in the whole layers, the loss function values only back propagates between fully connected layers. So it can not only save time but also pay more attention to train the classifier to get the same or better result compared with training with standard backpropagation all the time. Comprehensive experiments on JD (https://www.jd.com/) datasets demonstrate significant reduction in computational time, at the cost of negligible loss in accuracy.

Original languageEnglish
Title of host publicationDatabase Systems for Advanced Applications - DASFAA 2017 International Workshops
Subtitle of host publicationBDMS, BDQM, SeCoP, and DMMOOC, Proceedings
EditorsLijun Chang, Goce Trajcevski, Wen Hua, Zhifeng Bao
PublisherSpringer-Verlag
Pages256-266
Number of pages11
ISBN (Print)9783319557045
DOIs
Publication statusPublished - 1 Jan 2017
Externally publishedYes
EventInternational Workshops on Database Systems for Advanced Applications, DASFAA 2017, 4th International Workshop on Big Data Management and Service, BDMS 2017, 2nd Workshop on Big Data Quality Management, BDQM 2017, 4th International Workshop on Semantic Computing and Personalization, SeCoP 2017, 1st International Workshop on Data Management and Mining on MOOCs, DMMOOC 2017 - Suzhou, China
Duration: 27 Mar 201730 Mar 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10179 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Workshops on Database Systems for Advanced Applications, DASFAA 2017, 4th International Workshop on Big Data Management and Service, BDMS 2017, 2nd Workshop on Big Data Quality Management, BDQM 2017, 4th International Workshop on Semantic Computing and Personalization, SeCoP 2017, 1st International Workshop on Data Management and Mining on MOOCs, DMMOOC 2017
Country/TerritoryChina
CitySuzhou
Period27/03/1730/03/17

Keywords

  • Acceleration
  • Backpropagation
  • Convolutional Neural Networks

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this