On-Device Learning Systems for Edge Intelligence: A Software and Hardware Synergy Perspective

Qihua Zhou, Zhihao Qu, Song Guo, Boyuan Luo, Jingcai Guo, Zhenda Xu, Rajendra Akerkar

Research output: Journal article publicationReview articleAcademic researchpeer-review

16 Citations (Scopus)


Modern machine learning (ML) applications are often deployed in the cloud environment to exploit the computational power of clusters. However, this in-cloud computing scheme cannot satisfy the demands of emerging edge intelligence scenarios, including providing personalized models, protecting user privacy, adapting to real-time tasks, and saving resource cost. In order to conquer the limitations of conventional in-cloud computing, there comes the rise of on-device learning, which makes the end-to-end ML procedure totally on user devices, without unnecessary involvement of the cloud. In spite of the promising advantages of on-device learning, implementing a high-performance on-device learning system still faces with many severe challenges, such as insufficient user training data, backward propagation (BP) blocking, and limited peak processing speed. Observing the substantial improvement space in the implementation and acceleration of on-device learning systems, we intend to present a comprehensive analysis of the latest research progress and point out potential optimization directions from the system perspective. This survey presents a software and hardware synergy of on-device learning techniques, covering the scope of model-level neural network design, algorithm-level training optimization, and hardware-level instruction acceleration. We hope this survey could bring fruitful discussions and inspire the researchers to further promote the field of edge intelligence.

Original languageEnglish
Article number9366901
Pages (from-to)11916-11934
Number of pages19
JournalIEEE Internet of Things Journal
Issue number15
Publication statusPublished - 1 Aug 2021


  • Hardware acceleration
  • neural network design
  • on-device learning
  • system implementation
  • training algorithm optimization

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications


Dive into the research topics of 'On-Device Learning Systems for Edge Intelligence: A Software and Hardware Synergy Perspective'. Together they form a unique fingerprint.

Cite this