Resource-Constrained Edge AI with Early Exit Prediction

Rongkang Dong, Yuyi Mao, Jun Zhang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

4 Citations (Scopus)


By leveraging the data sample diversity, the early-exit network recently emerges as a prominent neural network architecture to accelerate the deep learning inference process. However, intermediate classifiers of the early exits introduce additional computation overhead, which is unfavorable for resource-constrained edge artificial intelligence (AI). In this paper, we propose an early exit prediction mechanism to reduce the on-device computation overhead in a device-edge co-inference system supported by early-exit networks. Specifically, we design a low-complexity module, namely the exit predic-tor, to guide some distinctly “hard” samples to bypass the computation of the early exits. Besides, considering the varying communication bandwidth, we extend the early exit prediction mechanism for latency-aware edge inference, which adapts the prediction thresholds of the exit predictor and the confidence thresholds of the early-exit network via a few simple regression models. Extensive experiment results demonstrate the effective-ness of the exit predictor in achieving a better tradeoff between accuracy and on-device computation overhead for early-exit networks. Besides, compared with the baseline methods, the proposed method for latency-aware edge inference attains higher inference accuracy under different bandwidth conditions.

Original languageEnglish
Pages (from-to)122-134
Number of pages13
JournalJournal of Communications and Information Networks
Issue number2
Publication statusPublished - Jun 2022


  • artificial intelligence (AI)
  • device-edge cooperative inference
  • early exit prediction
  • early-exit network
  • edge AI

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Resource-Constrained Edge AI with Early Exit Prediction'. Together they form a unique fingerprint.

Cite this