BottleNet++: An end-to-end approach for feature compression in device-edge co-inference systems

Jiawei Shao, Jun Zhang

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

5 Citations (Scopus)

Abstract

The emergence of various intelligent mobile applications demands the deployment of powerful deep learning models at resource-constrained mobile devices. The device-edge co-inference framework provides a promising solution by splitting a neural network at a mobile device and an edge computing server. In order to balance the on-device computation and the communication overhead, the splitting point needs to be carefully picked, while the intermediate feature needs to be compressed before transmission. Existing studies decoupled the design of model splitting, feature compression, and communication, which may lead to excessive resource consumption of the mobile device. In this paper, we introduce an end-to-end architecture, named BottleNet++, that consists of an encoder, a non-trainable channel layer, and a decoder for more efficient feature compression and transmission. The encoder and decoder essentially implement joint source-channel coding via lightweight convolutional neural networks (CNNs), while explicitly considering the effect of channel noise. By exploiting the strong sparsity and the fault-tolerant property of the intermediate feature in deep neural network (DNNs), BottleNet++ achieves a much higher compression ratio than existing methods. Compared with merely transmitting intermediate data without feature compression, BottleNet++ achieves up to 64× bandwidth reduction over the additive white Gaussian noise channel and up to 256× bit compression ratio in the binary erasure channel, with less than 2% reduction in accuracy of classification.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Communications Workshops, ICC Workshops 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728174402
DOIs
Publication statusPublished - Jun 2020
Event2020 IEEE International Conference on Communications Workshops, ICC Workshops 2020 - Dublin, Ireland
Duration: 7 Jun 202011 Jun 2020

Publication series

Name2020 IEEE International Conference on Communications Workshops, ICC Workshops 2020 - Proceedings

Conference

Conference2020 IEEE International Conference on Communications Workshops, ICC Workshops 2020
CountryIreland
CityDublin
Period7/06/2011/06/20

Keywords

  • Deep Learning
  • Device-Edge Co-Inference
  • Joint Source-Chanel Coding
  • Network Compression

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Signal Processing
  • Information Systems and Management
  • Control and Optimization

Cite this