BottleNet++: An end-to-end approach for feature compression in device-edge co-inference systems

The emergence of various intelligent mobile applications demands the deployment of powerful deep learning models at resource-constrained mobile devices. The device-edge co-inference framework provides a promising solution by splitting a neural network at a mobile device and an edge computing server. In order to balance the on-device computation and the communication overhead, the splitting point needs to be carefully picked, while the intermediate feature needs to be compressed before transmission. Existing studies decoupled the design of model splitting, feature compression, and communication, which may lead to excessive resource consumption of the mobile device. In this paper, we introduce an end-to-end architecture, named BottleNet++, that consists of an encoder, a non-trainable channel layer, and a decoder for more efficient feature compression and transmission. The encoder and decoder essentially implement joint source-channel coding via lightweight convolutional neural networks (CNNs), while explicitly considering the effect of channel noise. By exploiting the strong sparsity and the fault-tolerant property of the intermediate feature in deep neural network (DNNs), BottleNet++ achieves a much higher compression ratio than existing methods. Compared with merely transmitting intermediate data without feature compression, BottleNet++ achieves up to 64× bandwidth reduction over the additive white Gaussian noise channel and up to 256× bit compression ratio in the binary erasure channel, with less than 2% reduction in accuracy of classification.