On-Edge Multi-Task Transfer Learning: Model and Practice with Data-Driven Task Allocation

Qiong Chen, Zimu Zheng, Chuang Hu, Dan Wang, Fangming Liu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

11 Citations (Scopus)

Abstract

On edge devices, data scarcity occurs as a common problem where transfer learning serves as a widely-suggested remedy. Nevertheless, transfer learning imposes heavy computation burden to the resource-constrained edge devices. Existing task allocation works usually assume all submitted tasks are equally important, leading to inefficient resource allocation at a task level when directly applied in Multi-task Transfer Learning (MTL). To address these issues, we first reveal that it is crucial to measure the impact of tasks on overall decision performance improvement and quantify task importance. We then show that task allocation with task importance for MTL (TATIM) is a variant of NP-complete Knapsack problem, where the complicated computation to solve this problem needs to be conducted repeatedly under varying contexts. To solve TATIM with high computational efficiency, we propose a Data-driven Cooperative Task Allocation (DCTA) approach. Finally, we evaluate the performance of DCTA by not only a trace-driven simulation, but also a new comprehensive real-world AIOps case study which bridges model and practice via a new architecture and main components design within AIOps system. Extensive experiments show that our DCTA reduces 3.24 times of processing time, and saves 48.4 percent energy consumption compared with the state-of-the-art when solving TATIM.

Original languageEnglish
Article number8943178
Pages (from-to)1357-1371
Number of pages15
JournalIEEE Transactions on Parallel and Distributed Systems
Volume31
Issue number6
DOIs
Publication statusPublished - 1 Jun 2020

Keywords

  • data-driven task allocation
  • Edge computing
  • real-world application
  • transfer learning

ASJC Scopus subject areas

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

Cite this