Bejo: Behavior based job classification for resource consumption prediction in the cloud

Lin Xu, Jiannong Cao, Yan Wang, Lei Yang, Jing Li

Research output: Journal article publicationConference articleAcademic researchpeer-review

Abstract

Resource prediction (e.g. CPU/memory utilization) of cloud computing jobs has attracted substantial amount of attention. Existing works use regression methods based on historical information of jobs, with an impractical assumption that the job to be predicted has the same class as the historical jobs. To address this problem, we propose to take the category of the jobs into consideration for effective resource prediction. Existing works on job classification either ignores the temporal variance of resource consumption during job execution or use it in a naive way, resulting in unsatisfactory classification accuracy and/or slow speed. In this paper, we introduce a new and efficient job classification approach, called Bejo. Inspired by the textual document classification methods, which use distribution of text words to describe and classify a document, Bejo treats the job as a document, assigns each collected resource consumption snapshot to a certain 'resource word', and uses the distribution of the words to describe and classify a job. An l1 norm minimization formulation is used to assign each resource snapshot to a resource word, to especially address the unique challenges of high noise and tight time budget of cloud job classification. We collect a comprehensive dataset for job classification and resource consumption prediction on cloud platforms, and demonstrate superior quality and efficiency of Bejo over state-of-the-art algorithms. Experiments also show the relative error of resource consumption prediction can be dramatically reduced by adding an extra job classification step to the existing regression methods.
Original languageEnglish
Article number7037642
Pages (from-to)10-17
Number of pages8
JournalProceedings of the International Conference on Cloud Computing Technology and Science, CloudCom
Volume2015-February
Issue numberFebruary
DOIs
Publication statusPublished - 1 Jan 2015
Event2014 6th IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2014 - Singapore, Singapore
Duration: 15 Dec 201418 Dec 2014

Keywords

  • Bag of words (BoW)
  • Job classification
  • Resource consumption pattern
  • Resource prediction
  • Sparse coding

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Networks and Communications
  • Software
  • Theoretical Computer Science

Cite this