Renewable Energy-Aware Big Data Analytics in Geo-distributed Data Centers with Reinforcement Learning

Chenhan Xu, Kun Wang, Peng Li, Rui Xia, Song Guo, Minyi Guo

Research output: Journal article publicationJournal articleAcademic researchpeer-review

28 Citations (Scopus)

Abstract

In the age of big data, companies tend to deploy their services in data centers rather than their own servers. The demands of big data analytics grow significantly, which leads to an extremely high electricity consumption at data centers. In this paper, we investigate the cost minimization problem of big data analytics on geo-distributed data centers connected to renewable energy sources with unpredictable capacity. To solve this problem, we propose a Reinforcement Learning (RL) based job scheduling algorithm by combining RL with neural network (NN). Moreover, two techniques are developed to enhance the performance of our proposal. Specifically, Random Pool Sampling (RPS) is proposed to retrain the NN via accumulated training data, and a novel Unidirectional Bridge Network (UBN) structure is designed for further enhancing the training speed by using the historical knowledge stored in the trained NN. Experiment results on real Google cluster traces and electricity price from Energy Information Administration show that our approach is able to reduce the data centers' cost significantly compared with other benchmark algorithms.

Original languageEnglish
Article number8309283
Pages (from-to)205-215
Number of pages11
JournalIEEE Transactions on Network Science and Engineering
Volume7
Issue number1
DOIs
Publication statusPublished - 1 Jan 2020

Keywords

  • Artificial neural networks
  • Big Data
  • Big data
  • data center
  • Data centers
  • Energy consumption
  • Green products
  • load balancing
  • reinforcement learning
  • Renewable energy sources
  • Scheduling

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Computer Networks and Communications

Cite this