Abstract
In the age of big data, companies tend to deploy their services in data centers rather than their own servers. The demands of big data analytics grow significantly, which leads to an extremely high electricity consumption at data centers. In this paper, we investigate the cost minimization problem of big data analytics on geo-distributed data centers connected to renewable energy sources with unpredictable capacity. To solve this problem, we propose a Reinforcement Learning (RL) based job scheduling algorithm by combining RL with neural network (NN). Moreover, two techniques are developed to enhance the performance of our proposal. Specifically, Random Pool Sampling (RPS) is proposed to retrain the NN via accumulated training data, and a novel Unidirectional Bridge Network (UBN) structure is designed for further enhancing the training speed by using the historical knowledge stored in the trained NN. Experiment results on real Google cluster traces and electricity price from Energy Information Administration show that our approach is able to reduce the data centers' cost significantly compared with other benchmark algorithms.
Original language | English |
---|---|
Article number | 8309283 |
Pages (from-to) | 205-215 |
Number of pages | 11 |
Journal | IEEE Transactions on Network Science and Engineering |
Volume | 7 |
Issue number | 1 |
DOIs | |
Publication status | Published - 1 Jan 2020 |
Keywords
- Artificial neural networks
- Big Data
- Big data
- data center
- Data centers
- Energy consumption
- Green products
- load balancing
- reinforcement learning
- Renewable energy sources
- Scheduling
ASJC Scopus subject areas
- Control and Systems Engineering
- Computer Science Applications
- Computer Networks and Communications