Abstract
As the data processing demands have been increasing, different types of data processing systems are being developed. The new programming systems have different characteristics like types of data handled, processing technique and performance. However, multiple new systems have introduced difficulties for non-expert users like choosing the right system and usage methodology of the new systems. In order to relieve the burden of common users of conducting data processing tasks and taking relevant advantage of the systems features, we intend to integrate the popular programming systems and provide more efficient data processing services. In this paper, we propose to address the task scheduling problem for integrating multiple programming systems. We have designed a cluster based approach for task scheduling across multiple programming systems. This approach helps in minimizing the makespan of workflows and resource consumption. The simulation results show that the proposed approach can reduce the resource consumption significantly while achieving a low makespan for the workflows.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 15th International Symposium on Parallel and Distributed Computing, ISPDC 2016 |
| Publisher | IEEE |
| Pages | 222-229 |
| Number of pages | 8 |
| ISBN (Electronic) | 9781509041527 |
| DOIs | |
| Publication status | Published - 18 Apr 2017 |
| Event | 15th International Symposium on Parallel and Distributed Computing, ISPDC 2016 - Fuzhou, Fujian, China Duration: 8 Jul 2016 → 10 Jul 2016 |
Conference
| Conference | 15th International Symposium on Parallel and Distributed Computing, ISPDC 2016 |
|---|---|
| Country/Territory | China |
| City | Fuzhou, Fujian |
| Period | 8/07/16 → 10/07/16 |
Keywords
- Big data
- Cluster based
- Data processing systems
- Genetic algorithm
- Task scheduling
ASJC Scopus subject areas
- Computer Networks and Communications
- Hardware and Architecture
- Information Systems