Programming platforms for big data analysis

Jiannong Cao; Shailey Chawla; Yuqi Wang; Hanqing Wu

doi:10.1007/978-3-319-49340-4_3

Programming platforms for big data analysis

Jiannong Cao, Shailey Chawla, Yuqi Wang, Hanqing Wu

Research output: Chapter in book / Conference proceeding › Chapter in an edited book (as author) › Academic research › peer-review

2 Citations (Scopus)

Abstract

Big data analysis imposes new challenges and requirements on programming support. Programming platforms need to provide new abstractions and run time techniques with key features like scalability, fault tolerance, efficient task distribution, usability and processing speed. In this chapter, we first provide a comprehensive survey of the requirements, give an overviewand classify existing big data programming platforms based on different dimensions. Then, we present details of the architecture, methodology and features of major programming platforms like MapReduce, Storm, Spark, Pregel, GraphLab, etc. Last, we compare existing big data platforms, discuss the need for a unifying framework, present our proposed framework MatrixMap, and give a vision about future work.

Original language	English
Title of host publication	Handbook of Big Data Technologies
Publisher	Springer International Publishing
Pages	65-99
Number of pages	35
ISBN (Electronic)	9783319493404
ISBN (Print)	9783319493398
DOIs	https://doi.org/10.1007/978-3-319-49340-4_3
Publication status	Published - 25 Feb 2017

Keywords

Big data analysis
Data parallel
Graph parallel
Programming platforms
Stream processing
Task parallel
Unifying framework

ASJC Scopus subject areas

Engineering(all)
Computer Science(all)
Economics, Econometrics and Finance(all)
Business, Management and Accounting(all)

More information

10.1007/978-3-319-49340-4_3

Cite this

@inbook{b9fc30e103f649a9bbb1b455793a1335,

title = "Programming platforms for big data analysis",

abstract = "Big data analysis imposes new challenges and requirements on programming support. Programming platforms need to provide new abstractions and run time techniques with key features like scalability, fault tolerance, efficient task distribution, usability and processing speed. In this chapter, we first provide a comprehensive survey of the requirements, give an overviewand classify existing big data programming platforms based on different dimensions. Then, we present details of the architecture, methodology and features of major programming platforms like MapReduce, Storm, Spark, Pregel, GraphLab, etc. Last, we compare existing big data platforms, discuss the need for a unifying framework, present our proposed framework MatrixMap, and give a vision about future work.",

keywords = "Big data analysis, Data parallel, Graph parallel, Programming platforms, Stream processing, Task parallel, Unifying framework",

author = "Jiannong Cao and Shailey Chawla and Yuqi Wang and Hanqing Wu",

year = "2017",

month = feb,

day = "25",

doi = "10.1007/978-3-319-49340-4_3",

language = "English",

isbn = "9783319493398",

pages = "65--99",

booktitle = "Handbook of Big Data Technologies",

publisher = "Springer International Publishing",

}

TY - CHAP

T1 - Programming platforms for big data analysis

AU - Cao, Jiannong

AU - Chawla, Shailey

AU - Wang, Yuqi

AU - Wu, Hanqing

PY - 2017/2/25

Y1 - 2017/2/25

N2 - Big data analysis imposes new challenges and requirements on programming support. Programming platforms need to provide new abstractions and run time techniques with key features like scalability, fault tolerance, efficient task distribution, usability and processing speed. In this chapter, we first provide a comprehensive survey of the requirements, give an overviewand classify existing big data programming platforms based on different dimensions. Then, we present details of the architecture, methodology and features of major programming platforms like MapReduce, Storm, Spark, Pregel, GraphLab, etc. Last, we compare existing big data platforms, discuss the need for a unifying framework, present our proposed framework MatrixMap, and give a vision about future work.

AB - Big data analysis imposes new challenges and requirements on programming support. Programming platforms need to provide new abstractions and run time techniques with key features like scalability, fault tolerance, efficient task distribution, usability and processing speed. In this chapter, we first provide a comprehensive survey of the requirements, give an overviewand classify existing big data programming platforms based on different dimensions. Then, we present details of the architecture, methodology and features of major programming platforms like MapReduce, Storm, Spark, Pregel, GraphLab, etc. Last, we compare existing big data platforms, discuss the need for a unifying framework, present our proposed framework MatrixMap, and give a vision about future work.

KW - Big data analysis

KW - Data parallel

KW - Graph parallel

KW - Programming platforms

KW - Stream processing

KW - Task parallel

KW - Unifying framework

UR - http://www.scopus.com/inward/record.url?scp=85019874072&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-49340-4_3

DO - 10.1007/978-3-319-49340-4_3

M3 - Chapter in an edited book (as author)

SN - 9783319493398

SP - 65

EP - 99

BT - Handbook of Big Data Technologies

PB - Springer International Publishing

ER -

Programming platforms for big data analysis

Abstract

Keywords

ASJC Scopus subject areas

More information

Other files and links

Fingerprint

Cite this