Programming platforms for big data analysis

Jiannong Cao, Shailey Chawla, Yuqi Wang, Hanqing Wu

Research output: Chapter in book / Conference proceedingChapter in an edited book (as author)Academic researchpeer-review

2 Citations (Scopus)

Abstract

Big data analysis imposes new challenges and requirements on programming support. Programming platforms need to provide new abstractions and run time techniques with key features like scalability, fault tolerance, efficient task distribution, usability and processing speed. In this chapter, we first provide a comprehensive survey of the requirements, give an overviewand classify existing big data programming platforms based on different dimensions. Then, we present details of the architecture, methodology and features of major programming platforms like MapReduce, Storm, Spark, Pregel, GraphLab, etc. Last, we compare existing big data platforms, discuss the need for a unifying framework, present our proposed framework MatrixMap, and give a vision about future work.
Original languageEnglish
Title of host publicationHandbook of Big Data Technologies
PublisherSpringer International Publishing
Pages65-99
Number of pages35
ISBN (Electronic)9783319493404
ISBN (Print)9783319493398
DOIs
Publication statusPublished - 25 Feb 2017

Keywords

  • Big data analysis
  • Data parallel
  • Graph parallel
  • Programming platforms
  • Stream processing
  • Task parallel
  • Unifying framework

ASJC Scopus subject areas

  • Engineering(all)
  • Computer Science(all)
  • Economics, Econometrics and Finance(all)
  • Business, Management and Accounting(all)

Cite this