A disk based stream oriented approach for storing big data

Peter Membrey, Chun Chung Chan, Yuri Demchenko

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

4 Citations (Scopus)

Abstract

This paper proposes an extension to the generally accepted definition of Big Data and from this extended definition proposes a specialized database design for storing high throughput data from low-latency sources. It discusses the challenges a financial company faces with regards to processing and storing data and how existing database technologies are unsuitable for this niche task. A prototype database called CakeDB is built using a stream oriented, disk based storage design and insert throughput tests are conducted to demonstrate how effectively such a design would handle high throughput data as per the use case.
Original languageEnglish
Title of host publicationProceedings of the 2013 International Conference on Collaboration Technologies and Systems, CTS 2013
Pages56-64
Number of pages9
DOIs
Publication statusPublished - 5 Sep 2013
Event2013 International Conference on Collaboration Technologies and Systems, CTS 2013 - San Diego, CA, United States
Duration: 20 May 201324 May 2013

Conference

Conference2013 International Conference on Collaboration Technologies and Systems, CTS 2013
CountryUnited States
CitySan Diego, CA
Period20/05/1324/05/13

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Control and Systems Engineering

Cite this