A disk based stream oriented approach for storing big data

Peter Membrey; Chun Chung Chan; Yuri Demchenko

doi:10.1109/CTS.2013.6567204

A disk based stream oriented approach for storing big data

Peter Membrey, Chun Chung Chan, Yuri Demchenko

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

6 Citations (Scopus)

Abstract

This paper proposes an extension to the generally accepted definition of Big Data and from this extended definition proposes a specialized database design for storing high throughput data from low-latency sources. It discusses the challenges a financial company faces with regards to processing and storing data and how existing database technologies are unsuitable for this niche task. A prototype database called CakeDB is built using a stream oriented, disk based storage design and insert throughput tests are conducted to demonstrate how effectively such a design would handle high throughput data as per the use case.

Original language	English
Title of host publication	Proceedings of the 2013 International Conference on Collaboration Technologies and Systems, CTS 2013
Pages	56-64
Number of pages	9
DOIs	https://doi.org/10.1109/CTS.2013.6567204
Publication status	Published - 5 Sept 2013
Event	2013 International Conference on Collaboration Technologies and Systems, CTS 2013 - San Diego, CA, United States Duration: 20 May 2013 → 24 May 2013

Conference

Conference	2013 International Conference on Collaboration Technologies and Systems, CTS 2013
Country/Territory	United States
City	San Diego, CA
Period	20/05/13 → 24/05/13

ASJC Scopus subject areas

Computer Networks and Communications
Control and Systems Engineering

More information

10.1109/CTS.2013.6567204

Cite this

@inproceedings{5b1334ac2281408ead9dce234c4dcbcd,

title = "A disk based stream oriented approach for storing big data",

abstract = "This paper proposes an extension to the generally accepted definition of Big Data and from this extended definition proposes a specialized database design for storing high throughput data from low-latency sources. It discusses the challenges a financial company faces with regards to processing and storing data and how existing database technologies are unsuitable for this niche task. A prototype database called CakeDB is built using a stream oriented, disk based storage design and insert throughput tests are conducted to demonstrate how effectively such a design would handle high throughput data as per the use case.",

author = "Peter Membrey and Chan, {Chun Chung} and Yuri Demchenko",

year = "2013",

month = sep,

day = "5",

doi = "10.1109/CTS.2013.6567204",

language = "English",

isbn = "9781467364027",

pages = "56--64",

booktitle = "Proceedings of the 2013 International Conference on Collaboration Technologies and Systems, CTS 2013",

note = "2013 International Conference on Collaboration Technologies and Systems, CTS 2013 ; Conference date: 20-05-2013 Through 24-05-2013",

}

Membrey, P, Chan, CC & Demchenko, Y 2013, A disk based stream oriented approach for storing big data. in Proceedings of the 2013 International Conference on Collaboration Technologies and Systems, CTS 2013., 6567204, pp. 56-64, 2013 International Conference on Collaboration Technologies and Systems, CTS 2013, San Diego, CA, United States, 20/05/13. https://doi.org/10.1109/CTS.2013.6567204

A disk based stream oriented approach for storing big data. / Membrey, Peter; Chan, Chun Chung; Demchenko, Yuri.
Proceedings of the 2013 International Conference on Collaboration Technologies and Systems, CTS 2013. 2013. p. 56-64 6567204.

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - A disk based stream oriented approach for storing big data

AU - Membrey, Peter

AU - Chan, Chun Chung

AU - Demchenko, Yuri

PY - 2013/9/5

Y1 - 2013/9/5

N2 - This paper proposes an extension to the generally accepted definition of Big Data and from this extended definition proposes a specialized database design for storing high throughput data from low-latency sources. It discusses the challenges a financial company faces with regards to processing and storing data and how existing database technologies are unsuitable for this niche task. A prototype database called CakeDB is built using a stream oriented, disk based storage design and insert throughput tests are conducted to demonstrate how effectively such a design would handle high throughput data as per the use case.

AB - This paper proposes an extension to the generally accepted definition of Big Data and from this extended definition proposes a specialized database design for storing high throughput data from low-latency sources. It discusses the challenges a financial company faces with regards to processing and storing data and how existing database technologies are unsuitable for this niche task. A prototype database called CakeDB is built using a stream oriented, disk based storage design and insert throughput tests are conducted to demonstrate how effectively such a design would handle high throughput data as per the use case.

UR - http://www.scopus.com/inward/record.url?scp=84883274580&partnerID=8YFLogxK

U2 - 10.1109/CTS.2013.6567204

DO - 10.1109/CTS.2013.6567204

M3 - Conference article published in proceeding or book

SN - 9781467364027

SP - 56

EP - 64

BT - Proceedings of the 2013 International Conference on Collaboration Technologies and Systems, CTS 2013

T2 - 2013 International Conference on Collaboration Technologies and Systems, CTS 2013

Y2 - 20 May 2013 through 24 May 2013

ER -

A disk based stream oriented approach for storing big data

Abstract

Conference

ASJC Scopus subject areas

More information

Other files and links

Fingerprint

Cite this