Locality-sensitive bloom filter for approximate membership query

Yu Hua, Bin Xiao, Bharadwaj Veeravalli, Dan Feng

Research output: Journal article publicationJournal articleAcademic researchpeer-review

46 Citations (Scopus)

Abstract

In many network applications, Bloom filters are used to support exact-matching membership query for their randomized space-efficient data structure with a small probability of false answers. In this paper, we extend the standard Bloom filter to Locality-Sensitive Bloom Filter (LSBF) to provide Approximate Membership Query (AMQ) service. We achieve this by replacing uniform and independent hash functions with locality-sensitive hash functions. Such replacement makes the storage in LSBF to be locality sensitive. Meanwhile, LSBF is space efficient and query responsive by employing the Bloom filter design. In the design of the LSBF structure, we propose a bit vector to reduce False Positives (FP). The bit vector can verify multiple attributes belonging to one member. We also use an active overflowed scheme to significantly decrease False Negatives (FN). Rigorous theoretical analysis (e.g., on FP, FN, and space overhead) shows that the design of LSBF is space compact and can provide accurate response to approximate membership queries. We have implemented LSBF in a real distributed system to perform extensive experiments using real-world traces. Experimental results show that LSBF, compared with a baseline approach and other state-of-the-art work in the literature (SmartStore and LSB-tree), takes less time to respond AMQ and consumes much less storage space.
Original languageEnglish
Article number5928322
Pages (from-to)817-830
Number of pages14
JournalIEEE Transactions on Computers
Volume61
Issue number6
DOIs
Publication statusPublished - 8 May 2012

Keywords

  • Approximate membership query
  • bloom filters
  • locality sensitive hashing

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computational Theory and Mathematics

Cite this