BDPL: A Boundary Differentially Private Layer Against Machine Learning Model Extraction Attacks

Huadi Zheng, Qingqing Ye, Haibo Hu, Chengfang Fang, Jie Shi

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

12 Citations (Scopus)


Machine learning models trained by large volume of proprietary data and intensive computational resources are valuable assets of their owners, who merchandise these models to third-party users through prediction service API. However, existing literature shows that model parameters are vulnerable to extraction attacks which accumulate a large number of prediction queries and their responses to train a replica model. As countermeasures, researchers have proposed to reduce the rich API output, such as hiding the precise confidence level of the prediction response. Nonetheless, even with response being only one bit, an adversary can still exploit fine-tuned queries with differential property to infer the decision boundary of the underlying model. In this paper, we propose boundary differential privacy (ϵ -BDP) as a solution to protect against such attacks by obfuscating the prediction responses near the decision boundary. ϵ -BDP guarantees an adversary cannot learn the decision boundary by a predefined precision no matter how many queries are issued to the prediction API. We design and prove a perturbation algorithm called boundary randomized response that can achieve ϵ -BDP. The effectiveness and high utility of our solution against model extraction attacks are verified by extensive experiments on both linear and non-linear models.

Original languageEnglish
Title of host publicationComputer Security – ESORICS 2019 - 24th European Symposium on Research in Computer Security, Proceedings
EditorsKazue Sako, Steve Schneider, Peter Y.A. Ryan
Number of pages18
ISBN (Print)9783030299583
Publication statusPublished - 23 Sep 2019
Event24th European Symposium on Research in Computer Security, ESORICS 2019 - Luxembourg, Luxembourg
Duration: 23 Sep 201927 Sep 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11735 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference24th European Symposium on Research in Computer Security, ESORICS 2019

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this