Protecting Decision Boundary of Machine Learning Model With Differentially Private Perturbation

Huadi Zheng, Qingqing Ye, Haibo Hu, Chengfang Fang, Jie Shi

Research output: Journal article publicationJournal articleAcademic researchpeer-review

12 Citations (Scopus)


Machine learning service API allows model owners to monetize proprietary models by offering prediction services to third-party users. However, existing literature shows that model parameters are vulnerable to extraction attacks which accumulate prediction queries and their responses to train a replica model. As countermeasures, researchers have proposed to reduce the rich API output, such as hiding the precise confidence. Nonetheless, even with response being only one bit, an adversary can still exploit fine-tuned queries with differential property to infer the decision boundary of the underlying model. In this article, we propose boundary differential privacy (BDP) against such attacks by obfuscating the prediction responses with noises. BDP guarantees an adversary cannot learn the decision boundary of any two classes by a predefined precision no matter how many queries are issued to the prediction API. We first design a perturbation algorithm called boundary randomized response for a binary model. Then we prove it satisfies ϵ-BDP, followed by a generalization of this algorithm to a multiclass model. Finally, we generalize a hard boundary to soft boundary and design an adaptive perturbation algorithm that can still work in the latter case. The effectiveness and high utility of our solution are verified by extensive experiments on both linear and non-linear models.

Original languageEnglish
Pages (from-to)2007-2022
Number of pages16
JournalIEEE Transactions on Dependable and Secure Computing
Issue number3
Publication statusPublished - May 2022


  • Adversarial machine learning
  • Boundary differential privacy
  • Model defense
  • Model extraction

ASJC Scopus subject areas

  • General Computer Science
  • Electrical and Electronic Engineering


Dive into the research topics of 'Protecting Decision Boundary of Machine Learning Model With Differentially Private Perturbation'. Together they form a unique fingerprint.

Cite this