Protecting Decision Boundary of Machine Learning Model With Differentially Private Perturbation

Huadi Zheng, Qingqing Ye, Haibo Hu, Chengfang Fang, Jie Shi

Research output: Journal article publicationJournal articleAcademic researchpeer-review

1 Citation (Scopus)

Abstract

Machine learning service API allows model owners to monetize proprietary models by offering prediction services to third-party users. However, existing literature shows that model parameters are vulnerable to extraction attacks which accumulate prediction queries and their responses to train a replica model. As countermeasures, researchers have proposed to reduce the rich API output, such as hiding the precise confidence. Nonetheless, even with response being only one bit, an adversary can still exploit fine-tuned queries with differential property to infer the decision boundary of the underlying model. In this paper, we propose boundary differential privacy (BDP) against such attacks by obfuscating the prediction responses with noises. BDP guarantees an adversary cannot learn the decision boundary of any two classes by a predefined precision no matter how many queries are issued to the prediction API. We first design a perturbation algorithm called boundary randomized response for a binary model. Then we prove it satisfies ε-BDP, followed by a generalization of this algorithm to a multiclass model. Finally, we generalize a hard boundary to soft boundary and design an adaptive perturbation algorithm that can still work in the latter case. The effectiveness and high utility of our solution are verified by extensive experiments on both linear and non-linear models.

Original languageEnglish
JournalIEEE Transactions on Dependable and Secure Computing
DOIs
Publication statusAccepted/In press - 2020

Keywords

  • Adversarial Machine Learning
  • Boundary Differential Privacy
  • Model Defense
  • Model Extraction

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this