Margin maximization for robust classification using deep learning

Alexander Matyasko, Lap Pui Chau

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

5 Citations (Scopus)

Abstract

Deep neural networks have achieved significant success for image recognition problems. Despite the wide success, recent experiments demonstrated that neural networks are sensitive to small input perturbations, or adversarial noise. The lack of robustness is intuitively undesirable and limits neural networks applications in adversarial settings, and for image search and retrieval problems. Current approaches consider augmenting training dataset using adversarial examples to improve robustness. However, when using data augmentation, the model fails to anticipate changes in an adversary. In this paper, we consider maximizing the geometric margin of the classifier. Intuitively, a large margin relates to classifier robustness. We introduce novel margin maximization objective for deep neural networks. We theoretically show that the proposed objective is equivalent to the robust optimization problem for a neural network. Our work seamlessly generalizes SVM margin objective to deep neural networks. In the experiments, we extensively verify the effectiveness of the proposed margin maximization objective to improve neural network robustness and to reduce overfitting on MNIST and CIFAR-10 dataset.

Original languageEnglish
Title of host publication2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages300-307
Number of pages8
ISBN (Electronic)9781509061815
DOIs
Publication statusPublished - 30 Jun 2017
Externally publishedYes
Event2017 International Joint Conference on Neural Networks, IJCNN 2017 - Anchorage, United States
Duration: 14 May 201719 May 2017

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2017-May

Conference

Conference2017 International Joint Conference on Neural Networks, IJCNN 2017
Country/TerritoryUnited States
CityAnchorage
Period14/05/1719/05/17

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this