Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Viet Anh Nguyen, Soroosh Shafieezadeh Abadeh, Man Chung Yue, Daniel Kuhn, Wolfram Wiesemann

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighborhood of the nominal measure and that maximizes the probability of observing the given sample point. We show that when the neighborhood is constructed by the Kullback-Leibler divergence, by moment conditions or by the Wasserstein distance, then our optimistic likelihood can be determined through the solution of a convex optimization problem, and it admits an analytical expression in particular cases. We also show that the posterior inference problem with our optimistic likelihood approximation enjoys strong theoretical performance guarantees, and it performs competitively in a probabilistic classification task.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 32 (NIPS 2019)
Publication statusPublished - Dec 2019
EventThirty-third Conference on Neural Information Processing Systems - Vancouver Convention Center, Vancouver, Canada
Duration: 8 Dec 201914 Dec 2019

Conference

ConferenceThirty-third Conference on Neural Information Processing Systems
Abbreviated titleNeurIPS 2019
Country/TerritoryCanada
CityVancouver
Period8/12/1914/12/19

Keywords

  • Optimistic Robust Optimization
  • Likelihood Approximation
  • Wasserstein Distance
  • Distributionally Robust Optimization
  • Nonparametric Estimation
  • Kullback-Leibler Divergence
  • Variational Bayesian Inference

Cite this