Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Viet Anh Nguyen, Soroosh Shafieezadeh Abadeh, Man Chung Yue, Daniel Kuhn, Wolfram Wiesemann

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review


The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighborhood of the nominal measure and that maximizes the probability of observing the given sample point. We show that when the neighborhood is constructed by the Kullback-Leibler divergence, by moment conditions or by the Wasserstein distance, then our optimistic likelihood can be determined through the solution of a convex optimization problem, and it admits an analytical expression in particular cases. We also show that the posterior inference problem with our optimistic likelihood approximation enjoys strong theoretical performance guarantees, and it performs competitively in a probabilistic classification task.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 32 (NIPS 2019)
Number of pages25
Publication statusPublished - Dec 2019
EventThirty-third Conference on Neural Information Processing Systems - Vancouver Convention Center, Vancouver, Canada
Duration: 8 Dec 201914 Dec 2019


ConferenceThirty-third Conference on Neural Information Processing Systems
Abbreviated titleNeurIPS 2019


  • Optimistic Robust Optimization
  • Likelihood Approximation
  • Wasserstein Distance
  • Distributionally Robust Optimization
  • Nonparametric Estimation
  • Kullback-Leibler Divergence
  • Variational Bayesian Inference


Dive into the research topics of 'Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation'. Together they form a unique fingerprint.

Cite this