Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Viet Anh Nguyen; Soroosh Shafieezadeh Abadeh; Man Chung Yue; Daniel Kuhn; Wolfram Wiesemann

Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Viet Anh Nguyen, Soroosh Shafieezadeh Abadeh, Man Chung Yue, Daniel Kuhn, Wolfram Wiesemann

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

Abstract

The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighborhood of the nominal measure and that maximizes the probability of observing the given sample point. We show that when the neighborhood is constructed by the Kullback-Leibler divergence, by moment conditions or by the Wasserstein distance, then our optimistic likelihood can be determined through the solution of a convex optimization problem, and it admits an analytical expression in particular cases. We also show that the posterior inference problem with our optimistic likelihood approximation enjoys strong theoretical performance guarantees, and it performs competitively in a probabilistic classification task.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 32 (NIPS 2019)
Pages	1-25
Number of pages	25
Publication status	Published - Dec 2019
Event	Thirty-third Conference on Neural Information Processing Systems - Vancouver Convention Center, Vancouver, Canada Duration: 8 Dec 2019 → 14 Dec 2019

Conference

Conference	Thirty-third Conference on Neural Information Processing Systems
Abbreviated title	NeurIPS 2019
Country/Territory	Canada
City	Vancouver
Period	8/12/19 → 14/12/19

Keywords

Optimistic Robust Optimization
Likelihood Approximation
Wasserstein Distance
Distributionally Robust Optimization
Nonparametric Estimation
Kullback-Leibler Divergence
Variational Bayesian Inference

More information

Cite this

@inproceedings{fcfe3443393d443eb1464840a7ea1686,

title = "Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation",

abstract = "The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighborhood of the nominal measure and that maximizes the probability of observing the given sample point. We show that when the neighborhood is constructed by the Kullback-Leibler divergence, by moment conditions or by the Wasserstein distance, then our optimistic likelihood can be determined through the solution of a convex optimization problem, and it admits an analytical expression in particular cases. We also show that the posterior inference problem with our optimistic likelihood approximation enjoys strong theoretical performance guarantees, and it performs competitively in a probabilistic classification task.",

keywords = "Optimistic Robust Optimization, Likelihood Approximation, Wasserstein Distance, Distributionally Robust Optimization, Nonparametric Estimation, Kullback-Leibler Divergence, Variational Bayesian Inference",

author = "Nguyen, {Viet Anh} and {Shafieezadeh Abadeh}, Soroosh and Yue, {Man Chung} and Daniel Kuhn and Wolfram Wiesemann",

year = "2019",

month = dec,

language = "English",

pages = "1--25",

booktitle = "Advances in Neural Information Processing Systems 32 (NIPS 2019)",

note = "Thirty-third Conference on Neural Information Processing Systems, NeurIPS 2019 ; Conference date: 08-12-2019 Through 14-12-2019",

}

Nguyen, VA, Shafieezadeh Abadeh, S, Yue, MC, Kuhn, D & Wiesemann, W 2019, Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation. in Advances in Neural Information Processing Systems 32 (NIPS 2019). pp. 1-25, Thirty-third Conference on Neural Information Processing Systems, Vancouver, Canada, 8/12/19. <http://hdl.handle.net/10397/98575>

Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation. / Nguyen, Viet Anh; Shafieezadeh Abadeh, Soroosh; Yue, Man Chung et al.
Advances in Neural Information Processing Systems 32 (NIPS 2019). 2019. p. 1-25.

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

AU - Nguyen, Viet Anh

AU - Shafieezadeh Abadeh, Soroosh

AU - Yue, Man Chung

AU - Kuhn, Daniel

AU - Wiesemann, Wolfram

PY - 2019/12

Y1 - 2019/12

N2 - The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighborhood of the nominal measure and that maximizes the probability of observing the given sample point. We show that when the neighborhood is constructed by the Kullback-Leibler divergence, by moment conditions or by the Wasserstein distance, then our optimistic likelihood can be determined through the solution of a convex optimization problem, and it admits an analytical expression in particular cases. We also show that the posterior inference problem with our optimistic likelihood approximation enjoys strong theoretical performance guarantees, and it performs competitively in a probabilistic classification task.

AB - The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighborhood of the nominal measure and that maximizes the probability of observing the given sample point. We show that when the neighborhood is constructed by the Kullback-Leibler divergence, by moment conditions or by the Wasserstein distance, then our optimistic likelihood can be determined through the solution of a convex optimization problem, and it admits an analytical expression in particular cases. We also show that the posterior inference problem with our optimistic likelihood approximation enjoys strong theoretical performance guarantees, and it performs competitively in a probabilistic classification task.

KW - Optimistic Robust Optimization

KW - Likelihood Approximation

KW - Wasserstein Distance

KW - Distributionally Robust Optimization

KW - Nonparametric Estimation

KW - Kullback-Leibler Divergence

KW - Variational Bayesian Inference

M3 - Conference article published in proceeding or book

SP - 1

EP - 25

BT - Advances in Neural Information Processing Systems 32 (NIPS 2019)

T2 - Thirty-third Conference on Neural Information Processing Systems

Y2 - 8 December 2019 through 14 December 2019

ER -

Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Abstract

Conference

Keywords

More information

Fingerprint

Cite this