TY - JOUR

T1 - A fast globally linearly convergent algorithm for the computation of wasserstein barycenters

AU - Yang, Lei

AU - Li, Jia

AU - Sun, Defeng

AU - Toh, Kim Chuan

N1 - Funding Information:
of Defeng Sun was supported in part by a start-up research grant from the Hong Kong Polytechnic University. The research of Kim-Chuan Toh was supported in part by the Ministry of Education, Singapore, Academic Research Fund (Grant No. R-146-000-256-114).
Publisher Copyright:
© 2021 Microtome Publishing. All rights reserved.

PY - 2021/1

Y1 - 2021/1

N2 - We consider the problem of computing a Wasserstein barycenter for a set of discrete probability distributions with finite supports, which finds many applications in areas such as statistics, machine learning and image processing. When the support points of the barycenter are pre-specified, this problem can be modeled as a linear programming (LP) problem whose size can be extremely large. To handle this large-scale LP, we analyse the structure of its dual problem, which is conceivably more tractable and can be reformulated as a well-structured convex problem with 3 kinds of block variables and a coupling linear equality constraint. We then adapt a symmetric Gauss-Seidel based alternating direction method of multipliers (sGS-ADMM) to solve the resulting dual problem and establish its global convergence and global linear convergence rate. As a critical component for efficient computation, we also show how all the subproblems involved can be solved exactly and efficiently. This makes our method suitable for computing a Wasserstein barycenter on a large-scale data set, without introducing an entropy regularization term as is commonly practiced. In addition, our sGS-ADMM can be used as a subroutine in an alternating minimization method to compute a barycenter when its support points are not pre-specified. Numerical results on synthetic data sets and image data sets demonstrate that our method is highly competitive for solving large-scaleWasserstein barycenter problems, in comparison to two existing representative methods and the commercial software Gurobi.

AB - We consider the problem of computing a Wasserstein barycenter for a set of discrete probability distributions with finite supports, which finds many applications in areas such as statistics, machine learning and image processing. When the support points of the barycenter are pre-specified, this problem can be modeled as a linear programming (LP) problem whose size can be extremely large. To handle this large-scale LP, we analyse the structure of its dual problem, which is conceivably more tractable and can be reformulated as a well-structured convex problem with 3 kinds of block variables and a coupling linear equality constraint. We then adapt a symmetric Gauss-Seidel based alternating direction method of multipliers (sGS-ADMM) to solve the resulting dual problem and establish its global convergence and global linear convergence rate. As a critical component for efficient computation, we also show how all the subproblems involved can be solved exactly and efficiently. This makes our method suitable for computing a Wasserstein barycenter on a large-scale data set, without introducing an entropy regularization term as is commonly practiced. In addition, our sGS-ADMM can be used as a subroutine in an alternating minimization method to compute a barycenter when its support points are not pre-specified. Numerical results on synthetic data sets and image data sets demonstrate that our method is highly competitive for solving large-scaleWasserstein barycenter problems, in comparison to two existing representative methods and the commercial software Gurobi.

KW - Discrete Probability distribution

KW - Semi-proximal ADMM

KW - symmetric Gauss-Seidel

KW - Wasserstein barycenter

UR - http://www.scopus.com/inward/record.url?scp=85104804743&partnerID=8YFLogxK

M3 - Journal article

AN - SCOPUS:85104804743

VL - 22

SP - 1

EP - 37

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -