arXiv subjects

Steffen Borgwardt

Publications and source records attributed to Steffen Borgwardt.

34 records · Page 2Linked to original sources

arXiv2020

An LP-based, Strongly-Polynomial 2-Approximation Algorithm for Sparse Wasserstein Barycenters

Discrete Wasserstein barycenters correspond to optimal solutions of transportation problems for a set of probability measures with finite support. Discrete barycenters are measures with finite support themselves and exhibit two favorable properties: there always exists one with a provably sparse support, and any optimal transport to the input measures is non-mass splitting. It is open whether a discrete barycenter can be computed in polynomial time. It is possible to find an exact barycenter through linear programming, but these programs may scale exponentially. In this paper, we prove that there is a strongly-polynomial 2-approximation algorithm based on linear programming. First, we show that an exact computation over the union of supports of the input measures gives a tight 2-approximation. This computation can be done through a linear program with setup and solution in strongly-polynomial time. The resulting measure is sparse, but an optimal transport may split mass. We then devise a second, strongly-polynomial algorithm to improve this measure to one with a non-mass splitting transport of lower cost. The key step is an update of the possible support set to resolve mass split. Finally, we devise an iterative scheme that alternates between these two algorithms. The algorithm terminates with a 2-approximation that has both a sparse support and an associated non-mass splitting optimal transport. We conclude with some sample computations and an analysis of the scaling of our algorithms, exhibiting vast improvements in running time over exact LP-based computations and low practical errors.

math.OC↗

arXiv2020

Constructing Clustering Transformations

Clustering is one of the fundamental tasks in data analytics and machine learning. In many situations, different clusterings of the same data set become relevant. For example, different algorithms for the same clustering task may return dramatically different solutions. We are interested in applications in which one clustering has to be transformed into another; e.g., when a gradual transition from an old solution to a new one is required. In this paper, we devise methods for constructing such a transition based on linear programming and network theory. We use a so-called clustering-difference graph to model the desired transformation and provide methods for decomposing the graph into a sequence of elementary moves that accomplishes the transformation. These moves are equivalent to the edge directions, or circuits, of the underlying partition polytopes. Therefore, in addition to a conceptually new metric for measuring the distance between clusterings, we provide new bounds on the circuit diameter of these partition polytopes.

math.OC↗

arXiv2019

An implementation of steepest-descent augmentation for linear programs

Generalizing the simplex method, circuit augmentation schemes for linear programs follow circuit directions through the interior of the underlying polyhedron. Steepest-descent augmentation is especially promising, but an implementation of the iterative scheme is a significant challenge. We work towards a viable implementation through a model in which a single linear program is updated dynamically to remain in memory throughout. Computational experiments exhibit dramatic improvements over a naive approach and reveal insight into the next steps required for large-scale computations.

math.OC↗

arXiv2019

Circuit Walks in Integral Polyhedra

Circuits play a fundamental role in the theory of linear programming due to their intimate connection to algorithms of combinatorial optimization and the efficiency of the simplex method. We are interested in better understanding the properties of circuit walks in integral polyhedra. In this paper, we introduce a hierarchy for integral polyhedra based on different types of behavior exhibited by their circuit walks. Many problems in combinatorial optimization fall into the most interesting categories of this hierarchy - steps of circuit walks only stop at integer points, at vertices, or follow actual edges. We classify several classical families of polyhedra within the hierarchy, including 0/1-polytopes, polyhedra defined by totally unimodular matrices, and more specifically matroid polytopes, transportation polytopes, and partition polytopes. Finally, we prove three characterizations of the simple polytopes that appear in the bottom level of the hierarchy where all circuit walks are edge walks, showing that such polytopes constitute a generalization of simplices and parallelotopes.

math.CO↗

arXiv2019

A Polyhedral Model for Enumeration and Optimization over the Set of Circuits

Circuits play a fundamental role in polyhedral theory and linear programming. For instance, circuits are used as step directions in various augmentation schemes for solving linear programs or to leave degenerate vertices while running the simplex method. However, there are significant challenges when implementing these approaches: The set of circuits of a polyhedron may be of exponential size and is highly sensitive to the representation of the polyhedron. In this paper, we provide a universal framework for enumerating the set of circuits and optimizing over sets of circuits of a polyhedron in any representation - we propose a polyhedral model in which the circuits of the original polyhedron are encoded as extreme rays or vertices. Many methods in the literature and software assume that a polyhedron is in standard form; our framework is a direct generalization. We demonstrate its value by showing that the conversion of a general representation to standard form may introduce exponentially many new circuits. We then discuss the main advantages of the generalized polyhedral model. It enables the direct enumeration of useful subsets of circuits such as strictly feasible circuits or sign-compatible circuits, as well as optimization over these sets. In particular, this leads to the efficient computation of a steepest-descent circuit, which can be used in an augmentation scheme for solving linear programs or the construction of sign-compatible circuit walks with additional

math.OC↗

arXiv2019

Improved Linear Programs for Discrete Barycenters

Discrete barycenters are the optimal solutions to mass transport problems for a set of discrete measures. Such transport problems arise in many applications of operations research and statistics. The best known algorithms for exact barycenters are based on linear programming, but these programs scale exponentially in the number of measures, making them prohibitive for practical purposes. In this paper, we improve on these algorithms. First, by using the optimality conditions to restrict the search space, we provide a reduced linear program that contains dramatically fewer variables compared to previous formulations. Second, we recall a proof from the literature, which lends itself to a linear program that has not been considered for computations. We show that this second formulation is the best model for data in general position. Third, we combine the two programs into a single hybrid model that retains the best properties of both formulations for partially structured data. We study these models through an analysis of their scaling in size, the hardness of the required preprocessing, and computational experiments. In doing so, we show that each of the improved linear programs becomes the best model for different types of data.

math.OC↗

arXiv2018

On the Circuit Diameter Conjecture

From the point of view of optimization, a critical issue is relating the combinatorial diameter of a polyhedron to its number of facets $f$ and dimension $d$. In the seminal paper of Klee and Walkup [KW67], the Hirsch conjecture of an upper bound of $f-d$ was shown to be equivalent to several seemingly simpler statements, and was disproved for unbounded polyhedra through the construction of a particular 4-dimensional polyhedron $U_4$ with 8 facets. The Hirsch bound for bounded polyhedra was only recently disproved by Santos. We consider analogous properties for a variant of the combinatorial diameter called the circuit diameter. In this variant, the walks are built from the circuit directions of the polyhedron, which are the minimal non-trivial solutions to the system defining the polyhedron. We are able to prove that circuit variants of the so-called non-revisiting conjecture and $d$-step conjecture both imply the circuit analogue of the Hirsch conjecture. For the equivalences in [KW67], the wedge construction was a fundamental proof technique. We exhibit why it is not available in the circuit setting, and what are the implications of losing it as a tool. Further, we show the circuit analogue of the non-revisiting conjecture implies a linear bound on the circuit diameter of all unbounded polyhedra - in contrast to what is known for the combinatorial diameter. Finally, we give two proofs of a circuit version of the $4$-step conjecture. These results offer some hope that the circuit version of the Hirsch conjecture may hold in general. A challenge in the circuit setting is that different realizations of polyhedra of the same combinatorial structure may have different diameters. We adapt the notion of simplicity to work with circuits in the form of C-simple and wedge-simple polyhedra. We show that it suffices to consider such polyhedra.

math.CO↗

arXiv2017

Power Diagram Detection with Applications to Information Elicitation

Power diagrams, a type of weighted Voronoi diagrams, have many applications throughout operations research. We study the problem of power diagram detection: determining whether a given finite partition of $\mathbb{R}^d$ takes the form of a power diagram. This detection problem is particularly prevalent in the field of information elicitation, where one wishes to design contracts to incentivize self-minded agents to provide honest information. We devise a simple linear program to decide whether a polyhedral cell decomposition can be described as a power diagram. Further, we discuss applications to property elicitation, peer prediction, and mechanism design, where this question arises. Our model is able to efficiently decide the question for decompositions of $\mathbb{R}^d$ or of a restricted domain in $\mathbb{R}^d$. The approach is based on the use of an alternative representation of power diagrams, and invariance of a power diagram under uniform scaling of the parameters in this representation.

math.OC↗

arXiv2016

A balanced k-means algorithm for weighted point sets

The classical $k$-means algorithm for partitioning $n$ points in $\mathbb{R}^d$ into $k$ clusters is one of the most popular and widely spread clustering methods. The need to respect prescribed lower bounds on the cluster sizes has been observed in many scientific and business applications. In this paper, we present and analyze a generalization of $k$-means that is capable of handling weighted point sets and prescribed lower and upper bounds on the cluster sizes. We call it weight-balanced $k$-means. The key difference to existing models lies in the ability to handle the combination of weighted point sets with prescribed bounds on the cluster sizes. This imposes the need to perform partial membership clustering, and leads to significant differences. For example, while finite termination of all $k$-means variants for unweighted point sets is a simple consequence of the existence of only finitely many partitions of a given set of points, the situation is more involved for weighted point sets, as there are infinitely many partial membership clusterings. Using polyhedral theory, we show that the number of iterations of weight-balanced $k$-means is bounded above by $n^{O(dk)}$, so in particular it is polynomial for fixed $k$ and $d$. This is similar to the known worst-case upper bound for classical $k$-means for unweighted point sets and unrestricted cluster sizes, despite the much more general framework. We conclude with the discussion of some additional favorable properties of our method.

math.OC↗

arXiv2016

Efficient solutions for weight-balanced partitioning problems

We prove polynomial-time solvability of a large class of clustering problems where a weighted set of items has to be partitioned into clusters with respect to some balancing constraints. The data points are weighted with respect to different features and the clusters adhere to given lower and upper bounds on the total weight of their points with respect to each of these features. Further the weight-contribution of a vector to a cluster can depend on the cluster it is assigned to. Our interest in these types of clustering problems is motivated by an application in land consolidation where the ability to perform this kind of balancing is crucial. Our framework maximizes an objective function that is convex in the summed-up utility of the items in each cluster. Despite hardness of convex maximization and many related problems, for fixed dimension and number of clusters, we are able to show that our clustering model is solvable in time polynomial in the number of items if the weight-balancing restrictions are defined using vectors from a fixed, finite domain. We conclude our discussion with a new, efficient model and algorithm for land consolidation.

math.OC↗

arXiv2015

Discrete Wasserstein Barycenters: Optimal Transport for Discrete Data

Wasserstein barycenters correspond to optimal solutions of transportation problems for several marginals, and as such have a wide range of applications ranging from economics to statistics and computer science. When the marginal probability measures are absolutely continuous (or vanish on small sets) the theory of Wasserstein barycenters is well-developed (see the seminal paper [1]). However, exact continuous computation of Wasserstein barycenters in this setting is tractable in only a small number of specialized cases. Moreover, in many applications data is given as a set of probability measures with finite support. In this paper, we develop theoretical results for Wasserstein barycenters in this discrete setting. Our results rely heavily on polyhedral theory which is possible due to the discrete structure of the marginals. Our results closely mirror those in the continuous case with a few exceptions. In this discrete setting we establish that Wasserstein barycenters must also be discrete measures and there is always a barycenter which is provably sparse. Moreover, for each Wasserstein barycenter there exists a non-mass-splitting optimal transport to each of the discrete marginals. Such non-mass-splitting transports do not generally exist between two discrete measures unless special mass balance conditions hold. This makes Wasserstein barycenters in this discrete setting special in this regard. We illustrate the results of our discrete barycenter theory with a proof-of-concept computation for a hypothetical transportation problem with multiple marginals: distributing a fixed set of goods when the demand can take on different distributional shapes characterized by the discrete marginal distributions. A Wasserstein barycenter, in this case, represents an optimal distribution of inventory facilities which minimize the squared distance/transportation cost totaled over all demands.

math.OC↗

arXiv2015

The Hierarchy of Circuit Diameters and Transportation Polytopes

The study of the diameter of the graph of polyhedra is a classical problem in the theory of linear programming. While transportation polytopes are at the core of operations research and statistics it is still open whether the Hirsch conjecture is true for general $m{\times}n$--transportation polytopes. In earlier work the first three authors introduced a hierarchy of variations to the notion of graph diameter in polyhedra. The key reason was that this hierarchy provides some interesting lower bounds for the usual graph diameter. This paper has three contributions: First, we compare the hierarchy of diameters for the $m{\times}n$--transportation polytopes. We show that the Hirsch conjecture bound of $m+n-1$ is actually valid in most of these diameter notions. Second, we prove that for $3{\times}n$--transportation polytopes the Hirsch conjecture holds in the classical graph diameter. Third, we show for $2{\times}n$--transportation polytopes that the stronger monotone Hirsch conjecture holds and improve earlier bounds on the graph diameter.

math.CO↗

arXiv2014

Edges vs Circuits: a Hierarchy of Diameters in Polyhedra

The study of the graph diameter of polytopes is a classical open problem in polyhedral geometry and the theory of linear optimization. In this paper we continue the investigation initiated in [4] by introducing a vast hierarchy of generalizations to the notion of graph diameter. This hierarchy provides some interesting lower bounds for the usual graph diameter. After explaining the structure of the hierarchy and discussing these bounds, we focus on clearly explaining the differences and similarities among the many diameter notions of our hierarchy. Finally, we fully characterize the hierarchy in dimension two. It collapses into fewer categories, for which we exhibit the ranges of values that can be realized as diameters.

math.CO↗

arXiv2014

Quadratic diameter bounds for dual network flow polyhedra

Both the combinatorial and the circuit diameters of polyhedra are of interest to the theory of linear programming for their intimate connection to a best-case performance of linear programming algorithms. We study the diameters of dual network flow polyhedra associated to $b$-flows on directed graphs $G=(V,E)$ and prove quadratic upper bounds for both of them: the minimum of $(|V|-1)\cdot |E|$ and $\frac{1}{6}|V|^3$ for the combinatorial diameter, and $\frac{|V|\cdot (|V|-1)}{2}$ for the circuit diameter. The latter strengthens the cubic bound implied by a result in [De Loera, Hemmecke, Lee; 2014]. Previously, bounds on these diameters have only been known for bipartite graphs. The situation is much more involved for general graphs. In particular, we construct a family of dual network flow polyhedra with members that violate the circuit diameter bound for bipartite graphs by an arbitrary additive constant. Further, it provides examples of circuit diameter $\frac{4}{3}|V| - 4$.

math.OC↗

arXiv2014

On the circuit diameter of dual transportation polyhedra

In this paper we introduce the circuit diameter of polyhedra, which is always bounded from above by the combinatorial diameter. We consider dual transportation polyhedra defined on general bipartite graphs. For complete $M{\times}N$ bipartite graphs the Hirsch bound $(M{-}1)(N{-}1)$ on the combinatorial diameter is a known tight bound (Balinski, 1984). For the circuit diameter we show the much stronger bound $M{+}N{-}2$ for all dual transportation polyhedra defined on arbitrary bipartite graphs with $M{+}N$ nodes.

math.CO↗

arXiv2014

On Soft Power Diagrams

Many applications in data analysis begin with a set of points in a Euclidean space that is partitioned into clusters. Common tasks then are to devise a classifier deciding which of the clusters a new point is associated to, finding outliers with respect to the clusters, or identifying the type of clustering used for the partition. One of the common kinds of clusterings are (balanced) least-squares assignments with respect to a given set of sites. For these, there is a 'separating power diagram' for which each cluster lies in its own cell. In the present paper, we aim for efficient algorithms for outlier detection and the computation of thresholds that measure how similar a clustering is to a least-squares assignment for fixed sites. For this purpose, we devise a new model for the computation of a 'soft power diagram', which allows a soft separation of the clusters with 'point counting properties'; e.g. we are able to prescribe how many points we want to classify as outliers. As our results hold for a more general non-convex model of free sites, we describe it and our proofs in this more general way. Its locally optimal solutions satisfy the aforementioned point counting properties. For our target applications that use fixed sites, our algorithms are efficiently solvable to global optimality by linear programming.

cs.LG↗