arXiv subjects

Zikai Xiong

Publications and source records attributed to Zikai Xiong.

16 recordsLinked to original sources

arXiv2026

Level-Set Geometry and the Theoretical Performance of PDHG for Conic Linear Optimization

We consider solving (convex) conic linear optimization problems, at the scale where matrix-factorization-free methods are attractive or necessary. The restarted primal-dual hybrid gradient method (rPDHG) -- with heuristic enhancements and GPU implementation -- has been very successful in solving huge-scale linear optimization problems (LPs). However, its application to more general conic convex optimization problems is not so well-studied. We analyze the theoretical performance of rPDHG for general (convex) conic linear optimization, and LP as a special case thereof. We show a relationship between the geometry of the primal-dual $δ$-(sub-)level sets ${W}_δ$ and the convergence rate of rPDHG. Specifically, we prove a bound on the convergence rate of rPDHG that improves when there is a primal-dual (sub-)level set ${W}_δ$ for which (i) ${W}_δ$ is close to the optimal solution set in Hausdorff distance, and (ii) the ratio of the diameter to the ``conic radius'' of ${W}_δ$ is small. And in the special case of LP, the performance of rPDHG is bounded only by this ratio applied to the (sub-)level set corresponding to the best non-optimal extreme point. Depending on the problem instance, this ratio can take on extreme values and can result in excellent or poor performance of rPDHG both in theory and in practice.

math.OC↗

arXiv2026

High-Probability Polynomial-Time Complexity of Restarted PDHG for Linear Programming

The restarted primal-dual hybrid gradient method (rPDHG) is a first-order method recently known for its computational effectiveness in solving linear programming (LP) problems. Despite its impressive practical performance, the theoretical iteration bounds for rPDHG can be exponentially poor. To investigate this gap from a probabilistic perspective, we show that rPDHG achieves polynomial-time complexity in a high-probability sense, under assumptions on the probability distribution from which the data instance is generated. We consider both Gaussian and more general sub-Gaussian distribution models. For standard-form LP instances with $m$ constraints and $n$ variables, our bounds take a particularly simple form when $m$ is not too close to $n$: rPDHG iterates settle on the optimal basis in $\widetilde{O}\left(\tfrac{n^{2.5}m^{0.5}}δ\right)$ iterations, followed by $O\left(\frac{n^{0.5}m^{0.5}}δ\ln\big(\tfrac{1}{\varepsilon}\big)\right)$ iterations to compute an $\varepsilon $-optimal solution, with probability at least $1-δ$ for $δ$ that is not exponentially small. The Stage-I bound further improves to $\widetilde{O}\left(\frac{n^{2.5}}δ\right)$ in the Gaussian distribution model. Experimental results confirm the tail behavior and the polynomial-time dependence on problem dimensions of the iteration counts. As an application of our probabilistic analysis, we explore how the disparity among the components of the optimal solution bears on the performance of rPDHG, and we provide guidelines for generating challenging LP instances.

math.OC↗

arXiv2026

Symmetry-dependence in Rounding of a Convex Body

The symmetry measure of a convex body $S\subset\mathbb{R}^n$ is given by $$\operatorname{sym}(S):=\max\{α\ge0:\text{ there exists }x\in S\text{ such that }-α(S-x)\subseteq S-x\},$$ where such an $x$ is called a Minkowski center. We prove that every convex body $S$ admits a $\sqrt{\frac{n}{\operatorname{sym}(S)}}$-rounding of $S$, namely, there exists an origin-centered ellipsoid $E$ and a center $c$ such that the following rounding holds: $$E\subseteq S-c\subseteq\sqrt{\frac{n}{\operatorname{sym}(S)}}\,E.$$ This result was conjectured in 2005 by Belloni and Freund. As special cases, this recovers the known result of an $n$-rounding of $S$ (since it always holds that $\operatorname{sym}(S) \ge 1/n$), and also recovers the known result of a $\sqrt{n}$-rounding when $\operatorname{sym}(S) = 1$. In the case when $S$ is a polytope given as the convex hull of points, the desired rounding is produced by a regularized minimum volume covering ellipsoid problem where the regularization is with respect to the Minkowski center. Similarly, when $S$ is a polytope given as the intersection of halfspaces, such a rounding is produced by a regularized maximum volume inscribed ellipsoid problem. We also show that the factor $\sqrt{\frac{n}{\operatorname{sym}(S)}}$ is nearly tight in its dependence on dimension and symmetry. When $\frac{n+1}{1+\operatorname{sym}(S)}$ is an integer, we show by explicit construction that the factor $\sqrt{\frac{n}{\operatorname{sym}(S)}}$ is tight. In the more general case, for every dimension $n$ and every admissible symmetry value, we construct a polytope $S$ for which every rounding is at least $\sqrt{\frac{2}{3}}\sqrt{\frac{n}{\operatorname{sym}(S)}}$.

math.OC↗

arXiv2026

Accessible Complexity Bounds for Restarted PDHG on Linear Programs with a Unique Optimizer

The restarted primal-dual hybrid gradient method (rPDHG) has recently emerged as an important tool for solving large-scale linear programs (LPs). For LPs with unique optima, we present an iteration bound of $O\left(κΦ\cdot\ln\left(\frac{κΦ\|w^*\|}{\varepsilon}\right)\right)$, where $\varepsilon$ is the target tolerance, $κ$ is the standard matrix condition number, $\|w^*\|$ is the norm of the optimal solution, and $Φ$ is a geometric condition number of the LP sublevel sets. This iteration bound is "accessible" in the sense that computing it is typically no more difficult than computing the optimal solution itself. Indeed, we present a closed-form and tractably computable expression for $Φ$. This enables an analysis of the "two-stage performance" of rPDHG: we show that the first stage identifies the optimal basis in ${O}\left(κΦ\cdot\ln(κΦ)\right)$ iterations, and the second stage computes an $\varepsilon$-optimal solution in $O\left(\|B^{-1}\|\|A\|\cdot\ln\left(\fracξ{\varepsilon}\right)\right)$ additional iterations, where $A$ is the constraint matrix, $B$ is the optimal basis and $ξ$ is the smallest nonzero in the optimal solution. . Furthermore, computational tests are consistent with our iteration bounds. We also show a reciprocal relation between the iteration bound and stability under data perturbation, which is also equivalent to (i) proximity to multiple optima, and (ii) the LP sharpness of the instance.

math.OC↗

arXiv2026

Function-free Optimization via Comparison Oracles

In this work, we study optimization specified only through a comparison oracle: given two points, it reports which one is preferred. We call it function-free optimization because we do not assume access to, nor the existence of, a canonical application-given objective function. The goal is to find the most preferred feasible point, which we call the optimal solution. This model arises in preference- and ranking-based settings where objective values and derivatives are unavailable or meaningless. Even when a representative function exists, it may be nonsmooth, nonconvex, or discontinuous. We develop an analytical and algorithmic framework based on the geometry of preference level sets, which remains well-defined from comparisons alone. We introduce the level-set optimality gap, the distance from a preference level set to the optimal solutions, and the regularity radius, a stationarity certificate. Under regularity of the preference relation in a $d$-dimensional Euclidean space, we estimate normal directions to accuracy $ε$ using $O(d\log(d/ε))$ comparisons, nearly matching a lower bound of $Ω(d\log(1/ε))$. Under convexity, regularity, and a local growth condition on the regularity radius, the resulting normal direction descent method reaches an $ε$ level-set optimality gap using at most $\widetilde O(dD^2/ε^2)$ comparisons over $O(D^2/ε^2)$ normal direction estimation steps, where $D$ is the distance from the initial point to the optimal solutions. This number of steps matches the lower bound of $Ω(D^2/ε^2)$ for normal direction span-based methods. Since prior knowledge in practical applications is usually limited, we also develop adaptive schemes for estimating the normal direction and solving the optimization problem. They match the fixed-parameter complexity bounds up to logarithmic factors.

math.OC↗

arXiv2026

A Practical GPU-Enhanced Matrix-Free Primal-Dual Method for Large-Scale Conic Programs

In this paper, we introduce a practical GPU-enhanced matrix-free first-order method for solving large-scale conic programming problems, which we refer to as PDCS, standing for the Primal-Dual Conic Programming Solver. Problems that it solves include linear programs, second-order cone programs, convex quadratic programs, and exponential cone programs. The method avoids matrix factorizations and leverages sparse matrix-vector multiplication as its core computational operation, which is both memory-efficient and well-suited for GPU acceleration. The method builds on the restarted primal-dual hybrid gradient method but further incorporates several enhancements. Additionally, it employs a bisection-based method to compute projections onto rescaled cones. Furthermore, cuPDCS is a GPU implementation of PDCS and it implements customized computational schemes that utilize different levels of GPU architecture to handle cones of different types and sizes. Numerical experiments demonstrate that cuPDCS is generally more efficient than state-of-the-art commercial solvers and other first-order methods on large-scale conic program applications, including Fisher market equilibrium problems, Lasso regression, and multi-period portfolio optimization. Furthermore, cuPDCS also exhibits better scalability, efficiency, and robustness compared to other first-order methods on the conic program benchmark dataset CBLIB. These advantages are more pronounced in large-scale, lower-accuracy settings.

math.OC↗

arXiv2026

A Technical Note on the Implementation and Use of PDCS

This technical note documents the implementation and use of the Primal-Dual Conic Programming Solver (PDCS), a first-order solver for large-scale conic optimization problems introduced by Lin et al. (arXiv:2505.00311). It describes the algorithmic and implementation details underlying PDCS, including the restarted primal-dual hybrid gradient method framework, adaptive step-size selection, adaptive reflected Halpern iterations, adaptive restarts, and diagonal preconditioning. It also provides practical instructions for using PDCS, including its interfaces with JuMP and CVXPY, solver options, and illustrative code examples. PDCS is available at https://github.com/ZikaiXiong/PDCS under the Apache License 2.0.

math.OC↗

arXiv2026

Computational Guarantees for Restarted PDHG for LP based on "Limiting Error Ratios" and LP Sharpness

In recent years, there has been growing interest in solving linear optimization problems - or more simply "LP" - using first-order methods in order to avoid the costly matrix factorizations of traditional methods for huge-scale LP instances. The restarted primal-dual hybrid gradient method (PDHG) - together with some heuristic techniques - has emerged as a powerful tool for solving huge-scale LPs. However, the theoretical understanding of the restarted PDHG and the validation of various heuristic implementation techniques are still very limited. Existing complexity analyses have relied on the Hoffman constant of the LP KKT system, which is known to be overly conservative, difficult to compute, and fails to offer insight into instance-specific characteristics of the LP problems. These limitations have limited the capability to discern which characteristics of LP instances lead to easy versus difficult instances. With the goal of overcoming these limitations, we introduce and develop two purely geometry-based condition measures for LP instances: "limiting error ratio" and LP sharpness. We provide new computational guarantees for the restarted PDHG based on these two condition measures. For limiting error ratio, we provide a computable upper bound and show its relationship with the data instance's proximity to infeasibility under perturbation. For LP sharpness, we prove its equivalence to the stability of the LP optimal solution set under perturbation of the objective function. Our computational guarantees are validated by constructed instances. Conversely, our computational guarantees validate the practical efficacy of certain heuristic techniques (row preconditioners and step-size tuning) that improve computational performance. Finally, we present computational experiments on LP relaxations from the MIPLIB dataset that demonstrate the promise of various implementation strategies.

math.OC↗

arXiv2025

On the Relation Between LP Sharpness and Limiting Error Ratio and Complexity Implications for Restarted PDHG

There has been a recent surge in development of first-order methods (FOMs) for solving huge-scale linear programming (LP) problems. The attractiveness of FOMs for LP stems in part from the fact that they avoid costly matrix factorization computation. However, the efficiency of FOMs is significantly influenced - both in theory and in practice - by certain instance-specific LP condition measures. Xiong and Freund recently showed that the performance of the restarted primal-dual hybrid gradient method (PDHG) is predominantly determined by two specific condition measures: LP sharpness and Limiting Error Ratio. In this paper we examine the relationship between these two measures, particularly in the case when the optimal solution is unique (which is generic - at least in theory), and we present an upper bound on the Limiting Error Ratio involving the reciprocal of the LP sharpness. This shows that in LP instances where there is a dual nondegenerate optimal solution, the computational complexity of restarted PDHG can be characterized solely in terms of LP sharpness and the distance to optimal solutions, and simplifies the theoretical complexity upper bound of restarted PDHG for these instances.

math.OC↗

arXiv2024

From an Interior Point to a Corner Point: Smart Crossover

Identifying optimal basic feasible solutions to linear programming problems is a critical task for mixed integer programming and other applications. The crossover method, which aims at deriving an optimal extreme point from a suboptimal solution (the output of a starting method such as interior-point methods or first-order methods), is crucial in this process. This method, compared to the starting method, frequently represents the primary computational bottleneck in practical applications. We propose approaches to overcome this bottleneck by exploiting problem characteristics and implementing customized strategies. For problems arising from network applications and exhibiting network structures, we take advantage of the graph structure of the problem and the tree structure of the optimal solutions. Based on these structures, we propose a tree-based crossover method, aiming to recovering basic solutions by identifying nearby spanning tree structures. For general linear programs, we propose recovering an optimal basic solution by identifying the optimal face and employing controlled perturbations based on the suboptimal solution provided by interior-point methods. We prove that an optimal solution for the perturbed problem is an extreme point, and its objective value is at least as good as that of the initial interior point solution. Computational experiments show significant speed-ups achieved by our methods compared to state-of-the-art commercial solvers on classical linear programming problem benchmarks, network flow problem benchmarks, and optimal transport problems.

math.OC↗

arXiv2024

Fair Wasserstein Coresets

Data distillation and coresets have emerged as popular approaches to generate a smaller representative set of samples for downstream learning tasks to handle large-scale datasets. At the same time, machine learning is being increasingly applied to decision-making processes at a societal level, making it imperative for modelers to address inherent biases towards subgroups present in the data. While current approaches focus on creating fair synthetic representative samples by optimizing local properties relative to the original samples, their impact on downstream learning processes has yet to be explored. In this work, we present fair Wasserstein coresets (FWC), a novel coreset approach which generates fair synthetic representative samples along with sample-level weights to be used in downstream learning tasks. FWC uses an efficient majority minimization algorithm to minimize the Wasserstein distance between the original dataset and the weighted synthetic samples while enforcing demographic parity. We show that an unconstrained version of FWC is equivalent to Lloyd's algorithm for k-medians and k-means clustering. Experiments conducted on both synthetic and real datasets show that FWC: (i) achieves a competitive fairness-utility tradeoff in downstream models compared to existing approaches, (ii) improves downstream fairness when added to the existing training data and (iii) can be used to reduce biases in predictions from large language models (GPT-3.5 and GPT-4).

stat.ML↗

arXiv2024

FairWASP: Fast and Optimal Fair Wasserstein Pre-processing

Recent years have seen a surge of machine learning approaches aimed at reducing disparities in model outputs across different subgroups. In many settings, training data may be used in multiple downstream applications by different users, which means it may be most effective to intervene on the training data itself. In this work, we present FairWASP, a novel pre-processing approach designed to reduce disparities in classification datasets without modifying the original data. FairWASP returns sample-level weights such that the reweighted dataset minimizes the Wasserstein distance to the original dataset while satisfying (an empirical version of) demographic parity, a popular fairness criterion. We show theoretically that integer weights are optimal, which means our method can be equivalently understood as duplicating or eliminating samples. FairWASP can therefore be used to construct datasets which can be fed into any classification method, not just methods which accept sample weights. Our work is based on reformulating the pre-processing task as a large-scale mixed-integer program (MIP), for which we propose a highly efficient algorithm based on the cutting plane method. Experiments demonstrate that our proposed optimization algorithm significantly outperforms state-of-the-art commercial solvers in solving both the MIP and its linear program relaxation. Further experiments highlight the competitive performance of FairWASP in reducing disparities while preserving accuracy in downstream classification settings.

cs.LG↗

arXiv2024

The Role of Level-Set Geometry on the Performance of PDHG for Conic Linear Optimization

We consider solving huge-scale instances of (convex) conic linear optimization problems, at the scale where matrix-factorization-free methods are attractive or necessary. The restarted primal-dual hybrid gradient method (rPDHG) -- with heuristic enhancements and GPU implementation -- has been very successful in solving huge-scale linear programming (LP) problems; however its application to more general conic convex optimization problems is not so well-studied. We analyze the theoretical and practical performance of rPDHG for general (convex) conic linear optimization, and LP as a special case thereof. We show a relationship between the geometry of the primal-dual (sub-)level sets $W_\varepsilon$ and the convergence rate of rPDHG. Specifically, we prove a bound on the convergence rate of rPDHG that improves when there is a primal-dual (sub-)level set $W_\varepsilon$ for which (i) $W_\varepsilon$ is close to the optimal solution set (in Hausdorff distance), and (ii) the ratio of the diameter to the "conic radius" of $W_\varepsilon$ is small. And in the special case of LP problems, the performance of rPDHG is bounded only by this ratio applied to the (sub-)level set corresponding to the best non-optimal extreme point. Depending on the problem instance, this ratio can take on extreme values and can result in poor performance of rPDHG both in theory and in practice. To address this issue, we show how central-path-based linear transformations -- including conic rescaling -- can markedly enhance the convergence rate of rPDHG. Furthermore, we present computational results that demonstrate how such rescalings can accelerate convergence to high-accuracy solutions, and lead to more efficient methods for huge-scale linear optimization problems.

math.OC↗

arXiv2023

Using Taylor-Approximated Gradients to Improve the Frank-Wolfe Method for Empirical Risk Minimization

The Frank-Wolfe method has become increasingly useful in statistical and machine learning applications, due to the structure-inducing properties of the iterates, and especially in settings where linear minimization over the feasible set is more computationally efficient than projection. In the setting of Empirical Risk Minimization -- one of the fundamental optimization problems in statistical and machine learning -- the computational effectiveness of Frank-Wolfe methods typically grows linearly in the number of data observations $n$. This is in stark contrast to the case for typical stochastic projection methods. In order to reduce this dependence on $n$, we look to second-order smoothness of typical smooth loss functions (least squares loss and logistic loss, for example) and we propose amending the Frank-Wolfe method with Taylor series-approximated gradients, including variants for both deterministic and stochastic settings. Compared with current state-of-the-art methods in the regime where the optimality tolerance $\varepsilon$ is sufficiently small, our methods are able to simultaneously reduce the dependence on large $n$ while obtaining optimal convergence rates of Frank-Wolfe methods, in both the convex and non-convex settings. We also propose a novel adaptive step-size approach for which we have computational guarantees. Last of all, we present computational experiments which show that our methods exhibit very significant speed-ups over existing methods on real-world datasets for both convex and non-convex binary classification problems.

cs.LG↗

arXiv2022

Learning from Multiple Annotator Noisy Labels via Sample-wise Label Fusion

Data lies at the core of modern deep learning. The impressive performance of supervised learning is built upon a base of massive accurately labeled data. However, in some real-world applications, accurate labeling might not be viable; instead, multiple noisy labels (instead of one accurate label) are provided by several annotators for each data sample. Learning a classifier on such a noisy training dataset is a challenging task. Previous approaches usually assume that all data samples share the same set of parameters related to annotator errors, while we demonstrate that label error learning should be both annotator and data sample dependent. Motivated by this observation, we propose a novel learning algorithm. The proposed method displays superiority compared with several state-of-the-art baseline methods on MNIST, CIFAR-100, and ImageNet-100. Our code is available at: https://github.com/zhengqigao/Learning-from-Multiple-Annotator-Noisy-Labels.

cs.LG↗

arXiv2020

Interior-Point Methods Strike Back: Solving the Wasserstein Barycenter Problem

Computing the Wasserstein barycenter of a set of probability measures under the optimal transport metric can quickly become prohibitive for traditional second-order algorithms, such as interior-point methods, as the support size of the measures increases. In this paper, we overcome the difficulty by developing a new adapted interior-point method that fully exploits the problem's special matrix structure to reduce the iteration complexity and speed up the Newton procedure. Different from regularization approaches, our method achieves a well-balanced tradeoff between accuracy and speed. A numerical comparison on various distributions with existing algorithms exhibits the computational advantages of our approach. Moreover, we demonstrate the practicality of our algorithm on image benchmark problems including MNIST and Fashion-MNIST.

math.OC↗