arXiv subjects

Tianyun Tang

Publications and source records attributed to Tianyun Tang.

At least 19 recordsLinked to original sources

arXiv2026

Convex Relaxations for the Optimization of Markov Processes

In this paper, we study the problem of optimizing Markov processes that interpolate between two prescribed probability distributions while minimizing a given cost. The main computational challenge is the curse of dimensionality: in high-dimensional state spaces, representing the full distribution is intractable. To address this, we reformulate the problem in terms of sequential couplings and develop convex relaxations based on local marginals and cluster moments. These relaxations exploit locality and sparse interaction structure, provide computable lower bounds, and recover low-order statistics of the intermediate laws. We identify dynamic optimal transport as a special case of our Markov process optimization problem and develop a procedure for recovering the underlying Benamou--Brenier dynamics from the relaxed solution. We also show that the procedure extends to more general Markov processes and illustrate it with a constrained process between Ising models.

math.OC↗

arXiv2026

A preconditioned augmented Lagrangian method for solving semidefinite programming problems

In this work, we propose a preconditioned augmented Lagrangian method (ALM) for solving semidefinite programming (SDP) problems. The preconditioner is implemented via a weighted penalty function in the ALM subproblem, with the weight matrix derived from the projection operator onto the tangent space of the feasible region. This simple yet effective modification significantly accelerates ALM, particularly for ill-conditioned SDPs. By combining the preconditioned ALM with our previously developed feasible method SDPF, we develop SDPF+, an SDP solver capable of handling convex problems with possibly nonlinear objective functions. Extensive numerical experiments demonstrate the efficiency and robustness of SDPF+, showing that it can generally outperform other solvers on large-scale SDPs whose optimal solutions exhibit low-rank structure.

math.OC↗

arXiv2026

On the efficient computation of proximal operators of affine-constrained nonconvex functions

Proximal operators with affine constraints arise in numerous models in nonconvex projection, composite optimization, and structured regularization. However, their efficient computation remains challenging due to the simultaneous presence of affine constraints and nonsmooth, possibly nonconvex objectives. In this work, we develop a unified dual-representability framework for analyzing and computing affine-constrained proximal mappings. Specifically, we introduce a multiplier inclusion formulation that connects the primal affine-constrained proximal problem to an unconstrained convex dual problem. Based on this formulation, we prove that, whenever the associated dual inclusion problem admits a solution, strong duality holds. For convex functions and a broad class of prox-regular nonconvex functions, we establish that dual representability holds under a simple subdifferential sum rule, and further develop a hierarchy of verifiable regularity conditions that guarantee this sum rule. In addition, we analyze the smoothness and strong convexity properties of the dual objective, providing a rigorous foundation that guarantees fast local convergence rates for efficient first- and second-order methods. Numerical experiments demonstrate that the proposed dual reformulation enables the reliable computation of globally optimal solutions for a range of large-scale nonconvex proximal and projection problems using existing convex optimization solvers.

math.OC↗

arXiv2025

A Low-rank Augmented Lagrangian Method for Polyhedral-SDP and Moment-SOS Relaxations of Polynomial Optimization

Polynomial optimization problems (POPs) can be reformulated as geometric convex conic programs, as shown by Kim, Kojima, and Toh (SIOPT 30:1251-1273, 2020), though such formulations remain NP-hard. In this work, we prove that several well-known relaxations can be unified under a common polyhedral-SDP framework, which arises by approximating the intractable cone by tractable intersections of polyhedral cones with the positive semidefinite matrix cone. Although effective in providing tight lower bounds, these relaxations become computationally expensive as the number of variables and constraints grows at the rate of $\Omega(n^{2\tau})$ with the relaxation order $\tau$. To address this challenge, we propose RiNNAL-POP, a low-rank augmented Lagrangian method (ALM) tailored to solve large-scale polyhedral-SDP relaxations of POPs. To efficiently handle the $\Omega(n^{2\tau})$ nonnegativity and consistency constraints, we design a tailored projection scheme whose computational cost scales linearly with the number of variables. In addition, we identify a hidden facial structure in the polyhedral-SDP relaxation, which enables us to eliminate a large number of linear constraints by restricting the matrix variable to affine subspaces corresponding to exposed faces of the semidefinite cone. The latter enables us to efficiently solve the factorized ALM subproblems over the affine subspaces. At each ALM iteration, we additionally carry out a single projected gradient step with respect to the original matrix variable to automatically adjust the rank and escape from spurious local minima when necessary. We also extend our RiNNAL-POP algorithmic framework to solve moment-SOS relaxations of POPs. Extensive numerical experiments on various benchmark problems demonstrate the robustness and efficiency of RiNNAL-POP in solving large-scale polyhedral-SDP relaxations.

math.OC↗

arXiv2025

A Quadratically Convergent Alternating Projection Method for Nonconvex Sets

In this paper, we consider the feasibility problem, which aims to find a feasible point for the constraint set $\{x \in \mathbb{R}^n: c(x) = 0\}$ over a possibly non-regular subset $\mathcal{X} \subset \mathbb{R}^n$. Under the constraint nondegeneracy condition, we propose a modified alternating projection method. In our proposed method, based on the concept of projective mapping for $\mathcal{X}$, we alternate a Newton step for finding an inexact solution within the limiting tangent cone of $\mathcal{X}$ and a projection to $\mathcal{X}$. Under mild conditions, we prove the local quadratic convergence of our proposed method. Preliminary numerical experiments demonstrate the high efficiency of our proposed alternating projection method.

math.OC↗

arXiv2025

Convex relaxation approaches for high-dimensional optimal transport

Optimal transport (OT) is a powerful tool in mathematics and data science but faces severe computational and statistical challenges in high dimensions. We propose convex relaxation approaches based on marginal and cluster moment relaxations that exploit locality in the distributions. These methods approximate high-dimensional couplings using low-order marginals and sparse moment statistics, yielding semidefinite programs that provide lower bounds on the OT cost with greatly reduced complexity. For Gaussian measures with sparse correlations, we prove an exponential convergence rate for the cluster moment relaxation and an improved statistical error bound. We also establish approximation error bounds for the marginal relaxation when the reference measures are local perturbations of mean-field measures. In addition, we demonstrate how to extract transport maps from our relaxations, offering a simpler and interpretable alternative to neural networks in generative modeling. Extensive numerical experiments demonstrate strong empirical performance across a range of distributions. Our results suggest that convex relaxations can provide a promising path for dimension reduction in high-dimensional OT.

math.OC↗

arXiv2025

RiNNAL+: a Riemannian ALM Solver for SDP-RLT Relaxations of Mixed-Binary Quadratic Programs

Doubly nonnegative (DNN) relaxation usually provides a tight lower bound for a mixed-binary quadratic program (MBQP). However, solving DNN problems is challenging because: (1) the problem size is $Ω((n+l)^2)$ for an MBQP with $n$ variables and $l$ inequality constraints, and (2) the rank of optimal solutions cannot be estimated a priori due to the absence of theoretical bounds. In this work, we propose RiNNAL+, a Riemannian augmented Lagrangian method (ALM) for solving DNN problems. We prove that the DNN relaxation of an MBQP, with matrix dimension $(n+l+1)$, is equivalent to the SDP-RLT relaxation (based on the reformulation-linearization technique) with a smaller matrix dimension $(n+1)$. In addition, we develop a hybrid method that alternates between two phases to solve the ALM subproblems. In phase one, we apply low-rank matrix factorization and random perturbation to transform the feasible region into a lower-dimensional manifold so that we can use the Riemannian gradient descent method. In phase two, we apply a single projected gradient step to update the rank of the underlying variable and escape from spurious local minima arising in the first phase if necessary. To reduce the computation cost of the projected gradient step, we develop pre-processing and warm-start techniques for acceleration. Unlike traditional rank-adaptive methods that require extensive parameter tuning, our hybrid method requires minimal tuning. Extensive experiments confirm the efficiency and robustness of RiNNAL+ in solving various classes of large-scale DNN problems.

math.OC↗

arXiv2025

Stiefel optimization is NP-hard

We show that linearly constrained linear optimization over a Stiefel or Grassmann manifold is NP-hard in general. We show that the same is true for unconstrained quadratic optimization over a Stiefel manifold. We will show that unless $\mathrm{P}=\mathrm{NP}$, these optimization problems over a Stiefel manifold do not have $\mathrm{FPTAS}$. As an aside we extend our results to flag manifolds. Combined with earlier findings, this shows that manifold optimization is a difficult endeavor -- even the simplest problems like LP and unconstrained QP are already NP-hard on the most common manifolds.

math.OC↗

arXiv2025

An Exact Penalty Approach for Equality Constrained Optimization over a Convex Set

In this paper, we consider the nonlinear constrained optimization problem (NCP) with constraint set $\{x \in \mathcal{X}: c(x) = 0\}$, where $\mathcal{X}$ is a closed convex subset of $\mathbb{R}^n$. We propose an exact penalty approach, named constraint dissolving approach, that transforms (NCP) into its corresponding constraint dissolving problem (CDP). The transformed problem (CDP) admits $\mathcal{X}$ as its feasible region with a locally Lipschitz smooth objective function. We prove that (NCP) and (CDP) share the same first-order stationary points, second-order stationary points, second-order sufficient condition (SOSC) points, and strong SOSC points, in a neighborhood of the feasible region. Moreover, we prove that these equivalences extend globally under a particular error bound condition. Therefore, our proposed constraint dissolving approach enables direct implementations of optimization approaches over $\mathcal{X}$ and inherits their convergence properties to solve problems that take the form of (NCP). Preliminary numerical experiments illustrate the high efficiency of directly applying existing solvers for optimization over $\mathcal{X}$ to solve (NCP) through (CDP). These numerical results further demonstrate the practical potential of our proposed constraint dissolving approach.

math.OC↗

arXiv2025

A low-rank augmented Lagrangian method for doubly nonnegative relaxations of mixed-binary quadratic programs

Doubly nonnegative (DNN) programming problems are known to be challenging to solve because of their huge number of $Ω(n^2)$ constraints and $Ω(n^2)$ variables. In this work, we introduce RNNAL, a method for solving DNN relaxations of large-scale mixed-binary quadratic programs by leveraging their solutions' possible low-rank property. RNNAL is a globally convergent Riemannian augmented Lagrangian method (ALM) that penalizes the nonnegativity and complementarity constraints while preserving all other constraints as an algebraic variety. After applying the low-rank decomposition to the ALM subproblem, its feasible region becomes an algebraic variety with favorable geometric properties. Our low-rank decomposition model is different from the standard Burer-Monteiro (BM) decomposition model in that we make the key improvement to equivalently reformulate most of the quadratic constraints after the BM decomposition into fewer and more manageable affine constraints. This modification is also important in helping us to alleviate the violation of Slater's condition for the primal DNN problem. Moreover, we make the crucial step to show that the metric projection onto the algebraic variety, although non-convex, can be transformed into a solvable convex optimization problem under certain regularity conditions, which can be ensured by a constraint-relaxation strategy. RNNAL is able to handle general semidefinite programming (SDP) with additional polyhedral cone constraints, thus serving as a prototype algorithm for solving general DNN problems. Numerous numerical experiments are conducted to validate the efficiency of the proposed RNNAL method.

math.OC↗

arXiv2025

A Bregman ADMM for Bethe variational problem

In this work, we propose a novel Bregman ADMM with nonlinear dual update to solve the Bethe variational problem (BVP), a key optimization formulation in graphical models and statistical physics. Our algorithm provides rigorous convergence guarantees, even if the objective function of BVP is non-convex and non-Lipschitz continuous on the boundary. A central result of our analysis is proving that the entries in local minima of BVP are strictly positive, effectively resolving non-smoothness issues caused by zero entries. Beyond theoretical guarantees, the algorithm possesses high level of separability and parallelizability to achieve highly efficient subproblem computation. Our Bregman ADMM can be easily extended to solve the quantum Bethe variational problem. Numerical experiments are conducted to validate the effectiveness and robustness of the proposed method. Based on this research, we have released an open-source package of the proposed method at https://github.com/TTYmath/BADMM-BVP.

math.OC↗

arXiv2024

Exploring chordal sparsity in semidefinite programming with sparse plus low-rank data matrices

Semidefinite programming (SDP) problems are challenging to solve because of their high dimensionality. However, solving sparse SDP problems with small tree-width are known to be relatively easier because: (1) they can be decomposed into smaller multi-block SDP problems through chordal conversion; (2) they have low-rank optimal solutions. In this paper, we study more general SDP problems whose coefficient matrices have sparse plus low-rank (SPLR) structure. We develop a unified framework to convert such problems into sparse SDP problems with bounded tree-width. Based on this, we derive rank bounds for SDP problems with SPLR structure, which are tight in the worst case.

math.OC↗

arXiv2024

Optimization over convex polyhedra via Hadamard parametrizations

In this paper, we study linearly constrained optimization problems (LCP). After applying Hadamard parametrization, the feasible set of the parametrized problem (LCPH) becomes an algebraic variety, with conducive geometric properties which we explore in depth. We derive explicit formulas for the tangent cones and second-order tangent sets associated with the parametrized polyhedra. Based on these formulas, we develop a procedure to recover the Lagrangian multipliers associated with the constraints to verify the optimality conditions of the given primal variable without requiring additional constraint qualifications. Moreover, we develop a systematic way to stratify the variety into a disjoint union of finitely many Riemannian manifolds. This leads us to develop a hybrid algorithm combining Riemannian optimization and projected gradient to solve (LCP) with convergence guarantees. Numerical experiments are conducted to verify the effectiveness of our method compared with various state-of-the-art algorithms.

math.OC↗

arXiv2023

A feasible method for general convex low-rank SDP problems

In this work, we consider the low rank decomposition (SDPR) of general convex semidefinite programming problems (SDP) that contain both a positive semidefinite matrix and a nonnegative vector as variables. We develop a rank-support-adaptive feasible method to solve (SDPR) based on Riemannian optimization. The method is able to escape from a saddle point to ensure its convergence to a global optimal solution for generic constraint vectors. We prove its global convergence and local linear convergence without assuming that the objective function is twice differentiable. Due to the special structure of the low-rank SDP problem, our algorithm can achieve better iteration complexity than existing results for more general smooth nonconvex problems. In order to overcome the degeneracy issues of SDP problems, we develop two strategies based on random perturbation and dual refinement. These techniques enable us to solve some primal degenerate SDP problems efficiently, for example, Lovász theta SDPs. Our work is a step forward in extending the application range of Riemannian optimization approaches for solving SDP problems. Numerical experiments are conducted to verify the efficiency and robustness of our method.

math.OC↗

arXiv2023

Self-adaptive ADMM for semi-strongly convex problems

In this paper, we develop a self-adaptive ADMM that updates the penalty parameter adaptively. When one part of the objective function is strongly convex i.e., the problem is semi-strongly convex, our algorithm can update the penalty parameter adaptively with guaranteed convergence. We establish various types of convergence results including accelerated convergence rate of O(1/k^2), linear convergence and convergence of iteration points. This enhances various previous results because we allow the penalty parameter to change adaptively. We also develop a partial proximal point method with the subproblem solved by our adaptive ADMM. This enables us to solve problems without semi-strongly convex property. Numerical experiments are conducted to demonstrate the high efficiency and robustness of our method.

math.OC↗

arXiv2023

A Riemannian Dimension-reduced Second Order Method with Application in Sensor Network Localization

In this paper, we propose a cubic-regularized Riemannian optimization method (RDRSOM), which partially exploits the second order information and achieves the iteration complexity of $\mathcal{O}(1/ε^{3/2})$. In order to reduce the per-iteration computational cost, we further propose a practical version of (RDRSOM), which is an extension of the well known Barzilai-Borwein method and achieves the iteration complexity of $\mathcal{O}(1/ε^{3/2})$. We apply our method to solve a nonlinear formulation of the wireless sensor network localization problem whose feasible set is a Riemannian manifold that has not been considered in the literature before. Numerical experiments are conducted to verify the high efficiency of our algorithm compared to state-of-the-art Riemannian optimization methods and other nonlinear solvers.

math.OC↗

arXiv2023

A feasible method for solving an SDP relaxation of the quadratic knapsack problem

In this paper, we consider an SDP relaxation of the quadratic knapsack problem (QKP). After using the Burer-Monteiro factorization, we get a non-convex optimization problem, whose feasible region is an algebraic variety. Although there might be non-regular points on the algebraic variety, we prove that the algebraic variety is a smooth manifold except for a trivial point for a generic input data. We also analyze the local geometric properties of non-regular points on this algebraic variety. In order to maintain the equivalence between the SDP problem and its non-convex formulation, we derive a new rank condition under which these two problems are equivalent. This new rank condition can be much weaker than the classical rank condition if the coefficient matrix has certain special structures. We also prove that under an appropriate rank condition, any second order stationary point of the non-convex problem is also a global optimal solution without any regularity assumption. This result is distinguished from previous results based on LICQ-like smoothness assumption. With all these theoretical properties, we design an algorithm that equip a manifold optimization method with a strategy to escape from non-optimal non-regular points. Our algorithm can also be used as a heuristic for solving the quadratic knapsack problem. Numerical experiments are conducted to verify the high efficiency and robustness of our algorithm as compared to other SDP solvers and a heuristic method based on dynamic programming. In particular, our algorithm is able to solve the SDP relaxation of a one-million dimensional QKP with a sparse cost matrix very accurately in about 20 minutes on a modest desktop computer.

math.OC↗

arXiv2022

Solving graph equipartition SDPs on an algebraic variety

Semidefinite programs are generally challenging to solve due to their high dimensionality. Burer and Monteiro developed a non-convex approach to solve linear SDP problems by applying its low rank property. Their approach is fast because they used factorization to reduce the problem size. In this paper, we focus on solving the SDP relaxation of a graph equipartition problem, which involves an additional semidefinite upper bound constraint over the traditional linear SDP. By applying the factorization approach, we get a non-convex problem with an additional non-smooth spectral inequality constraint. We discuss when the non-convex problem is equivalent to the original SDP, and when a second order stationary point of the non-convex problem is also a global minimum. Our results generalize previous works on smooth non-convex factorization approaches for linear SDP to the non-smooth case. Moreover, the constraints of the non-convex problem involve an algebraic variety with some conducive properties that allow us to use Riemannian optimization techniques and non-convex augmented Lagrangian method to solve the SDP problem very efficiently with certified global optimality.

math.OC↗