arXiv subjects

Henry Wolkowicz

Publications and source records attributed to Henry Wolkowicz.

At least 19 recordsLinked to original sources

arXiv2025

Optimal Diagonal Preconditioning Beyond Worst-Case Conditioning: Theory and Practice of Omega Scaling

We study optimal diagonal preconditioning using the classical worst-case $\kappa$-condition number and the averaging-based $\omega$-condition number. For the $\kappa$-optimal preconditioning problem, we derive an affine-based pseudoconvex reformulation with three key advantages: all stationary points are global minima, subgradients are inexpensive to compute, and the optimization variable is an $n$-dimensional vector rather than an $n\times n$ matrix as in semidefinite programming (SDP) approaches. We develop a simple and highly efficient subgradient method, with convergence guarantees, for solving this pseudoconvex formulation that is substantially more scalable and accurate than existing SDP-based methods. For the $\omega$-condition number, we provide explicit characterizations of optimal diagonal and block diagonal preconditioners. In particular, we show that several classical preconditioners, including Jacobi and row/column normalization, are $\omega$-optimal, and that matrix balancing schemes monotonically reduce $\omega$ and converge to stationary points of the two-sided problem. To the best of our knowledge, this is the first unified and explicit characterization of optimality conditions for both $\kappa$ and $\omega$-based preconditioning. Our numerical experiments further reveal a striking phenomenon: although $\kappa$-optimal preconditioners achieve stronger reductions in the worst-case condition number, $\omega$-optimal preconditioners are substantially cheaper to compute and yield better performance for iterative methods such as preconditioned conjugate gradient (PCG) and least squares method (LSQR). Moreover, applying $\omega$-optimal scaling to linear systems that are already $\kappa$-optimally preconditioned leads to further improvements in PCG iterations.

math.OC↗

arXiv2025

Finding Maximum Determinant Principal Submatrices via Hadamard Bounds and Projection Methods

An important yet challenging problem in numerical linear algebra is finding a principal submatrix with maximum determinant from a given symmetric positive semidefinite matrix. This problem arises in experimental design, statistics, and machine learning. We study several exact and approximate approaches to this problem. We first derive an upper bound based on Hadamard's inequality, along with a projection scheme based on the Gram--Schmidt process without normalization. This combination yields a highly effective upper bound and leads to an exact branch-and-bound algorithm for moderate-sized instances. For larger scale problems we propose a continuous relaxation that facilitates reliable performance evaluation when the exact method returns only near-optimal solutions. We further prove that the projection scheme strengthens the upper bound derived from this relaxation. Numerical experiments demonstrate the effectiveness of the proposed methods across a broad range of datasets.

math.OC↗

arXiv2025

A necessary condition for the guarantee of the superiorization method

We study a method that involves principally convex feasibility-seeking and makes secondary efforts of objective function value reduction. This is the well-known superiorization method (SM), where the iterates of an asymptotically convergent iterative feasibility-seeking algorithm are perturbed by objective function nonascent steps. We investigate the question under what conditions a sequence generated by an SM algorithm asymptotically converges to a feasible point whose objective function value is superior (meaning smaller or equal) to that of a feasible point reached by the corresponding unperturbed one (i.e., the exactly same feasibility-seeking algorithm that the SM algorithm employs.) This question is yet only partially answered in the literature. We present a condition under which an SM algorithm that uses negative gradient descent steps in its perturbations fails to yield such a superior outcome. The significance of the discovery of this negative condition is that it necessitates that the inverse of this condition will have to be assumed to hold in any future guarantee result for the SM. The condition is important for practitioners who use the SM because it is avoidable in experimental work with the SM, thus increasing the success rate of the method in real-world applications.

math.OC↗

arXiv2024

On the local and global minimizers of the smooth stress function in Euclidean Distance Matrix problems

We consider the nonconvex minimization problem, with quartic objective function, that arises in the exact recovery of a configuration matrix $P\in \R^{nd}$ of $n$ points when a Euclidean distance matrix, \EDMp, is given with embedding dimension $d$. It is an open question in the literature whether there are conditions such that the minimization problem admits a local nonglobal minimizer, \lngmp. We prove that all second-order stationary points are global minimizers whenever $n \leq d + 1$. {And, for $d=1$ and $n\geq 7>d+1$, we present an example where we can analytically exhibit a local nonglobal minimizer. For more general cases,} we numerically find a second-order stationary point and then prove that there indeed exists a nearby \lngm for the quartic nonconvex minimization problem. Thus, we answer the previously open question about their existence in the affirmative. Our approach to finding the \lngm is novel in that we first exploit the translation and rotation invariance to remove the singularities of the Hessian, and reduce the size of the problem from $nd$ variables in $P$ to $(n-1)d - d(d-1)/2$ variables. This allows for stabilizing Newton's method, and for finding examples that satisfy the strict second order sufficient optimality conditions. The motivation for being able to find global minima is to obtain \emph{exact recovery} of the configuration matrix, even in the cases where the data is noisy and/or incomplete, without resorting to approximating solutions from convex (semidefinite programming) relaxations. In the process of our work we present new insights into when \lngmp s of the smooth stress function do and do not exist.

math.OC↗

arXiv2024

Projection, Degeneracy, and Singularity Degree for Spectrahedra

Facial reduction, FR, is a regularization technique for convex programs where the strict feasibility constraint qualification, CQ, fails.Though this CQ holds generically, failure is pervasive in applications such as semidefinite relaxations of hard discrete optimization problems. In this paper we relate FR to the analysis of the convergence behaviour of a semismooth Newton root finding method for the projection onto a spectrahedron, i.e., onto the intersection of a linear manifold and the semidefinite cone. We examine the effect of failure of strict feasibility on the projection problem. In the process, we derive an elegant formula for the projection onto a face of the semidefinite cone obtained via regularization and discuss pathologies that arise in the absence of strict feasibility. We show further that the ill-conditioning of the Jacobian of the Newton method near optimality characterizes the degeneracy of the nearest point in the spectrahedron. We apply the results, both theoretically and empirically, to the problem of finding nearest points to the sets of: (i) correlation matrices or the elliptope; and (ii) semidefinite relaxations of permutation matrices or the vontope, i.e., the feasible sets for the semidefinite relaxations of the max-cut and quadratic assignment problems, respectively.

math.OC↗

arXiv2024

Single Element Error Correction/ in a Euclidean Distance Matrix

We consider the \emph{exact} error correction of a noisy Euclidean distance matrix, EDM, where the elements are the squared distances between $n$ points in $R^d$. For our problem we are given two facts: (i) the embedding dimension, $d$, (ii) \emph{exactly one} distance in the data is corrupted by \emph{nonzero noise}. But we do \underline{not} know the magnitude nor position of the noise. Thus there is a combinatorial element to the problem. We present three solution techniques. These use three divide and conquer strategies in combination with three versions of facial reduction that use: exposing vectors, facial vectors, and Gale transforms. This sheds light on the connections between the various forms of facial reduction related to Gale transforms. Our highly successful empirics confirm the success of these approaches as we can solve huge problems of the order of $100,000$ nodes in approximately one minute to machine precision. \\Our algorithm depends on identifying whether a principal submatrix of the \EDM contains the corrupted element. We provide a theorem for doing this that is related to the existing results for identifying \emph{yielding} elements, i.e.,~we provide a characterization for guaranteeing the perturbed EDM remains an EDM with embedding dimension $d$. The characterization is particularly simple in the $d=2$ case. \\In addition, we characterize when the intuitive approach of the nearest EDM problem, solves our problem. In fact, we show that this happens if, and only if, the original distance element is $0$, degenerate, and the perturbation is negative.

math.OC↗

arXiv2023

Exact Solutions for the NP-hard Wasserstein Barycenter Problem using a Doubly Nonnegative Relaxation and a Splitting Method

The so-called \emph{simplified} Wasserstein barycenter problem, also known as the cheapest hub problem, consists in selecting one point from each of $k$ given sets, each set consisting of $n$ points, with the aim of minimizing the sum of distances to the barycenter of the $k$ chosen points. This problem is known to be NP-hard. We compute the Wasserstein barycenter by exploiting the Euclidean distance matrix structure to obtain a facially reduced doubly nonnegative, DNN, relaxation. The facial reduction provides a natural splitting for applying the symmetric alternating directions method of multipliers (sADMM) to the DNN relaxation. The sADMM method exploits structure in the subproblems to find strong upper and lower bounds. In addition, we extend the problem to allow varying $n_j$ points for the $j$-th set. The purpose of this paper is twofold. First we want to illustrate the strength of this DNN relaxation with the natural splitting approach mentioned above. Our numerical tests then illustrate the surprising success on random problems, as we generally, efficiently, find the provable exact solution of this NP-hard problem. Comparisons with current commercial software illustrate this surprising efficiency. However, we demonstrate and prove that there is a duality gap for problems with \emph{enough} multiple optimal solutions, and that this arises from problems with highly symmetrized structure.

math.OC↗

arXiv2023

The $\omega$-Condition Number: Applications to Optimal Preconditioning and Low Rank Generalized Jacobian Updating

Preconditioning is essential in iterative methods for solving linear systems. It is also the implicit objective in updating approximations of Jacobians in optimization methods, e.g.,in quasi-Newton methods. Motivated by the latter, we study a nonclassic matrix condition number, the $\omega$-condition number, $\omega$ for short. $\omega$ is the ratio of the arithmetic and geometric means of the singular values, rather than largest and smallest. Moreover, unlike the latter classical $\kappa$ condition number, $\omega$ is not invariant under inversion, an important point that allows one to recall that it is the conditioning of the inverse that is important. Our study is in the context of optimal conditioning for: (i) low rank updating of generalized Jacobians arising in the context of nonsmooth Newton methods; and (ii) iterative methods for linear systems; (iia) clustering of eigenvalues; (iib) convergence rates; and (iic) estimating the actual condition of a linear system. We emphasize that the simple functions in $\omega$ allow one to exploit optimality conditions and derive explicit formulae for $\omega$-optimal preconditioners of special structure. Connections to partial Cholesky type sparse preconditioners are made that modify the iterates of Cholesky decomposition by including the entire diagonal at each iteration. Our results confirm the efficacy of using the $\omega$-condition number compared to the classical $\kappa$-condition number.

math.NA↗

arXiv2023

Regularized Nonsmooth Newton Algorithms for Best Approximation

We consider the problem of finding the best approximation point from a polyhedral set, and its applications, in particular to solving large-scale linear programs. The classical projection problem has many various and many applications. We study a regularized nonsmooth Newton type solution method where the Jacobian is singular; and we compare the computational performance to that of the classical projection method of Halperin-Lions-Wittmann-Bauschke (HLWB). We observe empirically that the regularized nonsmooth method significantly outperforms the HLWB method. However, the HLWB has a convergence guarantee while the nonsmooth method is not monotonic and does not guarantee convergence due in part to singularity of the generalized Jacobian. Our application to solving large-scale linear programs uses a parametrized projection problem. This leads to a \emph{stepping stone external path following} algorithm. Other applications are finding triangles from branch and bound methods, and generalized constrained linear least squares. We include scaling methods that improve the efficiency and robustness.

math.OC↗

arXiv2023

A Peaceman-Rachford Splitting Method for the Protein Side-Chain Positioning Problem

We formulate a doubly nonnegative (DNN) relaxation of the protein side-chain positioning (SCP) problem. We inherit the natural splitting of variables that stems from the facial reduction technique in the semidefinite relaxation. We solve the relaxation using a variant of the Peaceman-Rachford splitting method. Our numerical experiments show that we solve almost all instances of the NP-hard SCP problem to optimality.

math.OC↗

arXiv2023

Revisiting Degeneracy, Strict Feasibility, Stability, in Linear Programming

Currently, the simplex method and the interior point method are indisputably the most popular algorithms for solving linear programs, LPs. Unlike general conic programs, LPs with a finite optimal value do not require strict feasibility in order to establish strong duality. Hence strict feasibility is seldom a concern, even though strict feasibility is equivalent to stability and a compact dual optimal set. This lack of concern is also true for other types of degeneracy of basic feasible solutions in LP. In this paper we discuss that the specific degeneracy that arises from lack of strict feasibility necessarily causes difficulties in both simplex and interior point methods. In particular, we show that the lack of strict feasibility implies that every basic feasible solution, BFS, is degenerate; thus conversely, the existence of a nondegenerate BFS implies that strict feasibility (regularity) holds. We prove the results using facial reduction and simple linear algebra. In particular, the facially reduced system reveals the implicit non-surjectivity of the linear map of the equality constraint system. As a consequence, we emphasize that facial reduction involves two steps where, the first guarantees strict feasibility, and the second recovers full row rank of the constraint matrix. This illustrates the implicit singularity of problems where strict feasibility fails, and also helps in obtaining new efficient techniques for preproccessing. We include an efficient preprocessing method that can be performed as an extension of phase-I of the two-phase simplex method. We show that this can be used to avoid the loss of precision for many well known problem sets in the literature, e.g., the NETLIB problem set.

math.OC↗

arXiv2022

Singularity degree of non-facially exposed faces

In this paper, we study the facial structure of the linear image of a cone. We define the singularity degree of a face of a cone to be the minimum number of steps it takes to expose it using exposing vectors from the dual cone. We show that the singularity degree of the linear image of a cone is exactly the number of facial reduction steps to obtain the minimal face in a corresponding primal conic optimization problem. This result generalizes the relationship between the complexity of general facial reduction algorithms and facial exposedness of conic images under a linear transform by Drusvyatskiy, Pataki and Wolkowicz to arbitrary singularity degree. We present our results in the original form and also in its nullspace form. As a by-product, we show that frameworks underlying a chordal graph have at most one level of stress matrix.

math.OC↗

arXiv2022

Robust Interior Point Method for Quantum Key Distribution Rate Computation

Security proof methods for quantum key distribution, QKD, that are based on the numerical key rate calculation problem, are powerful in principle. However, the practicality of the methods are limited by computational resources and the efficiency and accuracy of the underlying algorithms for convex optimization. We derive a stable reformulation of the convex nonlinear semidefinite programming, SDP, model for the key rate calculation problems. We use this to develop an efficient, accurate algorithm. The stable reformulation is based on novel forms of facial reduction, FR, for both the linear constraints and nonlinear quantum relative entropy objective function. This allows for a Gauss-Newton type interior-point approach that avoids the need for perturbations to obtain strict feasibility, a technique currently used in the literature. The result is high accuracy solutions with theoretically proven lower bounds for the original QKD from the FR stable reformulation. This provides novel contributions for FR for general SDP. We report on empirical results that dramatically improve on speed and accuracy, as well as solving previously intractable problems.

quant-ph↗

arXiv2022

Facial Reduction for Symmetry Reduced Semidefinite Doubly Nonnegative Programs

We consider both facial reduction, \FRp, and symmetry reduction, \SRp, techniques for semidefinite programming, \SDPp. We show that the two together fit surprisingly well in an alternating direction method of multipliers, \ADMMp, approach. In fact, this approach allows for simply adding on nonnegativity constraints, and solving the doubly nonnegative, \DNN, relaxation of many classes of hard combinatorial problems. We also show that the singularity degree remains the same after \SRp, and that the \DNN relaxations considered here have singularity degree one, that is reduced to zero after \FRp. The combination of \FR and \SR leads to a significant improvement in both numerical stability and running time for both the \ADMM and interior point approaches. We test our method on various \DNN relaxations of hard combinatorial problems including quadratic assignment problems with sizes of more than $n=500$. This translates to a semidefinite constraint of order $250,000$ and $625\times 10^8$ nonnegative constrained variables, before applying the reduction techniques.

math.OC↗

arXiv2021

A Semismooth Newton-Type Method for the Nearest Doubly Stochastic Matrix Problem

We study a semismooth Newton-type method for the nearest doubly stochastic matrix problem where both differentiability and nonsingularity of the Jacobian can fail. The optimality conditions for this problem are formulated as a system of strongly semismooth functions. We show that the so-called local error bound condition does not hold for this system. Thus the guaranteed convergence rate of Newton-type methods is at most superlinear. By exploiting the problem structure, we construct a modified two step semismooth Newton method that guarantees a nonsingular Jacobian matrix at each iteration, and that converges to the nearest doubly stochastic matrix quadratically. To the best of our knowledge, this is the first Newton-type method which converges $Q$-quadratically in the absence of the local error bound condition.

math.OC↗

arXiv2020

A Restricted Dual Peaceman-Rachford Splitting Method for QAP

We revisit and strengthen splitting methods for solving doubly nonnegative, DNN, relaxations of the quadratic assignment problem, QAP. We use a modified restricted contractive splitting method, PRSM, approach. Our strengthened bounds and new dual multiplier estimates improve on the bounds and convergence results in the literature.

math.OC↗

arXiv2019

Error Bounds and Singularity Degree in Semidefinite Programming

In semidefinite programming a proposed optimal solution may be quite poor in spite of having sufficiently small residual in the optimality conditions. This issue may be framed in terms of the discrepancy between forward error (the unmeasurable `true error') and backward error (the measurable violation of optimality conditions). In his seminal work, Sturm provided an upper bound on forward error in terms of backward error and singularity degree. In this paper we provide a method to bound the maximum rank over all solutions and use this result to obtain a lower bound on forward error for a class of convergent sequences. This lower bound complements the upper bound of Sturm. The results of Sturm imply that semidefinite programs with slow convergence necessarily have large singularity degree. Here we show that large singularity degree is, in some sense, also a sufficient condition for slow convergence for a family of external-type `central' paths. Our results are supported by numerical observations.

math.OC↗

arXiv2019

Parametric Convex Quadratic Relaxation of the Quadratic Knapsac Problem

We consider a parametric convex quadratic programming, CQP, relaxation for the quadratic knapsack problem, QKP. This relaxation maintains partial quadratic information from the original QKP by perturbing the objective function to obtain a concave quadratic term. The nonconcave part generated by the perturbation is then linearized by a standard approach that lifts the problem to the matrix space. We present a primal-dual interior point method to optimize the perturbation of the quadratic function, in a search for the tightest upper bound for the \QKPp. We prove that the same perturbation approach, when applied in the context of semidefinite programming, SDP, relaxations of the QKP, cannot improve the upper bound given by the corresponding linear \SDP relaxation. The result also applies to more general integer quadratic problems. Finally, we propose new valid inequalities on the lifted matrix variable, derived from cover and knapsack inequalities for the QKP, and present the separation problems to generate cuts for the current solution of the CQP relaxation. Our best bounds are obtained from alternating between optimizing the parametric quadratic relaxation over the perturbation and adding cutting planes generated by the valid inequalities proposed.

math.OC↗