arXiv subjects

Pablo A. Parrilo

Publications and source records attributed to Pablo A. Parrilo.

At least 37 records · Page 2Linked to original sources

arXiv2021

On the local stability of semidefinite relaxations

We consider a parametric family of quadratically constrained quadratic programs (QCQP) and their associated semidefinite programming (SDP) relaxations. Given a nominal value of the parameter at which the SDP relaxation is exact, we study conditions (and quantitative bounds) under which the relaxation will continue to be exact as the parameter moves in a neighborhood around the nominal value. Our framework captures a wide array of statistical estimation problems including tensor principal component analysis, rotation synchronization, orthogonal Procrustes, camera triangulation and resectioning, essential matrix estimation, system identification, and approximate GCD. Our results can also be used to analyze the stability of SOS relaxations of general polynomial optimization problems.

math.OC↗

arXiv2021

Near-linear convergence of the Random Osborne algorithm for Matrix Balancing

We revisit Matrix Balancing, a pre-conditioning task used ubiquitously for computing eigenvalues and matrix exponentials. Since 1960, Osborne's algorithm has been the practitioners' algorithm of choice and is now implemented in most numerical software packages. However, its theoretical properties are not well understood. Here, we show that a simple random variant of Osborne's algorithm converges in near-linear time in the input sparsity. Specifically, it balances $K\in\mathbb{R}_{\geq 0}^{n\times n}$ after $O(mε^{-2}\logκ)$ arithmetic operations, where $m$ is the number of nonzeros in $K$, $ε$ is the $\ell_1$ accuracy, and $κ=\sum_{ij}K_{ij}/(\min_{ij:K_{ij}\neq 0}K_{ij})$ measures the conditioning of $K$. Previous work had established near-linear runtimes either only for $\ell_2$ accuracy (a weaker criterion which is less relevant for applications), or through an entirely different algorithm based on (currently) impractical Laplacian solvers. We further show that if the graph with adjacency matrix $K$ is moderately connected--e.g., if $K$ has at least one positive row/column pair--then Osborne's algorithm initially converges exponentially fast, yielding an improved runtime $O(mε^{-1}\logκ)$. We also address numerical precision by showing that these runtime bounds still hold when using $O(\log(nκ/ε))$-bit numbers. Our results are established through an intuitive potential argument that leverages a convex optimization perspective of Osborne's algorithm, and relates the per-iteration progress to the current imbalance as measured in Hellinger distance. Unlike previous analyses, we critically exploit log-convexity of the potential. Our analysis extends to other variants of Osborne's algorithm: along the way, we establish significantly improved runtime bounds for cyclic, greedy, and parallelized variants.

math.OC↗

arXiv2021

Kernel approximation on algebraic varieties

Low-rank approximation of kernels is a fundamental mathematical problem with widespread algorithmic applications. Often the kernel is restricted to an algebraic variety, e.g., in problems involving sparse or low-rank data. We show that significantly better approximations are obtainable in this setting: the rank required to achieve a given error depends on the variety's dimension rather than the ambient dimension, which is typically much larger. This is true in both high-precision and high-dimensional regimes. Our results are presented for smooth isotropic kernels, the predominant class of kernels used in applications. Our main technical insight is to approximate smooth kernels by polynomial kernels, and leverage two key properties of polynomial kernels that hold when they are restricted to a variety. First, their ranks decrease exponentially in the variety's co-dimension. Second, their maximum values are governed by their values over a small set of points. Together, our results provide a general approach for exploiting (approximate) "algebraic structure" in datasets in order to efficiently solve large-scale data science problems.

cs.LG↗

arXiv2021

On Approximations of the PSD Cone by a Polynomial Number of Smaller-sized PSD Cones

We study the problem of approximating the cone of positive semidefinite (PSD) matrices with a cone that can be described by smaller-sized PSD constraints. Specifically, we ask the question: "how closely can we approximate the set of unit-trace $n \times n$ PSD matrices, denoted by $D$, using at most $N$ number of $k \times k$ PSD constraints?" In this paper, we prove lower bounds on $N$ to achieve a good approximation of $D$ by considering two constructions of an approximating set. First, we consider the unit-trace $n \times n$ symmetric matrices that are PSD when restricted to a fixed set of $k$-dimensional subspaces in $\mathbb{RR}^n$. We prove that if this set is a good approximation of $D$, then the number of subspaces must be at least exponentially large in $n$ for any $k = o(n)$. % Second, we show that any set $S$ that approximates $D$ within a constant approximation ratio must have superpolynomial $\mathbf{S}_+^k$-extension complexity. To be more precise, if $S$ is a constant factor approximation of $D$, then $S$ must have $\mathbf{S}_+^k$-extension complexity at least $\exp( C \cdot \min \{ \sqrt{n}, n/k \})$ where $C$ is some absolute constant. In addition, we show that any set $S$ such that $D \subseteq S$ and the Gaussian width of $D$ is at most a constant times larger than the Gaussian width of $D$ must have $\mathbf{S}_+^k$-extension complexity at least $\exp( C \cdot \min \{ n^{1/3}, \sqrt{n/k} \})$. These results imply that the cone of $n \times n$ PSD matrices cannot be approximated by a polynomial number of $k \times k$ PSD constraints for any $k = o(n / \log^2 n)$. These results generalize the recent work of Fawzi on the hardness of polyhedral approximations of $\mathbf{S}_+^n$, which corresponds to the special case with $k=1$.

math.OC↗

arXiv2021

Semidefinite Relaxations of Products of Nonnegative Forms on the Sphere

We study the problem of maximizing the geometric mean of $d$ low-degree non-negative forms on the real or complex sphere in $n$ variables. We show that this highly non-convex problem is NP-hard even when the forms are quadratic and is equivalent to optimizing a homogeneous polynomial of degree $O(d)$ on the sphere. The standard Sum-of-Squares based convex relaxation for this polynomial optimization problem requires solving a semidefinite program (SDP) of size $n^{O(d)}$, with multiplicative approximation guarantees of $Ω(\frac{1}{n})$. We exploit the compact representation of this polynomial to introduce a SDP relaxation of size polynomial in $n$ and $d$, and prove that it achieves a constant factor multiplicative approximation when maximizing the geometric mean of non-negative quadratic forms. We also show that this analysis is asymptotically tight, with a sequence of instances where the gap between the relaxation and true optimum approaches this constant factor as $d \rightarrow \infty$. Next we propose a series of intermediate relaxations of increasing complexity that interpolate to the full Sum-of-Squares relaxation, as well as a rounding algorithm that finds an approximate solution from the solution of any intermediate relaxation. Finally we show that this approach can be generalized for relaxations of products of non-negative forms of any degree.

math.OC↗

arXiv2021

Maximizing Products of Linear Forms, and The Permanent of Positive Semidefinite Matrices

We study the convex relaxation of a polynomial optimization problem, maximizing a product of linear forms over the complex sphere. We show that this convex program is also a relaxation of the permanent of Hermitian positive semidefinite (HPSD) matrices. By analyzing a constructive randomized rounding algorithm, we obtain an improved multiplicative approximation factor to the permanent of HPSD matrices, as well as computationally efficient certificates for this approximation. We also propose an analog of van der Waerden's conjecture for HPSD matrices, where the polynomial optimization problem is interpreted as a relaxation of the permanent.

math.OC↗

arXiv2020

Sums of squares in Macaulay2

The Macaulay2 package SumsOfSquares decomposes polynomials as sums of squares. It is based on methods to rationalize sum-of-squares decompositions due to Parrilo and Peyrl. The package features a data type for sums-of-squares polynomials, support for external semidefinite programming solvers, and optimization over varieties.

math.OC↗

arXiv2019

Convergence Rate of Block-Coordinate Maximization Burer-Monteiro Method for Solving Large SDPs

Semidefinite programming (SDP) with diagonal constraints arise in many optimization problems, such as Max-Cut, community detection and group synchronization. Although SDPs can be solved to arbitrary precision in polynomial time, generic convex solvers do not scale well with the dimension of the problem. In order to address this issue, Burer and Monteiro proposed to reduce the dimension of the problem by appealing to a low-rank factorization and solve the subsequent non-convex problem instead. In this paper, we present coordinate ascent based methods to solve this non-convex problem with provable convergence guarantees. More specifically, we prove that the block-coordinate maximization algorithm applied to the non-convex Burer-Monteiro method globally converges to a first-order stationary point with a sublinear rate without any assumptions on the problem. We further show that this algorithm converges linearly around a local maximum provided that the objective function exhibits quadratic decay. We establish that this condition generically holds when the rank of the factorization is sufficiently large. Furthermore, incorporating Lanczos method to the block-coordinate maximization, we propose an algorithm that is guaranteed to return a solution that provides $1-O(1/r)$ approximation to the original SDP without any assumptions, where $r$ is the rank of the factorization. This approximation ratio is known to be optimal (up to constants) under the unique games conjecture, and we can explicitly quantify the number of iterations to obtain such a solution.

math.OC↗

arXiv2019

Minimum-Strain Symmetrization of Bravais Lattices

Bravais lattices are the most fundamental building blocks of crystallography. They are classified into groups according to their translational, rotational, and inversion symmetries. In computational analysis of Bravais lattices, fulfilment of symmetry conditions is usually determined by analysis of the metric tensor, using either a numerical tolerance to produce a binary (i.e. yes or no) classification, or a distance function which quantifies the deviation from an ideal lattice type. The metric tensor, though, is not scale-invariant, which complicates the choice of threshold and the interpretation of the distance function. Here, we quantify the distance of a lattice from a target Bravais class using strain. For an arbitrary lattice, we find the minimum-strain transformation needed to fulfil the symmetry conditions of a desired Bravais lattice type; the norm of the strain tensor is used to quantify the degree of symmetry breaking. The resulting distance is invariant to scale and rotation, and is a physically intuitive quantity. By symmetrizing to all Bravais classes, each lattice can be placed in a 14 dimensional space, which we use to create a map of the space of Bravais lattices and the transformation paths between them. A software implementation is available online under a permissive license.

cond-mat.mtrl-sci↗

arXiv2019

An entropy-based bound for the computational complexity of a switched system

The joint spectral radius (JSR) of a set of matrices characterizes the maximal asymptotic growth rate of an infinite product of matrices of the set. This quantity appears in a number of applications including the stability of switched and hybrid systems. A popular method used for the stability analysis of these systems searches for a Lyapunov function with convex optimization tools. We analyse the accuracy of this method for constrained switched systems, a class of systems that has attracted increasing attention recently. We provide a new guarantee for the upper bound provided by the sum of squares implementation of the method. This guarantee relies on the p-radius of the system and the entropy of the language of allowed switching sequences. We end this paper with a method to reduce the computation of the JSR of low rank matrices to the computation of the constrained JSR of matrices of small dimension.

math.OC↗

arXiv2019

Lyapunov Exponent of Rank One Matrices: Ergodic Formula and Inapproximability of the Optimal Distribution

The Lyapunov exponent corresponding to a set of square matrices $\mathcal{A} = \{A_1, \dots, A_n \}$ and a probability distribution $p$ over $\{1, \dots, n\}$ is $λ(\mathcal{A},p) := \lim_{k \to \infty} \frac{1}{k} \,\mathbb{E} \log \|A_{σ_k} \cdots A_{σ_2}A_{σ_1}\|$, where $σ_i$ are i.i.d. according to $p$. This quantity is of fundamental importance to control theory since it determines the asymptotic convergence rate $e^{λ(\mathcal{A},p)}$ of the stochastic linear dynamical system $x_{k+1} = A_{σ_k} x_k$. This paper investigates the following "design problem": given $\mathcal{A}$, compute the distribution $p$ minimizing $λ(\mathcal{A},p)$. Our main result is that it is NP-hard to decide whether there exists a distribution $p$ for which $λ(\mathcal{A},p)< 0$, i.e. it is NP-hard to decide whether this dynamical system can be stabilized. This hardness result holds even in the "simple"' case where $\mathcal{A}$ contains only rank-one matrices. Somewhat surprisingly, this is in stark contrast to the Joint Spectral Radius -- the deterministic kindred of the Lyapunov exponent -- for which the analogous optimization problem for rank-one matrices is known to be exactly computable in polynomial time. To prove this hardness result, we first observe via Birkhoff's Ergodic Theorem that the Lyapunov exponent of rank-one matrices admits a simple formula and in fact is a quadratic form in $p$. Hardness of the design problem is shown through a reduction from the Independent Set problem. Along the way, simple examples are given illustrating that $p \mapsto λ(\mathcal{A},p)$ is neither convex nor concave in general. We conclude with extensions to continuous distributions, exchangeable processes, Markov processes, and stationary ergodic processes.

math.OC↗

arXiv2019

Dimension reduction for semidefinite programs via Jordan algebras

We propose a new method for simplifying semidefinite programs (SDP) inspired by symmetry reduction. Specifically, we show if an orthogonal projection map satisfies certain invariance conditions, restricting to its range yields an equivalent primal-dual pair over a lower-dimensional symmetric cone---namely, the cone-of-squares of a Jordan subalgebra of symmetric matrices. We present a simple algorithm for minimizing the rank of this projection and hence the dimension of this subalgebra. We also show that minimizing rank optimizes the direct-sum decomposition of the algebra into simple ideals, yielding an optimal "block-diagonalization" of the SDP. Finally, we give combinatorial versions of our algorithm that execute at reduced computational cost and illustrate effectiveness of an implementation on examples. Through the theory of Jordan algebras, the proposed method easily extends to linear and second-order-cone programming and, more generally, symmetric cone optimization.

math.OC↗

arXiv2018

Semidefinite approximations of the matrix logarithm

The matrix logarithm, when applied to Hermitian positive definite matrices, is concave with respect to the positive semidefinite order. This operator concavity property leads to numerous concavity and convexity results for other matrix functions, many of which are of importance in quantum information theory. In this paper we show how to approximate the matrix logarithm with functions that preserve operator concavity and can be described using the feasible regions of semidefinite optimization problems of fairly small size. Such approximations allow us to use off-the-shelf semidefinite optimization solvers for convex optimization problems involving the matrix logarithm and related functions, such as the quantum relative entropy. The basic ingredients of our approach apply, beyond the matrix logarithm, to functions that are operator concave and operator monotone. As such, we introduce strategies for constructing semidefinite approximations that we expect will be useful, more generally, for studying the approximation power of functions with small semidefinite representations.

math.OC↗

arXiv2017

Sum of squares certificates for stability of planar, homogeneous, and switched systems

We show that existence of a global polynomial Lyapunov function for a homogeneous polynomial vector field or a planar polynomial vector field (under a mild condition) implies existence of a polynomial Lyapunov function that is a sum of squares (sos) and that the negative of its derivative is also a sum of squares. This result is extended to show that such sos-based certificates of stability are guaranteed to exist for all stable switched linear systems. For this class of systems, we further show that if the derivative inequality of the Lyapunov function has an sos certificate, then the Lyapunov function itself is automatically a sum of squares. These converse results establish cases where semidefinite programming is guaranteed to succeed in finding proofs of Lyapunov inequalities. Finally, we demonstrate some merits of replacing the sos requirement on a polynomial Lyapunov function with an sos requirement on its top homogeneous component. In particular, we show that this is a weaker algebraic requirement in addition to being cheaper to impose computationally.

math.OC↗

arXiv2017

Sampling algebraic varieties for sum of squares programs

We study sum of squares (SOS) relaxations to optimize polynomial functions over a set $V\cap R^n$, where $V$ is a complex algebraic variety. We propose a new methodology that, rather than relying on some algebraic description, represents $V$ with a generic set of complex samples. This approach depends only on the geometry of $V$, avoiding representation issues such as multiplicity and choice of generators. It also takes advantage of the coordinate ring structure to reduce the size of the corresponding semidefinite program (SDP). In addition, the input can be given as a straight-line program. Our methods are particularly appealing for varieties that are easy to sample from but for which the defining equations are complicated, such as $SO(n)$, Grassmannians or rank $k$ tensors. For arbitrary varieties we can obtain the required samples by using the tools of numerical algebraic geometry. In this way we connect the areas of SOS optimization and numerical algebraic geometry.

math.OC↗

arXiv2017

How to flatten a soccer ball

This is an experimental case study in real algebraic geometry, aimed at computing the image of a semialgebraic subset of 3-space under a polynomial map into the plane. For general instances, the boundary of the image is given by two highly singular curves. We determine these curves and show how they demarcate the "flattened soccer ball". We explore cylindrical algebraic decompositions, by working through concrete examples. Maps onto convex polygons and connections to convex optimization are also discussed.

math.AG↗

arXiv2016

Chordal networks of polynomial ideals

We introduce a novel representation of structured polynomial ideals, which we refer to as chordal networks. The sparsity structure of a polynomial system is often described by a graph that captures the interactions among the variables. Chordal networks provide a computationally convenient decomposition into simpler (triangular) polynomial sets, while preserving the underlying graphical structure. We show that many interesting families of polynomial ideals admit compact chordal network representations (of size linear in the number of variables), even though the number of components is exponentially large. Chordal networks can be computed for arbitrary polynomial systems using a refinement of the chordal elimination algorithm from [Cifuentes-Parrilo-2016]. Furthermore, they can be effectively used to obtain several properties of the variety, such as its dimension, cardinality, and equidimensional components, as well as an efficient probabilistic test for radical ideal membership. We apply our methods to examples from algebraic statistics and vector addition systems; for these instances, algorithms based on chordal networks outperform existing techniques by orders of magnitude.

cs.SC↗

arXiv2016

Semidefinite Programming Approach to Gaussian Sequential Rate-Distortion Trade-offs

Sequential rate-distortion (SRD) theory provides a framework for studying the fundamental trade-off between data-rate and data-quality in real-time communication systems. In this paper, we consider the SRD problem for multi-dimensional time-varying Gauss-Markov processes under mean-square distortion criteria. We first revisit the sensor-estimator separation principle, which asserts that considered SRD problem is equivalent to a joint sensor and estimator design problem in which data-rate of the sensor output is minimized while the estimator's performance satisfies the distortion criteria. We then show that the optimal joint design can be performed by semidefinite programming. A semidefinite representation of the corresponding SRD function is obtained. Implications of the obtained result in the context of zero-delay source coding theory and applications to networked control theory are also discussed.

math.OC↗