arXiv subjects

Guoyin Li

Publications and source records attributed to Guoyin Li.

26 records · Page 2Linked to original sources

arXiv2015

Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems

We adapt the Douglas-Rachford (DR) splitting method to solve nonconvex feasibility problems by studying this method for a class of nonconvex optimization problem. While the convergence properties of the method for convex problems have been well studied, far less is known in the nonconvex setting. In this paper, for the direct adaptation of the method to minimize the sum of a proper closed function $g$ and a smooth function $f$ with a Lipschitz continuous gradient, we show that if the step-size parameter is smaller than a computable threshold and the sequence generated has a cluster point, then it gives a stationary point of the optimization problem. Convergence of the whole sequence and a local convergence rate are also established under the additional assumption that $f$ and $g$ are semi-algebraic. We also give simple sufficient conditions guaranteeing the boundedness of the sequence generated. We then apply our nonconvex DR splitting method to finding a point in the intersection of a closed convex set $C$ and a general closed set $D$ by minimizing the squared distance to $C$ subject to $D$. We show that if either set is bounded and the step-size parameter is smaller than a computable threshold, then the sequence generated from the DR splitting method is actually bounded. Consequently, the sequence generated will have cluster points that are stationary for an optimization problem, and the whole sequence is convergent under an additional assumption that $C$ and $D$ are semi-algebraic. We achieve these results based on a new merit function constructed particularly for the DR splitting method. Our preliminary numerical results indicate that our DR splitting method usually outperforms the alternating projection method in finding a sparse solution of a linear system, in terms of both the solution quality and the number of iterations taken.

math.OC↗

arXiv2015

Global convergence of splitting methods for nonconvex composite optimization

We consider the problem of minimizing the sum of a smooth function $h$ with a bounded Hessian, and a nonsmooth function. We assume that the latter function is a composition of a proper closed function $P$ and a surjective linear map $\cal M$, with the proximal mappings of $τP$, $τ> 0$, simple to compute. This problem is nonconvex in general and encompasses many important applications in engineering and machine learning. In this paper, we examined two types of splitting methods for solving this nonconvex optimization problem: alternating direction method of multipliers and proximal gradient algorithm. For the direct adaptation of the alternating direction method of multipliers, we show that, if the penalty parameter is chosen sufficiently large and the sequence generated has a cluster point, then it gives a stationary point of the nonconvex problem. We also establish convergence of the whole sequence under an additional assumption that the functions $h$ and $P$ are semi-algebraic. Furthermore, we give simple sufficient conditions to guarantee boundedness of the sequence generated. These conditions can be satisfied for a wide range of applications including the least squares problem with the $\ell_{1/2}$ regularization. Finally, when $\cal M$ is the identity so that the proximal gradient algorithm can be efficiently applied, we show that any cluster point is stationary under a slightly more flexible constant step-size rule than what is known in the literature for a nonconvex $h$.

math.OC↗

arXiv2015

SOS Tensor Decomposition: Theory and Applications

In this paper, we examine structured tensors which have sum-of-squares (SOS) tensor decomposition, and study the SOS-rank of SOS tensor decomposition. We first show that several classes of even order symmetric structured tensors available in the literature have SOS tensor decomposition. These include positive Cauchy tensors, weakly diagonally dominated tensors, $B_0$-tensors, double $B$-tensors, quasi-double $B_0$-tensors, $MB_0$-tensors, $H$-tensors, absolute tensors of positive semi-definite $Z$-tensors and extended $Z$-tensors. We also examine the SOS-rank of SOS tensor decomposition and the SOS-width for SOS tensor cones. The SOS-rank provides the minimal number of squares in the SOS tensor decomposition, and, for a given SOS tensor cone, its SOS-width is the maximum possible SOS-rank for all the tensors in this cone. We first deduce an upper bound for general tensors that have SOS decomposition and the SOS-width for general SOS tensor cone using the known results in the literature of polynomial theory. Then, we provide an explicit sharper estimate for the SOS-rank of SOS tensor decomposition with bounded exponent and identify the SOS-width for the tensor cone consisting of all tensors with bounded exponent that have SOS decompositions. Finally, as applications, we show how the SOS tensor decomposition can be used to compute the minimum $H$-eigenvalue of an even order symmetric extended $Z$-tensor and test the positive definiteness of an associated multivariate form. Numerical experiments are also provided to show the efficiency of the proposed numerical methods ranging from small size to large size numerical examples.

math.SP↗

arXiv2015

Further Results on Cauchy Tensors and Hankel Tensors

In this article, we present various new results on Cauchy tensors and Hankel tensors. { We first introduce the concept of generalized Cauchy tensors which extends Cauchy tensors in the current literature, and provide several conditions characterizing positive semi-definiteness of generalized Cauchy tensors with nonzero entries.} As a consequence, we show that Cauchy tensors are positive semi-definite if and only if they are SOS (Sum-of-squares) tensors.} Furthermore, we prove that all positive semi-definite Cauchy tensors are completely positive tensors, which means every positive semi-definite Cauchy tensor can be decomposed { as} the sum of nonnegative rank-1 tensors. We also establish that all the H-eigenvalues of nonnegative Cauchy tensors are nonnegative. Secondly, we present new mathematical properties of Hankel tensors. { We prove that an even order Hankel tensor is Vandermonde positive semi-definite if and only if its associated plane tensor is positive semi-definite. We also show that, if the Vandermonde rank of a Hankel tensor $\mathcal{A}$ is less than the dimension of the underlying space, then positive semi-definiteness of $\mathcal{A}$ is equivalent to the fact that $\mathcal{A}$ is a complete Hankel tensor, and so, is further equivalent to the SOS property of $\mathcal{A}$. Lastly, we introduce a new structured tensor called Cauchy-Hankel tensors, which is a special case of Cauchy tensors and Hankel tensors simultaneously.} Sufficient and necessary conditions are established for an even order Cauchy-Hankel tensor to be positive definite. Final remarks are listed at the end of the paper.

math.SP↗

arXiv2014

Positive Semi-Definiteness of Generalized Anti-Circulant Tensors

Anti-circulant tensors have applications in exponential data fitting. They are special Hankel tensors. In this paper, we extend the definition of anti-circulant tensors to generalized anti-circulant tensors by introducing a circulant index $r$ such that the entries of the generating vector of a Hankel tensor are circulant with module $r$. In the special case when $r =n$, where $n$ is the dimension of the Hankel tensor, the generalized anticirculant tensor reduces to the anti-circulant tensor. Hence, generalized anti-circulant tensors are still special Hankel tensors. For the cases that $GCD(m, r) =1$, $GCD(m, r) = 2$ and some other cases, including the matrix case that $m=2$, we give necessary and sufficient conditions for positive semi-definiteness of even order generalized anti-circulant tensors, and show that in these cases, they are SOS tensors. This shows that, in these cases, there are no PNS (positive semidefinite tensors which are not sum of squares) Hankel tensors.

math.CO↗

arXiv2014

SOS-Hankel Tensors: Theory and Application

Hankel tensors arise from signal processing and some other applications. SOS (sum-of-squares) tensors are positive semi-definite symmetric tensors, but not vice versa. The problem for determining an even order symmetric tensor is an SOS tensor or not is equivalent to solving a semi-infinite linear programming problem, which can be done in polynomial time. On the other hand, the problem for determining an even order symmetric tensor is positive semi-definite or not is NP-hard. In this paper, we study SOS-Hankel tensors. Currently, there are two known positive semi-definite Hankel tensor classes: even order complete Hankel tensors and even order strong Hankel tensors. We show complete Hankel tensors are strong Hankel tensors, and even order strong Hankel tensors are SOS-Hankel tensors. We give several examples of positive semi-definite Hankel tensors, which are not strong Hankel tensors. However, all of them are still SOS-Hankel tensors. Does there exist a positive semi-definite non-SOS-Hankel tensor? The answer to this question remains open. If the answer to this question is no, then the problem for determining an even order Hankel tensor is positive semi-definite or not is solvable in polynomial-time. An application of SOS-Hankel tensors to the positive semi-definite tensor completion problem is discussed. We present an ADMM algorithm for solving this problem. Some preliminary numerical results on this algorithm are reported.

math.SP↗

arXiv2014

A Tensor Analogy of Yuan's Theorem of the Alternative and Polynomial Optimization with Sign structure

Yuan's theorem of the alternative is an important theoretical tool in optimization, which provides a checkable certificate for the infeasibility of a strict inequality system involving two homogeneous quadratic functions. In this paper, we provide a tractable extension of Yuan's theorem of the alternative to the symmetric tensor setting. As an application, we establish that the optimal value of a class of nonconvex polynomial optimization problems with suitable sign structure (or more explicitly, with essentially non-positive coefficients) can be computed by a related convex conic programming problem, and the optimal solution of these nonconvex polynomial optimization problems can be recovered from the corresponding solution of the convex conic programming problem. Moreover, we obtain that this class of nonconvex polynomial optimization problems enjoy exact sum-of-squares relaxation, and so, can be solved via a single semidefinite programming problem.

math.OC↗

arXiv2013

Analysis of the convergence rate for the cyclic projection algorithm applied to basic semi-algebraic convex sets

In this paper, we study the rate of convergence of the cyclic projection algorithm applied to finitely many basic semi-algebraic convex sets. We establish an explicit convergence rate estimate which relies on the maximum degree of the polynomials that generate the basic semi-algebraic convex sets and the dimension of the underlying space. We achieve our results by exploiting the algebraic structure of the basic semi-algebraic convex sets.

math.FA↗