arXiv subjects

Zheng-Hai Huang

Publications and source records attributed to Zheng-Hai Huang.

At least 19 recordsLinked to original sources

arXiv2026

Optimality Conditions for Sparse Bilinear Least Squares Problems

The first-order optimality conditions of sparse bilinear least squares problems are studied. The so-called T-type and N-type stationary points for this problem are characterized in terms of tangent cone and normal cone in Bouligand and Clarke senses, and another stationarity concept called the coordinate-wise minima is introduced and discussed. Moreover, the L-like stationary point for this problem is introduced and analyzed through the newly introduced concept of like-projection, and the M-stationary point is also investigated via a complementarity-type reformulation of the problem. The relationship between these stationary points is discussed as well. It turns out that all stationary points discussed in this work satisfy the necessary optimality conditions for the sparse bilinear least squares problem.

math.OC↗

arXiv2024

Dynamic Thresholding Algorithm with Memory for Linear Inverse Problems

The relaxed optimal $k$-thresholding pursuit (ROTP) is a recent algorithm for linear inverse problems. This algorithm is based on the optimal $k$-thresholding technique which performs vector thresholding and error metric reduction simultaneously. Although ROTP can be used to solve small to medium-sized linear inverse problems, the computational cost of this algorithm is high when solving large-scale problems. By merging the optimal $k$-thresholding technique and iterative method with memory as well as optimization with sparse search directions, we propose the so-called dynamic thresholding algorithm with memory (DTAM), which iteratively and dynamically selects vector bases to construct the problem solution. At every step, the algorithm uses more than one or all iterates generated so far to construct a new search direction, and solves only the small-sized quadratic subproblems at every iteration. Thus the computational complexity of DTAM is remarkably lower than that of ROTP-type methods. It turns out that DTAM can locate the solution of linear inverse problems if the matrix involved satisfies the restricted isometry property. Experiments on synthetic data, audio signal reconstruction and image denoising demonstrate that the proposed algorithm performs comparably to several mainstream thresholding and greedy algorithms, and it works much faster than the ROTP-type algorithms especially when the sparsity level of signal is relatively low.

cs.IT↗

arXiv2020

An existence result for weakly homogeneous variational inequalities

In this paper, what we concern about is the weakly homogeneous variational inequality over a finite dimensional real Hilbert space. We achieve an existence result {under} copositivity of leading term of the involved map, norm-coercivity of the natural map and several additional conditions. These conditions we used are easier to check and cross each other with those utilized in the main result established by Gowda and Sossa (Math Program 177:149-171, 2019). As a corollary, we obtain a result on the solvability of nonlinear equations with weakly homogeneous maps involved. Our result enriches the theory for weakly homogeneous variational inequalities and its subcategory problems in the sense that the main result established by Gowda and Sossa covers a majority of existence results on the subcategory problems of weakly homogeneous variational inequalities. Besides, we compare our {existence} result with the well-known coercivity result obtained for general variational inequalities and a norm-coercivity result obtained for general complementarity problems, respectively.

math.OC↗

arXiv2020

Multi-Tubal Rank of Third Order Tensor and Related Low Rank Tensor Completion Problem

Recently, a tensor factorization based method for a low tubal rank tensor completion problem of a third order tensor was proposed, which performed better than some existing methods. Tubal rank is only defined on one mode of third order tensor without low rank structure in the other two modes. That is, low rank structures on the other two modes are missing. Motivated by this, we first introduce multi-tubal rank, and then establish a relationship between multi-tubal rank and Tucker rank. Based on the multi-tubal rank, we propose a novel low rank tensor completion model. For this model, a tensor factorization based method is applied and the corresponding convergence anlysis is established. In addition, spatio-temporal characteristics are intrinsic features in video and internet traffic tensor data. To get better performance, we make full use of such features and improve the established tensor completion model. Then we apply tensor factorization based method for the improved model. Finally, numerical results are reported on the completion of image, video and internet traffic data to show the efficiency of our proposed methods. From the reported numerical results, we can assert that our methods outperform the existing methods.

math.OC↗

arXiv2020

Eigenvalue-corrected Natural Gradient Based on a New Approximation

Using second-order optimization methods for training deep neural networks (DNNs) has attracted many researchers. A recently proposed method, Eigenvalue-corrected Kronecker Factorization (EKFAC) (George et al., 2018), proposes an interpretation of viewing natural gradient update as a diagonal method, and corrects the inaccurate re-scaling factor in the Kronecker-factored eigenbasis. Gao et al. (2020) considers a new approximation to the natural gradient, which approximates the Fisher information matrix (FIM) to a constant multiplied by the Kronecker product of two matrices and keeps the trace equal before and after the approximation. In this work, we combine the ideas of these two methods and propose Trace-restricted Eigenvalue-corrected Kronecker Factorization (TEKFAC). The proposed method not only corrects the inexact re-scaling factor under the Kronecker-factored eigenbasis, but also considers the new approximation method and the effective damping technique proposed in Gao et al. (2020). We also discuss the differences and relationships among the Kronecker-factored approximations. Empirically, our method outperforms SGD with momentum, Adam, EKFAC and TKFAC on several DNNs.

cs.LG↗

arXiv2020

A Trace-restricted Kronecker-Factored Approximation to Natural Gradient

Second-order optimization methods have the ability to accelerate convergence by modifying the gradient through the curvature matrix. There have been many attempts to use second-order optimization methods for training deep neural networks. Inspired by diagonal approximations and factored approximations such as Kronecker-Factored Approximate Curvature (KFAC), we propose a new approximation to the Fisher information matrix (FIM) called Trace-restricted Kronecker-factored Approximate Curvature (TKFAC) in this work, which can hold the certain trace relationship between the exact and the approximate FIM. In TKFAC, we decompose each block of the approximate FIM as a Kronecker product of two smaller matrices and scaled by a coefficient related to trace. We theoretically analyze TKFAC's approximation error and give an upper bound of it. We also propose a new damping technique for TKFAC on convolutional neural networks to maintain the superiority of second-order optimization methods during training. Experiments show that our method has better performance compared with several state-of-the-art algorithms on some deep network architectures.

cs.LG↗

arXiv2020

Unique solvability of weakly homogeneous generalized variational inequalities

An interesting observation is that most pairs of weakly homogeneous mappings have no strongly monotonic property, which is one of the key conditions to ensure the unique solvability of the generalized variational inequality. This paper focuses on studying the unique solvability of the generalized variational inequality with a pair of weakly homogeneous mappings. By using a weaker condition than the strong monotonicity and some additional conditions, we achieve several results on the unique solvability of the underlying problem. These results are exported by making use of the exceptional family of elements or derived from new obtained Karamardian-type theorems or established under the exceptional regularity condition. They are new even when the problem comes down to its important subclasses studied in recent years.

math.OC↗

arXiv2020

T-positive semidefiniteness of third-order symmetric tensors and T-semidefinite programming

The T-product for third-order tensors has been used extensively in the literature. In this paper, we first introduce the first-order and second-order T-derivatives for the multi-vector real-valued function with the tensor T-product; and inspired by an equivalent characterization of a twice continuously T-differentiable multi-vector real-valued function being convex, we present a definition of the T-positive semidefiniteness of third-order symmetric tensors. After that, we extend many properties of positive semidefinite matrices to the case of third-order symmetric tensors. In particular, analogue to the widely used semidefinite programming (SDP for short), we introduce the semidefinite programming over the third-order symmetric tensor space (T-semidefinite programming or TSDP for short), and provide a way to solve the TSDP problem by converting it into an SDP problem in the complex domain. Furthermore, we give several examples which can be formulated (or relaxed) as TSDP problems, and report preliminary numerical results for two unconstrained polynomial optimization problems. Experiments show that finding the global minimums of polynomials via the TSDP relaxation outperforms the traditional SDP relaxation for the test examples.

math.OC↗

arXiv2018

Stationary probability vectors of higher-order two-dimensional transition probability tensors

In this paper we investigate stationary probability vectors of higher-order two-dimensional symmetric transition probability tensors. We show that there are two special symmetric transition probability tensors of order $m$ dimension 2, which have and only have two stationary probability vectors; and any other symmetric transition probability tensor of order $m$ dimension 2 has a unique stationary probability vector. As a byproduct, we obtain that any symmetric transition probability tensor of order $m$ dimension 2 has a unique positive stationary probability vector; and that any symmetric irreducible transition probability tensor of order $m$ dimension 2 has a unique stationary probability vector.

math.SP↗

arXiv2017

A Theoretical Perspective of Solving Phaseless Compressed Sensing via Its Nonconvex Relaxation

As a natural extension of compressive sensing and the requirement of some practical problems, Phaseless Compressed Sensing (PCS) has been introduced and studied recently. Many theoretical results have been obtained for PCS with the aid of its convex relaxation. Motivated by successful applications of nonconvex relaxed methods for solving compressive sensing, in this paper, we try to investigate PCS via its nonconvex relaxation. Specifically, we relax PCS in the real context by the corresponding $\ell_p$-minimization with $p\in (0,1)$. We show that there exists a constant $p^\ast\in (0,1]$ such that for any fixed $p\in(0, p^\ast)$, every optimal solution to the $\ell_p$-minimization also solves the concerned problem; and derive an expression of such a constant $p^\ast$ by making use of the known data and the sparsity level of the concerned problem. These provide a theoretical basis for solving this class of problems via the corresponding $\ell_p$-minimization.

math.OC↗

arXiv2017

Introduction to Tensor Variational Inequalities

In this paper, we introduce a class of variational inequalities, where the involved function is the sum of an arbitrary given vector and a homogeneous polynomial defined by a tensor; and we call it the tensor variational inequality (TVI). The TVI is a natural extension of the affine variational inequality and the tensor complementarity problem. We show that a class of multi-person noncooperative games can be formulated as a TVI. In particular, we investigate the global uniqueness and solvability of the TVI. To this end, we first introduce two classes of structured tensors and discuss some related properties; and then, we show that the TVI has the property of global uniqueness and solvability under some assumptions, which is different from the existed result for the general variational inequality.

math.OC↗

arXiv2016

Formulating an $n$-person noncooperative game as a tensor complementarity problem

In this paper, we consider a class of $n$-person noncooperative games, where the utility function of every player is given by a homogeneous polynomial defined by the payoff tensor of that player, which is a natural extension of the bimatrix game where the utility function of every player is given by a quadratic form defined by the payoff matrix of that player. We will call such a problem the multilinear game. We reformulate the multilinear game as a tensor complementarity problem, a generalization of the linear complementarity problem; and show that finding a Nash equilibrium point of the multilinear game is equivalent to finding a solution of the resulted tensor complementarity problem. Especially, we present an explicit relationship between the solutions of the multilinear game and the tensor complementarity problem, which builds a bridge between these two classes of problems. We also apply a smoothing-type algorithm to solve the resulted tensor complementarity problem and give some preliminary numerical results for solving the multilinear games.

math.OC↗

arXiv2015

On $Q$-Tensors

One of the central problems in the theory of linear complementarity problems (LCPs) is to study the class of $Q$-matrices since it characterizes the solvability of LCP. Recently, the concept of $Q$-matrix has been extended to the case of tensor, called $Q$-tensor, which characterizes the solvability of the corresponding tensor complementarity problem -- a generalization of LCP; and some basic results related to $Q$-tensors have been obtained in the literature. In this paper, we extend two famous results related to $Q$-matrices to the tensor space, i.e., we show that within the class of strong $P_0$-tensors or nonnegative tensors, four classes of tensors, i.e., $R_0$-tensors, $R$-tensors, $ER$-tensors and $Q$-tensors, are all equivalent. We also construct several examples to show that three famous results related to $Q$-matrices cannot be extended to the tensor space; and one of which gives a negative answer to a question raised recently by Song and Qi.

math.OC↗

arXiv2015

Exceptionally Regular Tensors and Tensor Complementarity Problems

Recently, many structured tensors are defined and their properties are discussed in the literature. In this paper, we introduce a new class of structured tensors, called exceptionally regular tensor, which is relevant to the tensor complementarity problem. We show that this class of tensors is a wide class of tensors which includes many important structured tensors as its special cases. By constructing two examples, we demonstrate that an exceptionally regular tensor can be, but not always, an $R$-tensor. We also show that within the class of the semi-positive tensors, the class of exceptionally regular tensors coincides with the class of $R$-tensors. In addition, we consider the tensor complementarity problem with an exceptionally regular tensor or an $R$-tensor or a $P_0+R_0$-tensor, and show that the solution sets of these classes of tensor complementarity problems are nonempty and compact.

math.OC↗

arXiv2015

Global Uniqueness and Solvability for Tensor Complementarity Problems

Recently, the tensor complementarity problem (TCP for short) has been investigated in the literature. An important question involving the property of global uniqueness and solvability (GUS-property) for a class of TCPs was proposed by Song and Qi in their paper "Properties of Some Classes of Structured Tensors". In the present paper, we give an answer to this question by constructing two counter-examples. We also show that the solution set of this class of TCPs is nonempty and compact. In particular, we introduce a class of related structured tensors, and show that the corresponding TCP has the GUS-property.

math.OC↗

arXiv2014

Minimum $n$-Rank Approximation via Iterative Hard Thresholding

The problem of recovering a low $n$-rank tensor is an extension of sparse recovery problem from the low dimensional space (matrix space) to the high dimensional space (tensor space) and has many applications in computer vision and graphics such as image inpainting and video inpainting. In this paper, we consider a new tensor recovery model, named as minimum $n$-rank approximation (MnRA), and propose an appropriate iterative hard thresholding algorithm with giving the upper bound of the $n$-rank in advance. The convergence analysis of the proposed algorithm is also presented. Particularly, we show that for the noiseless case, the linear convergence with rate $\frac{1}{2}$ can be obtained for the proposed algorithm under proper conditions. Additionally, combining an effective heuristic for determining $n$-rank, we can also apply the proposed algorithm to solve MnRA when $n$-rank is unknown in advance. Some preliminary numerical results on randomly generated and real low $n$-rank tensor completion problems are reported, which show the efficiency of the proposed algorithms.

math.OC↗

arXiv2013

A Splitting Augmented Lagrangian Method for Low Multilinear-Rank Tensor Recovery

This paper studies a recovery task of finding a low multilinear-rank tensor that fulfills some linear constraints in the general settings, which has many applications in computer vision and graphics. This problem is named as the low multilinear-rank tensor recovery problem. The variable splitting technique and convex relaxation technique are used to transform this problem into a tractable constrained optimization problem. Considering the favorable structure of the problem, we develop a splitting augmented Lagrangian method to solve the resulting problem. The proposed algorithm is easily implemented and its convergence can be proved under some conditions. Some preliminary numerical results on randomly generated and real completion problems show that the proposed algorithm is very effective and robust for tackling the low multilinear-rank tensor completion problem.

math.OC↗

arXiv2011

Finding the Spectral Radius of a Nonnegative Tensor

In this paper, we introduce a new class of nonnegative tensors --- strictly nonnegative tensors. A weakly irreducible nonnegative tensor is a strictly nonnegative tensor but not vice versa. We show that the spectral radius of a strictly nonnegative tensor is always positive. We give some sufficient and necessary conditions for the six well-conditional classes of nonnegative tensors, introduced in the literature, and a full relationship picture about strictly nonnegative tensors with these six classes of nonnegative tensors. We then establish global R-linear convergence of a power method for finding the spectral radius of a nonnegative tensor under the condition of weak irreducibility. We show that for a nonnegative tensor T, there always exists a partition of the index set such that every tensor induced by the partition is weakly irreducible; and the spectral radius of T can be obtained from those spectral radii of the induced tensors. In this way, we develop a convergent algorithm for finding the spectral radius of a general nonnegative tensor without any additional assumption. The preliminary numerical results demonstrate the feasibility and effectiveness of the proposed algorithm.

math.NA↗