arXiv subjects

Yi-Shuai Niu

Publications and source records attributed to Yi-Shuai Niu.

29 records · Page 2Linked to original sources

arXiv2023

An Accelerated DC Programming Approach with Exact Line Search for The Symmetric Eigenvalue Complementarity Problem

In this paper, we are interested in developing an accelerated Difference-of-Convex (DC) programming algorithm based on the exact line search for efficiently solving the Symmetric Eigenvalue Complementarity Problem (SEiCP) and Symmetric Quadratic Eigenvalue Complementarity Problem (SQEiCP). We first proved that any SEiCP is equivalent to SEiCP with symmetric positive definite matrices only. Then, we established DC programming formulations for two equivalent formulations of SEiCP (namely, the logarithmic formulation and the quadratic formulation), and proposed the accelerated DC algorithm (BDCA) by combining the classical DCA with inexpensive exact line search by finding real roots of a binomial for acceleration. We demonstrated the equivalence between SQEiCP and SEiCP, and extended BDCA to SQEiCP. Numerical simulations of the proposed BDCA and DCA against KNITRO, FILTERED and MATLAB FMINCON for SEiCP and SQEiCP on both synthetic datasets and Matrix Market NEP Repository are reported. BDCA demonstrated dramatic acceleration to the convergence of DCA to get better numerical solutions, and outperformed KNITRO, FILTERED, and FMINCON solvers in terms of the average CPU time and average solution precision, especially for large-scale cases.

math.OC↗

arXiv2022

High-order Moment Portfolio Optimization via An Accelerated Difference-of-Convex Programming Approach and Sums-of-Squares

The Mean-Variance-Skewness-Kurtosis (MVSK) portfolio optimization model is a quartic nonconvex polynomial minimization problem over a polytope, which can be formulated as a Difference-of-Convex (DC) program. In this manuscript, we investigate four DC programming approaches for solving the MVSK model. First, two DC formulations based on the projective DC decomposition and the Difference-of-Convex-Sums-of-Squares (DC-SOS) decomposition are established, where the second one is novel. Then, DCA is applied to solve these DC formulations. The convergence analysis of DCA for the MVSK model is established. Second, we propose an accelerated DCA (Boosted-DCA) for solving a general convex constrained DC program involving both smooth and nonsmooth functions. The acceleration is realized by an inexact line search of the Armijo-type along the DC descent direction generated by two consecutive iterates of DCA. The convergence analysis of the Boosted-DCA is established. Numerical simulations of the proposed four DC algorithms on both synthetic and real portfolio datasets are reported. Comparisons with KNITRO, FILTERSD, IPOPT and MATLAB fmincon optimization solvers demonstrate good performance of our methods. Particularly, two DC algorithms with DC-SOS decomposition require less number of iterations, which demonstrates that DC-SOS decomposition can provide better convex over-approximations for polynomials. Moreover, the accelerated versions indeed reduce the number of iterations and achieve the best numerical results.

math.OC↗

arXiv2022

A Variable Metric and Nesterov Extrapolated Proximal DCA with Backtracking for A Composite DC Program

In this paper, we consider a composite difference-of-convex (DC) program, whose objective function is the sum of a smooth convex function with Lipschitz continuous gradient, a proper closed and convex function, and a continuous concave function. This problem has many applications in machine learning and data science. The proximal DCA (pDCA), a special case of the classical DCA, as well as two Nesterov-type extrapolated DCA -- ADCA (Phan et al. IJCAI:1369--1375, 2018) and pDCAe (Wen et al. Comput Optim Appl 69:297--324, 2018) -- can solve this problem. The algorithmic step-sizes of pDCA, pDCAe, and ADCA are fixed and determined by estimating a prior the smoothness parameter of the loss function. However, such an estimate may be hard to obtain or poor in some real-world applications. Motivated by this difficulty, we propose a variable metric and Nesterov extrapolated proximal DCA with backtracking (SPDCAe), which combines the backtracking line search procedure (not necessarily monotone) and the Nesterov's extrapolation for potential acceleration; moreover, the variable metric method is incorporated for better local approximation. Numerical simulations on sparse binary logistic regression and compressed sensing with Poisson noise demonstrate the effectiveness of our proposed method.

math.OC↗

arXiv2022

Discrete Dynamical System Approaches for Boolean Polynomial Optimization

In this article, we discuss the numerical solution of Boolean polynomial programs by algorithms borrowing from numerical methods for differential equations, namely the Houbolt scheme, the Lie scheme, and a Runge-Kutta scheme. We first introduce a quartic penalty functional (of Ginzburg-Landau type) to approximate the Boolean program by a continuous one and prove some convergence results as the penalty parameter $\varepsilon$ converges to $0$. We prove also that, under reasonable assumptions, the distance between local minimizers of the penalized problem and the set $\{\pm1\}^n$ is of order $O(\sqrt{n}\varepsilon)$. Next, we introduce algorithms for the numerical solution of the penalized problem, these algorithms relying on the Houbolt, Lie and Runge-Kutta schemes, classical methods for the numerical solution of ordinary or partial differential equations. We performed numerical experiments to investigate the impact of various parameters on the convergence of the algorithms. We have tested our ODE approaches and compared with the classical nonlinear optimization solver IPOPT and a quadratic binary formulation approach (QB-G) as well as an exhaustive method using parallel computing techniques. The numerical results on various datasets (including small and large-scale randomly generated synthetic datasets of general Boolean polynomial optimization problems, and a large-scale heterogeneous MQLib benchmark dataset of Max-Cut and Quadratic Unconstrained Binary Optimization (QUBO) problems) show good performances for our ODE approaches. As a result, our ODE algorithms often converge faster than the other compared methods to better integer solutions of the Boolean program.

math.OC↗

arXiv2022

A Refined Inertial DC Algorithm for DC Programming

In this paper we consider the difference-of-convex (DC) programming problems, whose objective function is the difference of two convex functions. The classical DC Algorithm (DCA) is well-known for solving this kind of problems, which generally returns a critical point. Recently, an inertial DC algorithm (InDCA) equipped with heavy-ball inertial-force procedure was proposed in de Oliveira et al. (Set-Valued and Variational Analysis 27(4):895--919, 2019), which potentially helps to improve both the convergence speed and the solution quality. Based on InDCA, we propose a refined inertial DC algorithm (RInDCA) equipped with enlarged inertial step-size compared with InDCA. Empirically, larger step-size accelerates the convergence. We demonstrate the subsequential convergence of our refined version to a critical point. In addition, by assuming the Kurdyka-Łojasiewicz (KL) property of the objective function, we establish the sequential convergence of RInDCA. Numerical simulations on checking copositivity of matrices and image denoising problem show the benefit of larger step-size.

math.OC↗

arXiv2021

Power-product matrix: nonsingularity, sparsity and determinant

We prove the nonsingularity of a class of integer matrices V(n,d), namely power-product matrix, for positive integers n and d. Some technical proofs are mainly based on linear algebra and enumerative combinatorics, particularly the generating function method and involution principle. We will show that the matrix V(n,d) is nonsingular for all positive integers n and d, and often with sparse structure. Special attention is given to the computation of the determinant V(2,d) with positive integer d.

math.CO↗

arXiv2021

Spatio-Temporal Neural Network for Fitting and Forecasting COVID-19

We established a Spatio-Temporal Neural Network, namely STNN, to forecast the spread of the coronavirus COVID-19 outbreak worldwide in 2020. The basic structure of STNN is similar to the Recurrent Neural Network (RNN) incorporating with not only temporal data but also spatial features. Two improved STNN architectures, namely the STNN with Augmented Spatial States (STNN-A) and the STNN with Input Gate (STNN-I), are proposed, which ensure more predictability and flexibility. STNN and its variants can be trained using Stochastic Gradient Descent (SGD) algorithm and its improved variants (e.g., Adam, AdaGrad and RMSProp). Our STNN models are compared with several classical epidemic prediction models, including the fully-connected neural network (BPNN), and the recurrent neural network (RNN), the classical curve fitting models, as well as the SEIR dynamical system model. Numerical simulations demonstrate that STNN models outperform many others by providing more accurate fitting and prediction, and by handling both spatial and temporal data.

cs.LG↗

arXiv2021

A Difference-of-Convex Cutting Plane Algorithm for Mixed-Binary Linear Program

In this paper, we propose a cutting plane algorithm based on DC (Difference-of-Convex) programming and DC cut for globally solving Mixed-Binary Linear Program (MBLP). We first use a classical DC programming formulation via the exact penalization to formulate MBLP as a DC program, which can be solved by DCA algorithm. Then, we focus on the construction of DC cuts, which serves either as a local cut (namely type-I DC cut) at feasible local minimizer of MBLP, or as a global cut (namely type-II DC cut) at infeasible local minimizer of MBLP if some particular assumptions are verified. Otherwise, the constructibility of DC cut is still unclear, and we propose to use classical global cuts (such as the Lift-and-Project cut) instead. Combining DC cut and classical global cuts, a cutting plane algorithm, namely DCCUT, is established for globally solving MBLP. The convergence theorem of DCCUT is proved. Restarting DCA in DCCUT helps to quickly update the upper bound solution and to introduce more DC cuts for lower bound improvement. A variant of DCCUT by introducing more classical global cuts in each iteration is proposed, and parallel versions of DCCUT and its variant are also designed which use the power of multiple processors for better performance. Numerical simulations of DCCUT type algorithms comparing with the classical cutting plane algorithm using Lift-and-Project cuts are reported. Tests on some specific samples and the MIPLIB 2017 benchmark dataset demonstrate the benefits of DC cut and good performance of DCCUT algorithms.

math.OC↗

arXiv2021

A Difference-of-Convex Programming Approach With Parallel Branch-and-Bound For Sentence Compression Via A Hybrid Extractive Model

Sentence compression is an important problem in natural language processing with wide applications in text summarization, search engine and human-AI interaction system etc. In this paper, we design a hybrid extractive sentence compression model combining a probability language model and a parse tree language model for compressing sentences by guaranteeing the syntax correctness of the compression results. Our compression model is formulated as an integer linear programming problem, which can be rewritten as a Difference-of-Convex (DC) programming problem based on the exact penalty technique. We use a well-known efficient DC algorithm -- DCA to handle the penalized problem for local optimal solutions. Then a hybrid global optimization algorithm combining DCA with a parallel branch-and-bound framework, namely PDCABB, is used for finding global optimal solutions. Numerical results demonstrate that our sentence compression model can provide excellent compression results evaluated by F-score, and indicate that PDCABB is a promising algorithm for solving our sentence compression model.

cs.AI↗

arXiv2019

Sentence Compression via DC Programming Approach

Sentence compression is an important problem in natural language processing. In this paper, we firstly establish a new sentence compression model based on the probability model and the parse tree model. Our sentence compression model is equivalent to an integer linear program (ILP) which can both guarantee the syntax correctness of the compression and save the main meaning. We propose using a DC (Difference of convex) programming approach (DCA) for finding local optimal solution of our model. Combing DCA with a parallel-branch-and-bound framework, we can find global optimal solution. Numerical results demonstrate the good quality of our sentence compression model and the excellent performance of our proposed solution algorithm.

cs.CL↗

arXiv2019

Improved DC Programming Approaches for Solving the Quadratic Eigenvalue Complementarity Problem

In this paper, we discuss the solution of a Quadratic Eigenvalue Complementarity Problem (QEiCP) by using Difference of Convex (DC) programming approaches. We first show that QEiCP can be represented as dc programming problem. Then we investigate different dc programming formulations of QEiCP and discuss their dc algorithms based on a well-known method -- DCA. A new local dc decomposition is proposed which aims at constructing a better dc decomposition regarding to the specific feature of the target problem in some neighborhoods of the iterates. This new procedure yields faster convergence and better precision of the computed solution. Numerical results illustrate the efficiency of the new dc algorithms in practice.

math.OC↗