arXiv subjects

Hoai An Le Thi

Publications and source records attributed to Hoai An Le Thi.

11 recordsLinked to original sources

arXiv2026

Finding directional stationary points of DC programs

We address the problem of computing stationary points for non-smooth, non-convex optimization problems. While this topic is well studied in the smooth setting, fewer algorithmic and theoretical results exist for the non-smooth case. Within Difference-of-Convex functions (DC) programming, the well-known DC Algorithm (DCA) is a standard method for computing critical points, whose definition depends on the chosen DC decomposition. More recently, some works have focused on computing directional stationary points - a stronger notion that does not depend on any particular DC decomposition - for specific non-smooth DC programs, where the second DC component is the pointwise maximum of finitely many smooth convex functions. In this contribution, we propose a new and unified approach for identifying directional stationary points of non-smooth DC programs, where both DC components may be non-smooth. Our framework generalizes both the classical DCA and the above recent approaches. It applies when the second DC component is the pointwise maximum of a continuous family of smooth convex functions, and more generally, to any continuous convex function on the whole space. We also establish strong convergence results. Specifically, when the second DC component is the pointwise maximum of finitely many $C^{1,1}$ - smooth convex functions, we prove under the Łojasiewicz inequality that the entire sequence generated by our algorithm converges. This extends previous results requiring global $C^{1,1}$ - smoothness. Finally, we introduce a randomized (stochastic) variant of our method and prove its almost sure convergence, thereby extending our deterministic results to the randomized setting.

math.OC↗

arXiv2024

On Difference-of-SOS and Difference-of-Convex-SOS Decompositions for Polynomials

In this article, we are interested in developing polynomial decomposition techniques based on sums-of-squares (SOS), namely the difference-of-sums-of-squares (D-SOS) and the difference-of-convex-sums-of-squares (DC-SOS). In particular, the DC-SOS decomposition is very useful for difference-of-convex (DC) programming formulation of polynomial optimization problems. First, we introduce the cone of convex-sums-of-squares (CSOS) polynomials and discuss its relationship to the sums-of-squares (SOS) polynomials, the non-negative polynomials and the SOS-convex polynomials. Then, we propose the set of D-SOS and DC-SOS polynomials, and prove that any polynomial can be formulated as D-SOS and DC-SOS. The problem of finding D-SOS and DC-SOS decompositions can be formulated as a semi-definite program and solved for any desired precision in polynomial time using interior point methods. Some algebraic properties of CSOS, D-SOS and DC-SOS are established. Second, we focus on establishing several practical algorithms for exact D-SOS and DC-SOS polynomial decompositions without solving any SDP. The numerical performance of the proposed D-SOS and DC-SOS decomposition algorithms and their parallel versions, tested on a dataset of 1750 randomly generated polynomials, is reported.

math.OC↗

arXiv2023

Solving the problem of batch deletion and insertion members in the Logical Key Hierarchy structure by a DC Programming approach

In secure group communications, users of a group share a common group key to prevent eavesdropping and protect the exchange content. A key server distributes the group key as well as performs group rekeying whenever the membership changes dynamically. Instead of rekeying after each join or leave request, we use batch rekeying to alleviate the out-of-sync problem and improve the efficiency. In this paper, we propose an optimization approach to the problem of updating group key in the Logical Key Hierarchy (LKH) structure with batch rekeying. A subtree of new nodes can be appended below a leaf node or is replaced the position of leaving node on the binary key tree. The latter has a lower updating key cost than the former since when a member leaves, all the keys on the path from the root to the deletion node must be updated anyway. We aim to minimize the total rekeying cost, which is the cost of deletion and insertion members while keeping the tree as balanced as possible. The mentioned problem is represented by a unified (deterministic) optimization model whose objective function contains discontinuous step functions with binary variables. Thanks to an exact penalty technique, the problem is equivalently reformulated as a standard DC (Difference of Convex functions) program that can be solved efficiently by DCA (DC algorithm). Numerical experiments have been studied intensively to justify the merit of our proposed approach as well as the corresponding DCA.

math.OC↗

arXiv2022

High-order Moment Portfolio Optimization via An Accelerated Difference-of-Convex Programming Approach and Sums-of-Squares

The Mean-Variance-Skewness-Kurtosis (MVSK) portfolio optimization model is a quartic nonconvex polynomial minimization problem over a polytope, which can be formulated as a Difference-of-Convex (DC) program. In this manuscript, we investigate four DC programming approaches for solving the MVSK model. First, two DC formulations based on the projective DC decomposition and the Difference-of-Convex-Sums-of-Squares (DC-SOS) decomposition are established, where the second one is novel. Then, DCA is applied to solve these DC formulations. The convergence analysis of DCA for the MVSK model is established. Second, we propose an accelerated DCA (Boosted-DCA) for solving a general convex constrained DC program involving both smooth and nonsmooth functions. The acceleration is realized by an inexact line search of the Armijo-type along the DC descent direction generated by two consecutive iterates of DCA. The convergence analysis of the Boosted-DCA is established. Numerical simulations of the proposed four DC algorithms on both synthetic and real portfolio datasets are reported. Comparisons with KNITRO, FILTERSD, IPOPT and MATLAB fmincon optimization solvers demonstrate good performance of our methods. Particularly, two DC algorithms with DC-SOS decomposition require less number of iterations, which demonstrates that DC-SOS decomposition can provide better convex over-approximations for polynomials. Moreover, the accelerated versions indeed reduce the number of iterations and achieve the best numerical results.

math.OC↗

arXiv2021

Online Stochastic DCA with applications to Principal Component Analysis

Stochastic algorithms are well-known for their performance in the era of big data. In convex optimization, stochastic algorithms have been studied in depth and breadth. However, the current body of research on stochastic algorithms for nonsmooth, nonconvex optimization is relatively limited. In this paper, we propose new stochastic algorithms based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) - the backbone of nonconvex, nonsmooth optimization. Since most real-world nonconvex programs fall into the framework of DC programming, our proposed methods can be employed in various situations, in which they confront stochastic nature and nonconvexity simultaneously. The convergence analysis of the proposed algorithms is studied intensively with the help of tools from modern convex analysis and martingale theory. Finally, we study several aspects of the proposed algorithms on an important problem in machine learning: the expected problem in Principal Component Analysis.

math.OC↗

arXiv2021

DCA based Algorithm with Extrapolation for Nonconvex Nonsmooth Optimization

In this paper, we focus on the problem of minimizing the sum of a nonconvex differentiable function and a DC (Difference of Convex functions) function, where the differentiable function is not restricted to the global Lipschitz gradient continuity assumption. This problem covers a broad range of applications in machine learning and statistics such as compressed sensing, signal recovery, sparse dictionary learning, and matrix factorization, etc. We take inspiration from the Nesterov's acceleration technique and the DC algorithm to develop a novel algorithm for the considered problem. Analyzing the convergence, we study the subsequential convergence of our algorithm to a critical point. Furthermore, we justify the global convergence of the whole sequence generated by our algorithm to a critical point and establish its convergence rate under the Kurdyka-Lojasiewicz condition. Numerical experiments on the nonnegative matrix completion problem are performed to demonstrate the efficiency of our algorithm and its superiority over well-known methods.

math.OC↗

arXiv2019

Stochastic DCA for minimizing a large sum of DC functions with application to Multi-class Logistic Regression

We consider the large sum of DC (Difference of Convex) functions minimization problem which appear in several different areas, especially in stochastic optimization and machine learning. Two DCA (DC Algorithm) based algorithms are proposed: stochastic DCA and inexact stochastic DCA. We prove that the convergence of both algorithms to a critical point is guaranteed with probability one. Furthermore, we develop our stochastic DCA for solving an important problem in multi-task learning, namely group variables selection in multi class logistic regression. The corresponding stochastic DCA is very inexpensive, all computations are explicit. Numerical experiments on several benchmark datasets and synthetic datasets illustrate the efficiency of our algorithms and their superiority over existing methods, with respect to classification accuracy, sparsity of solution as well as running time.

math.OC↗

arXiv2018

A DCA-Like Algorithm and its Accelerated Version with Application in Data Visualization

In this paper, we present two variants of DCA (Different of Convex functions Algorithm) to solve the constrained sum of differentiable function and composite functions minimization problem, with the aim of increasing the convergence speed of DCA. In the first variant, DCA-Like, we introduce a new technique to iteratively modify the decomposition of the objective function. This successive decomposition could lead to a better majorization and consequently a better convergence speed than the basic DCA. We then incorporate the Nesterov's acceleration technique into DCA-Like to give rise to the second variant, named Accelerated DCA-Like. The convergence properties and the convergence rate under Kudyka-Lojasiewicz assumption of both variants are rigorously studied. As an application, we investigate our algorithms for the t-distributed stochastic neighbor embedding. Numerical experiments on several benchmark datasets illustrate the efficiency of our algorithms.

math.OC↗

arXiv2014

DC approximation approaches for sparse optimization

Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC approximation of the zero-norm including all standard sparse inducing penalty functions, we studied the consistency between global minimums (resp. local minimums) of approximate and original problems. We showed that, in several cases, some global minimizers (resp. local minimizers) of the approximate problem are also those of the original problem. Using exact penalty techniques in DC programming, we proved stronger results for some particular approximations, namely, the approximate problem, with suitable parameters, is equivalent to the original problem. The efficiency of several sparse inducing penalty functions have been fully analyzed. Four DCA (DC Algorithm) schemes were developed that cover all standard algorithms in nonconvex sparse approximation approaches as special versions. They can be viewed as, an $\ell _{1}$-perturbed algorithm / reweighted-$\ell _{1}$ algorithm / reweighted-$\ell _{1}$ algorithm. We offer a unifying nonconvex approximation approach, with solid theoretical tools as well as efficient algorithms based on DC programming and DCA, to tackle the zero-norm and sparse optimization. As an application, we implemented our methods for the feature selection in SVM (Support Vector Machine) problem and performed empirical comparative numerical experiments on the proposed algorithms with various approximation functions.

math.NA↗

arXiv2014

A DC programming approach for constrained two-dimensional non-guillotine cutting problem

We investigate a new application of Difference of Convex functions programming and DCA in solving the constrained two-dimensional non-guillotine cutting problem. This problem consists of cutting a number of rectangular pieces from a large rectangular object. The cuts are done under some constraints and the objective is to maximize the total value of the pieces cut. We reformulate this problem as a DC program and solve it by DCA. The performance of the approach is compared with the standard solver CPLEX.

cs.CE↗

arXiv2014

Portfolio Selection Under Buy-In Threshold Constraints Using DC Programming and DCA

In matter of Portfolio selection, we consider a generalization of the Markowitz Mean-Variance model which includes buy-in threshold constraints. These constraints limit the amount of capital to be invested in each asset and prevent very small investments in any asset. The new model can be converted into a NP-hard mixed integer quadratic programming problem. The purpose of this paper is to investigate a continuous approach based on DC programming and DCA for solving this new model. DCA is a local continuous approach to solve a wide variety of nonconvex programs for which it provided quite often a global solution and proved to be more robust and efficient than standard methods. Preliminary comparative results of DCA and a classical Branch-and-Bound algorithm will be presented. These results show that DCA is an efficient and promising approach for the considered portfolio selection problem.

cs.CE↗