arXiv subjects

Penghe Zhang

Publications and source records attributed to Penghe Zhang.

7 recordsLinked to original sources

arXiv2026

A Proximal Point Semismooth Newton Algorithm for Structural Composite Cardinality Optimization

Many image-processing problems can be formulated as composite cardinality optimization (CCOP), whose objective is the sum of two convex terms and a cardinality function composed with a linear operator. The composite cardinality term creates major algorithmic challenges: the separability of the cardinality function is lost and convergence analysis often requires surjectivity-type assumptions on the linear operators. To overcome these challenges, we consider the stationary dual formulation of CCOP, which has more favorable structure consisting of two convex terms and a separable cardinality function. In this paper, we develop an efficient proximal point algorithm (PPA) to solve the stationary dual problem. The efficiency of our PPA stems from two aspects. Firstly, the key step of its subproblem solver minimizes a smooth convex function over a low-dimensional subspace by the classic semismooth Newton algorithm (SNA), which has global convergence and local superlinear rate under suitable conditions. Secondly, implementable inexact criteria are proposed for early termination of the SNA subroutine. These components form the basic framework of our inexact PPA. Under suitable conditions, it enjoys global convergence and local linear convergence rate. In particular, we provide examples in which the convergence assumptions are automatically satisfied. Finally, the SNA subroutine is incorporated into our inexact PPA to solve jump-sparse signal recovery and computed tomography (CT) image restoration. Numerical results demonstrate the time efficiency and solution accuracy of our proposed method.

math.OC↗

arXiv2026

On the Stationary Duality of Structural Composite Cardinality Optimization

Simple cardinality refers to counting nonzero elements of an independent variable satisfying certain properties. Composite cardinality is a simple counting process composited with an affine mapping, and is therefore more complicated than the simple cardinality. We study the composite cardinality optimization problem (CCOP) with structures covering a wide range of applications. Through the use of the stationary duality, we reduce the composite counting to simple counting, and thereby obtain a dual formulation of CCOP. For both primal and dual problems, we investigate the sufficient conditions for the existence of global solutions. Those conditions are validated on representative examples from existing literature. We then show that local solutions of the primal and dual problems are equivalent to their stationary points. This result further helps us establish a one-to-one correspondences between primal and dual local solutions. We also demonstrate that the correspondence holds for a pair of global solutions to the primal and dual problems, provided that the dual weighted parameters are appropriately selected. The reported theoretical results lay foundation for developing numerical algorithms for CCOP in future.

math.OC↗

arXiv2026

Local Duality for Sparse Support Vector Machines

Due to the rise of cardinality minimization in optimization, sparse support vector machines (SSVMs) have attracted much attention lately and show certain empirical advantages over convex SVMs. A common way to derive an SSVM is to add a cardinality function such as $\ell_0$-norm to the dual problem of a convex SVM. However, this process lacks theoretical justification. This paper fills the gap by developing a local duality theory for such an SSVM formulation and exploring its relationship with the hinge-loss SVM (hSVM) and the ramp-loss SVM (rSVM). In particular, we prove that the derived SSVM is exactly the dual problem of the 0/1-loss SVM, and the linear representer theorem holds for their local solutions. The local solution of SSVM also provides guidelines on selecting hyperparameters of hSVM and rSVM. {Under specific conditions, we show that a sequence of global solutions of hSVM converges to a local solution of 0/1-loss SVM. Moreover, a local minimizer of 0/1-loss SVM is a local minimizer of rSVM.} This explains why a local solution induced by SSVM outperforms hSVM and rSVM in the prior empirical study. We further conduct numerical tests on real datasets and demonstrate potential advantages of SSVM by working with locally nice solutions proposed in this paper.

cs.LG↗

arXiv2025

Composite Optimization with Indicator Functions: Stationary Duality and a Semismooth Newton Method

Indicator functions of taking values of zero or one are essential to numerous applications in machine learning and statistics. The corresponding primal optimization model has been researched in several recent works. However, its dual problem is a more challenging topic that has not been well addressed. One possible reason is that the Fenchel conjugate of any indicator function is finite only at the origin. This work aims to explore the dual optimization for the sum of a strongly convex function and a composite term with indicator functions on positive intervals. For the first time, a dual problem is constructed by extending the classic conjugate subgradient property to the indicator function. This extension further helps us establish the equivalence between the primal and dual solutions. The dual problem turns out to be a sparse optimization with a $\ell_0$ regularizer and a nonnegative constraint. The proximal operator of the sparse regularizer is used to identify a dual subspace to implement gradient and/or semismooth Newton iteration with low computational complexity. This gives rise to a dual Newton-type method with both global convergence and local superlinear (or quadratic) convergence rate under mild conditions. Finally, when applied to AUC maximization and sparse multi-label classification, our dual Newton method demonstrates satisfactory performance on computational speed and accuracy.

math.OC↗

arXiv2023

Sparse SVM with Hard-Margin Loss: a Newton-Augmented Lagrangian Method in Reduced Dimensions

The hard margin loss function has been at the core of the support vector machine (SVM) research from the very beginning due to its generalization capability.On the other hand, the cardinality constraint has been widely used for feature selection, leading to sparse solutions. This paper studies the sparse SVM with the hard-margin loss (SSVM-HM) that integrates the virtues of both worlds. However, SSVM-HM is one of the most challenging models to solve. In this paper, we cast the problem as a composite optimization with the cardinality constraint. We characterize its local minimizers in terms of {\rm P}-stationarity that well captures the combinatorial structure of the problem. We then propose an inexact proximal augmented Lagrangian method (iPAL). The different parts of the inexactness measurements from the {\rm P}-stationarity are controlled at different scales in a way that the generated sequence converges both globally and at a linear rate. This matches the best convergence theory for composite optimization. To make iPAL practically efficient, we propose a gradient-Newton method in a subspace for the iPAL subproblem. This is accomplished by detecting active samples and features with the help of the proximal operator of the hard margin loss and the projection of cardinality constraint. Extensive numerical results on both simulated and real datasets demonstrate that the proposed method is fast, produces sparse solution of high accuracy, and can lead to effective reduction on active samples and features when compared with several leading solvers.

math.OC↗

arXiv2023

iNALM: An inexact Newton Augmented Lagrangian Method for Zero-One Composite Optimization

Zero-One Composite Optimization (0/1-COP) is a prototype of nonsmooth, nonconvex optimization problems and it has attracted much attention recently. The augmented Lagrangian Method (ALM) has stood out as a leading methodology for such problems. The main purpose of this paper is to extend the classical theory of ALM from smooth problems to 0/1-COP. We propose, for the first time, second-order optimality conditions for 0/1-COP. In particular, under a second-order sufficient condition (SOSC), we prove the R-linear convergence rate of the proposed ALM. In order to identify the subspace used in SOSC, we employ the proximal operator of the 0/1-loss function, leading to an active-set identification technique. Built around this identification process, we design practical stopping criteria for any algorithm to be used for the subproblem of ALM. We justify that Newton's method is an ideal candidate for the subproblem and it enjoys both global and local quadratic convergence. Those considerations result in an inexact Newton ALM (iNALM). The method of iNALM is unique in the sense that it is active-set based, it is inexact (hence more practical), and SOSC plays an important role in its R-linear convergence analysis. The numerical results on both simulated and real datasets show the fast running speed and high accuracy of iNALM when compared with several leading solvers.

math.OC↗

arXiv2021

Global Convergence of Inexact Augmented Lagrangian Method for Zero-One Composite Optimization

We consider the problem of minimizing the sum of a smooth function and a composition of a zero-one loss function with a linear operator, namely zero-one composite optimization problem (0/1-COP). It is a versatile model including the support vector machine (SVM), multi-label classification (MLC), maximum rank correlation (MRC) and so on. However, due to the nonconvexity, discontinuity and NP-hardness of the 0/1-COP, it is intractable to design a globally convergent algorithm and the work attempting to solve it directly is scarce. In this paper, we first define and characterize the proximal stationarity to derive the minimum and the strongly exact penalization of the Lyapunov function, which is a variant of the augmented Lagrangian function for the 0/1-COP. Based on this, we propose an inexact augmented Lagrangian method (IALM) for solving 0/1-COP, where the subproblem is solved by the zero-one Bregman alternating linearized minimization (0/1-BALM) algorithm with low computational complexity. Under some suitable assumptions, we prove that the whole sequence generated by the IALM converges to the local minimizer of 0/1-COP. As a direct application, we obtain the global convergence of IALM under the assumption that the data matrix is full row rank for solving the SVM, MLC and MRC.

math.OC↗