arXiv subjects

Qingna Li

Publications and source records attributed to Qingna Li.

At least 19 recordsLinked to original sources

arXiv2026

Exact-Penalty Prox-Linear Methods for Bilevel Optimization with $\ell_1$ Lower-Level Gradient Penalty

Bilevel optimization is a fundamental framework for hierarchical decision-making, but its solution is challenging due to the implicit and typically set-valued nature of the lower-level optimality condition. In this paper, we study bilevel optimization problems through an exact-penalty reformulation based on the $\ell_1$-norm of the lower-level gradient. Under suitable regularity assumptions, we show that this penalty defines a distance-bound function and yields an exact penalty property for sufficiently large penalty parameters. To solve the resulting nonsmooth penalized problem, we propose an exact-penalty prox-linear (EPPL) method and establish a stationarity-oriented convergence guarantee. We further specialize the method to the simple bilevel setting, where the subproblem admits an explicit dual reformulation as a box-constrained quadratic program. This structure leads to a dual spectral projected gradient method with closed-form primal recovery, for which convergence of the inner dual iterates is proved. Numerical experiments on a minimum-norm least-squares bilevel model show that the proposed method is effective in reducing both the lower-level and upper-level gaps to high accuracy. Compared with several existing methods, the proposed approach attains the best final solution accuracy on the tested instance.

math.OC↗

arXiv2026

Fast Algorithms for Optimal Damping in Mechanical Systems

Optimal damping aims at determining a vector of damping coefficients $\nu$ that maximizes the decay rate of a mechanical system's response. This problem can be formulated as the minimization of the trace of the solution of a Lyapunov equation whose coefficient matrix depends on $\nu$. For physical relevance, the damping coefficients must be nonnegative and the resulting system must be asymptotically stable. We identify conditions under which the system is never stable or may lose stability for certain choices of $\nu$. In the latter case, we propose replacing the constraint $\nu \ge 0$ with $\nu \ge d$, where $d$ is a nonzero nonnegative vector chosen to ensure stability. We derive explicit expressions for the gradient and Hessian of the objective function and show that the Karush--Kuhn--Tucker conditions are equivalent to the vanishing of a nonlinear residual function at an optimal solution. To compute such a solution, we propose a Barzilai--Borwein residual minimization algorithm (BBRMA), which is simple and efficient but not globally convergent, and a spectral projected gradient (SPG) method, which is globally convergent. By exploiting the structure of the problem, we show how to efficiently compute the objective function and its gradient, with eigenvalue decompositions constituting the dominant cost in terms of execution time. Numerical experiments show that both methods require fewer eigenvalue decompositions than the fast optimal damping algorithm (FODA), and that, although SPG may incur additional decompositions due to line search, it often converges faster than BBRMA, leading to a lower overall computational effort.

math.NA↗

arXiv2025

Bilevel Models for Adversarial Learning and A Case Study

Adversarial learning has been attracting more and more attention thanks to the fast development of machine learning and artificial intelligence. However, due to the complicated structure of most machine learning models, the mechanism of adversarial attacks is not well interpreted. How to measure the effect of attacks is still not quite clear. In this paper, we investigate the adversarial learning from the perturbation analysis point of view. We characterize the robustness of learning models through the calmness of the solution mapping. In the case of convex clustering models, we identify the conditions under which the clustering results remain the same under perturbations. When the noise level is large, it leads to an attack. Therefore, we propose two bilevel models for adversarial learning where the effect of adversarial learning is measured by some deviation function. Specifically, we systematically study the so-called $\delta$-measure and show that under certain conditions, it can be used as a deviation function in adversarial learning for convex clustering models. Finally, we conduct numerical tests to verify the above theoretical results as well as the efficiency of the two proposed bilevel models.

cs.LG↗

arXiv2025

A Robust EDM Optimization Approach for 3D Single-Source Localization with Angle and Range Measurements

Accurate source localization in Multi-Platform Radar Networks (MPRNs) benefits from exploiting both range and angle measurements under robust estimation. In this paper, we propose a robust Euclidean distance matrix (EDM) optimization model that simultaneously integrates range measurements, angle information, and the least absolute deviation ($\ell_1$-norm) criterion for the case of 3D single-source localization (3DSSL). A key theoretical contribution of this work is the rigorous reformulation of {existing} 3D angle measurements into simple box constraints on the Euclidean distances. Unlike previous approximations, we achieve this by reducing each of the 3D angle measurements to a two-dimensional nonlinear optimization problem, whose global minimum and maximum solutions can be characterized and utilized to get the lower and upper bounds of the distances from the unknown source to the sensors. To solve the resulting rank-constrained EDM problem, we develop an efficient algorithm based on the majorization penalty method. Extensive numerical experiments confirm that the new EDM model significantly outperforms leading solvers in terms of localization accuracy and computational efficiency, particularly in low Signal-to-Noise Ratio (SNR) scenarios.

eess.SP↗

arXiv2025

On Constraint Qualifications for MPECs with Applications to Bilevel Hyperparameter Optimization for Machine Learning

Constraint qualifications for a Mathematical Program with Equilibrium Constraints (MPEC) are essential for analyzing stationarity properties and establishing convergence results. In this paper, we explore several classical MPEC constraint qualifications and clarify the relationships among them. We subsequently examine the behavior of these constraint qualifications in the context of a specific MPEC derived from bilevel hyperparameter optimization (BHO) for L1-loss support vector classification. In particular, for such an MPEC, we provide a complete characterization of the well-known MPEC linear independence constraint qualification (MPEC-LICQ), therefore, establishing conditions under which it holds or fails for our BHO for support vector machines.

math.OC↗

arXiv2025

Modified Block Newton Algorithm for $\ell_0$-Regularized Optimization

In this paper, we propose a globally convergent Newton type method to solve $\ell_0$ regularized sparse optimization problem. In fact, a line search strategy is applied to the Newton method to obtain global convergence. The Jacobian matrix of the original problem is a block upper triangular matrix. To reduce the computational burden, our method only requires the calculation of the block diagonal. We also introduced regularization to overcome matrix singularity. Although we only use the block-diagonal part of the Jacobian matrix, our algorithm still maintains global convergence and achieves a local quadratic convergence rate. Numerical results demonstrate the efficiency of our method.

math.OC↗

arXiv2025

An Efficient Smoothing Damped Newton Method for Large-Scale Mathematical Programs with Equilibrium Constraints

Bilevel hyperparameter optimization has received growing attention thanks to the fast development of machine learning. Due to the tremendous size of data sets, the scale of bilevel hyperparameter optimization problem could be extremely large, posing great challenges in designing efficient numerical algorithms. In this paper, we focus on solving the large-scale mathematical programs with equilibrium constraints (MPEC) derived from hyperparameter selection of L1 support vector classification (L1-SVC). We propose a highly efficient smoothing damped Newton method (SDNM) for solving such MPEC. Compared with most existing algorithms where subproblems are solved by packages, our approach fully takes advantage of the structure of MPEC and therefore is package-free. Moreover, the proposed SDNM converges to C-stationary point under MPEC-LICQ with subproblem enjoys a quadratic convergence rate under proper assumptions. Extensive numerical results over LIBSVM dataset show the superior performance of SDNM over other state-of-art algorithms.

math.OC↗

arXiv2025

Proximal Iterative Hard Thresholding Algorithm for Sparse Group $\ell_0$-Regularized Optimization with Box Constraint

This paper investigates a general class of problems in which a lower bounded smooth convex function incorporating $\ell_{0}$ and $\ell_{2,0}$ regularization is minimized over a box constraint. Although such problems arise frequently in practical applications, their inherent non-convexity poses significant challenges for solution methods. In particular, we focus on the proximal operator associated with these regularizations, which incorporates both group-sparsity and element-wise sparsity terms. Besides, we introduce the concepts of $\tau$-stationary point and support optimal (SO) point then analyze their relationship with the minimizer of the considered problem. Based on the proximal operator, we propose a novel proximal iterative hard thresholding algorithm to solve the problem. Furthermore, we establish the global convergence and the computational complexity analysis of the proposed method. Finally, extensive experiments demonstrate the effectiveness and efficiency of our method.

math.OC↗

arXiv2025

Subspace Newton's Method for $\ell_0$-Regularized Optimization Problems with Box Constraint

This paper investigates the box-constrained $\ell_0$-regularized sparse optimization problem. We introduce the concept of a $\tau$-stationary point and establish its connection to the local and global minima of the box-constrained $\ell_0$-regularized sparse optimization problem. We utilize the $\tau$-stationary points to define the support set, which we divide into active and inactive components. Subsequently, the Newton's method is employed to update the non-active variables, while the proximal gradient method is utilized to update the active variables. If the Newton's method fails, we use the proximal gradient step to update all variables. Under some mild conditions, we prove the global convergence and the local quadratic convergence rate. Finally, experimental results demonstrate the efficiency of our method.

math.OC↗

arXiv2025

Inexact Regularized Quasi-Newton Algorithm for Solving Monotone Variational Inequality Problems

Newton's method has been an important approach for solving variational inequalities, quasi-Newton method is a good alternative choice to save computational cost. In this paper, we propose a new method for solving monotone variational inequalities where we introduce a merit function based on the merit function. With the help of the merit function, we can locally accepts unit step size. And a globalization technique based on the hyperplane is applied to the method. The proposed method applied to monotone variational inequality problems is globally convergent in the sense that subproblems always have unique solutions, and the whole sequence of iterates converges to a solution of the problem without any regularity assumptions. We also provide extensive numerical results to demonstrate the efficiency of the proposed algorithm.

math.OC↗

arXiv2025

A Spectral Projected Gradient Method for Computational Protein Design problem

In this paper, we consider the computational protein design (CPD) problem, which is usually modeled as 0/1 programming and is extremely challenging due to its combinatorial properties. As a quadratic semi-assignment problem (QSAP), the CPD problem has been proved to be equivalent to its continuous relaxation problem (RQSAP), in terms of sharing the same optimal objective value. However, since the current algorithm for solving this RQSAP uses the projected Newton method, which requires direct computation of the Hessian matrix, its computational cost remains quite high. Precisely for this reason, we choose to employ the spectral projected gradient (SPG) method to solve the CPD problem, whose effectiveness relies on choosing the step lengths according to novel ideas that are related to the spectrum of the underlying local Hessian. Specifically, we apply the SPG method in two distinct ways: direct solving the relaxation problem and applying a penalty method. Numerical results on benchmark instances verify the superior performance of our approach over the current algorithms in both quality and efficiency.

math.OC↗

arXiv2025

A Matrix Optimization Method for Blind Extraction of External Equitable Partitions from Low Pass Graph Signals

Seeking the external equitable partitions (EEPs) of networks under unknown structures is an emerging problem in network analysis. The special structure of EEPs has found widespread applications in many fields such as cluster synchronization and consensus dynamics. While most literature focuses on utilizing the special structural properties of EEPs for network studies, there has been little work on the extraction of EEPs or their connection with graph signals. In this paper, we address the interesting connection between low pass graph signals and EEPs, which, as far as we know, is the first time. We provide a method BE-EEPs for extracting EEPs from low pass graph signals and propose an optimization model, which is essentially a problem involving nonnegative orthogonality matrix decomposition. We derive theoretical error bounds for the performance of our proposed method under certain assumptions and apply three algorithms to solve the resulting model, including the K-means algorithm, the practical exact penalty method and the iterative Lagrangian approach. Numerical experiments verify the effectiveness of the proposed method. Under strong low pass graph signals, the iterative Lagrangian and K-means perform equally well, outperforming the exact penalty method. However, under complex weak low pass signals, all three perform equally well.

math.OC↗

arXiv2025

An Efficient Quadratic Penalty Method for a Class of Graph Clustering Problems

Community-based graph clustering is one of the most popular topics in the analysis of complex social networks. This type of clustering involves grouping vertices that are considered to share more connections, whereas vertices in different groups share fewer connections. A successful clustering result forms densely connected induced subgraphs. This paper studies a specific form of graph clustering problems that can be formulated as semi-assignment problems, where the objective function exhibits block properties. We reformulate these problems as sparse-constrained optimization problems and relax them to continuous optimization models. We then apply the quadratic penalty method and the quadratic penalty regularized method to the relaxation problem, respectively. Extensive numerical experiments demonstrate that both methods effectively solve graph clustering tasks for both synthetic and real-world network datasets. For small-scale problems, the quadratic penalty regularized method demonstrates greater efficiency, whereas the quadratic penalty method proves more suitable for large-scale cases.

math.OC↗

arXiv2024

A Highly Efficient Adaptive-Sieving-Based Algorithm for the High-Dimensional Rank Lasso Problem

The high-dimensional rank lasso (hdr lasso) model is an efficient approach to deal with high-dimensional data analysis. It was proposed as a tuning-free robust approach for the high-dimensional regression and was demonstrated to enjoy several statistical advantages over other approaches. The hdr lasso problem is essentially an $L_1$-regularized optimization problem whose loss function is Jaeckel's dispersion function with Wilcoxon scores. Due to the nondifferentiability of the above loss function, many classical algorithms for lasso-type problems are unable to solve this model. In this paper, inspired by the adaptive sieving strategy for the exclusive lasso problem [1], we propose an adaptive-sieving-based algorithm to solve the hdr lasso problem. The proposed algorithm makes full use of the sparsity of the solution. In each iteration, a subproblem with the same form as the original model is solved, but in a much smaller size. We apply the proximal point algorithm to solve the subproblem, which fully takes advantage of the two nonsmooth terms. Extensive numerical results demonstrate that the proposed algorithm (AS-PPA) is robust for different types of noises, which verifies the attractive statistical property as shown in [2]. Moreover, AS-PPA is also highly efficient, especially for the case of high-dimensional features, compared with other methods.

math.OC↗

arXiv2023

Global relaxation-based LP-Newton method for multiple hyperparameter selection in support vector classification with feature selection

Support vector classification (SVC) is an effective tool for classification tasks in machine learning. Its performance relies on the selection of appropriate hyperparameters. This paper focuses on optimizing the regularization hyperparameter C and determining feature bounds for feature selection within SVC, leading to a potentially large hyperparameter space. It is well known in machine learning that this can lead to the so-called curse of dimensionality. To address this challenge of multiple hyperparameter selection, the problem is formulated as a bilevel optimization problem, which is then transformed into a mathematical program with equilibrium constraints (MPEC). Our primary contributions are twofold. First, we establish the satisfaction of the MPEC-MFCQ for our problem reformulation. Furthermore, we introduce a novel global relaxation-based linear programming (LP)-Newton method (GRLPN) for solving this problem and provide corresponding convergence results. Typically, in global relaxation methods for MPECs, the algorithm for the corresponding subproblem is treated as a black box. Possibly for the first time in the literature, the subproblem is specifically studied in detail. Numerical experiments demonstrate GRLPN's superiority in efficiency and accuracy over both grid search and traditional global relaxation methods solved using the well-known nonlinear programming solver SNOPT.

math.OC↗

arXiv2023

A Fast Smoothing Newton Method for Bilevel Hyperparameter Optimization for SVC with Logistic Loss

Support vector classification (SVC) with logistic loss has excellent theoretical properties in classification problems where the label values are not continuous. In this paper, we reformulate the hyperparameter selection for SVC with logistic loss as a bilevel optimization problem in which the upper-level problem and the lower-level problem are both based on logistic loss. The resulting bilevel optimization model is converted to a single-level nonlinear programming (NLP) problem based on the KKT conditions of the lower-level problem. Such NLP contains a set of nonlinear equality constraints and a simple lower bound constraint. The second-order sufficient condition is characterized, which guarantees that the strict local optimizers are obtained. To solve such NLP, we apply the smoothing Newton method proposed in \cite{Liang} to solve the KKT conditions, which contain one pair of complementarity constraints. We show that the smoothing Newton method has a superlinear convergence rate. Extensive numerical results verify the efficiency of the proposed approach and strict local minimizers can be achieved both numerically and theoretically. In particular, compared with other methods, our algorithm can achieve competitive results while consuming less time than other methods.

math.OC↗

arXiv2023

An Efficient Algorithm for Computational Protein Design Problem

In this paper, we consider the computational protein design (CPD) problem, which is usually modeled as a 0/1 programming and is extremely challenging due to its combinatorial properties. We propose an efficient algorithm for solving it. Specifically, we study the quadratic semi-assignment problem formulation (QSAP) of the CPD problem, and show that it is equivalent to its continuous relaxation problem (RQSAP), in terms of sharing the same optimal objective value. Then, we propose an efficient penalty method to solve the QSAP based on the proposed formulations, which is guaranteed to converge to a global solution of the QSAP under certain conditions. Compared with existing branch-and-bound approaches that suffer from high computational complexity, the proposed algorithm is based on a continuous problem and enjoys a low per-iteration complexity, which makes it particularly suitable for solving large-scale CPD problems. Numerical results on benchmark instances verify the superior performance of our approach over the state-of-the-art branch-and-cut solvers. In particular, the proposed algorithm outperforms the state-of-the-art solvers by three order of magnitude in CPU time in most cases, while it still returns high-quality solutions.

math.OC↗

arXiv2022

An Efficient Method for Sample Adversarial Perturbations against Nonlinear Support Vector Machines

Adversarial perturbations have drawn great attentions in various machine learning models. In this paper, we investigate the sample adversarial perturbations for nonlinear support vector machines (SVMs). Due to the implicit form of the nonlinear functions mapping data to the feature space, it is difficult to obtain the explicit form of the adversarial perturbations. By exploring the special property of nonlinear SVMs, we transform the optimization problem of attacking nonlinear SVMs into a nonlinear KKT system. Such a system can be solved by various numerical methods. Numerical results show that our method is efficient in computing adversarial perturbations.

cs.LG↗