arXiv subjects

Yangjing Zhang

Publications and source records attributed to Yangjing Zhang.

15 recordsLinked to original sources

arXiv2026

Exact low-dimensional reformulations for regularized spectral approximation

We study exact low-dimensional reformulations for a regularized spectral approximation problem and its weighted variant. In the unweighted setting, weak-majorization monotonicity of the fidelity term enables an exact reformulation over singular values. In the weighted setting, we establish such a reformulation under compatibility and simultaneous diagonalization conditions. We further show, via a counterexample, that this exact reformulation can fail in the absence of simultaneous diagonalization.

math.OC↗

arXiv2026

Scalable Kernel Quantile Regression: A Preconditioned Augmented Lagrangian Method

Kernel quantile regression (KQR) extends classical quantile regression to nonlinear settings using kernel methods, offering a powerful tool for modeling conditional distributions. However, its application to large-scale datasets remains challenging due to two intrinsic difficulties: the nonsmoothness of the quantile check loss and the computational burden imposed by the large, dense kernel matrix. Existing state-of-the-art solvers often struggle to handle both challenges simultaneously, leading to limited scalability and high computational cost. In this paper, we propose PALM-KQR, a highly efficient two-phase preconditioned augmented Lagrangian method for large-scale KQR. In the first phase, an inexact alternating direction method of multipliers (ADMM) is employed to compute a warm-start solution efficiently. The second phase refines this solution using an efficient semismooth Newton augmented Lagrangian method (ALM). Our key innovations include a dual semismooth Newton approach for handling the nonsmooth quantile check loss, and a specialized preconditioning strategy based on low-rank approximations of the kernel matrix, which exploits its structure and mitigates ill-conditioning in the linear systems arising in ALM, thereby significantly accelerating the iterative solvers. Extensive numerical experiments demonstrate that PALM-KQR substantially outperforms existing commercial and specialized KQR solvers in both efficiency and scalability.

math.OC↗

arXiv2026

Graph-Guided Fused Regularization for Single- and Multi-Task Regression on Spatiotemporal Data

Spatiotemporal matrix-valued data arise frequently in modern applications, yet performing effective regression analysis remains challenging due to complex, dimension-specific dependencies. In this work, we propose a regularized framework for spatiotemporal matrix regression that characterizes temporal and spatial dependencies through tailored penalties. Specifically, the model incorporates a fused penalty to capture smooth temporal evolution and a graph-guided penalty to promote spatial similarity. The framework also extends to the multi-task setting, enabling joint estimation across related tasks. We provide a comprehensive analysis of the framework from both theoretical and computational perspectives. Theoretically, we establish the statistical consistency of the proposed estimators. Computationally, we develop an efficient solver based on the Halpern Peaceman-Rachford method for the resulting composite convex optimization problem. The proposed algorithm achieves a fast global non-ergodic $\mathcal{O}(1/k)$ convergence rate with low per-iteration complexity. Extensive numerical experiments demonstrate that our method significantly outperforms state-of-the-art approaches in terms of predictive accuracy and estimation error, while also exhibiting superior computational efficiency and scalability.

math.OC↗

arXiv2025

On some perturbation properties of nonsmooth optimization on Riemannian manifolds with applications

This paper presents a perturbation analysis framework for nonsmooth optimization on connected Riemannian manifolds to bridge the gap between the rapid development of algorithmic approaches and a robust theoretical foundation. Using tangent-space local models, we transport core notions from Euclidean variational analysis, such as strong regularity, the Aubin property, and isolated calmness of the Karush-Kuhn-Tucker (KKT) solution mapping, to the manifold setting. Furthermore, we introduce the manifold (strong) variational sufficiency and show that its strong version is intrinsic, i.e., independent of the chosen retraction, and for polyhedral, second-order cone, and semidefinite programs, it coincides with the manifold strong second-order sufficient condition. These insights yield concrete algorithmic consequences. We show that the Riemannian Sequential Quadratic Programming achieves local superlinear and, under mild additional assumptions, quadratic convergence without strict complementarity, while the Riemannian Augmented Lagrangian Method attains R-linear convergence even when Lagrange multipliers are nonunique. Moreover, the proposed condition guarantees positive definiteness of the generalized Hessians associated with the augmented Lagrangian, enabling superlinear semismooth Newton steps in inner solves. Numerical experiments on robust matrix completion and compressed modes validate the theoretical predictions.

math.OC↗

arXiv2025

Alternating minimization for square root principal component pursuit

Recently, the square root principal component pursuit (SRPCP) model has garnered significant research interest. It is shown in the literature that the SRPCP model guarantees robust matrix recovery with a universal, constant penalty parameter. While its statistical advantages are well-documented, the computational aspects from an optimization perspective remain largely unexplored. In this paper, we focus on developing efficient optimization algorithms for solving the SRPCP problem. Specifically, we propose a tuning-free alternating minimization (AltMin) algorithm, where each iteration involves subproblems enjoying closed-form optimal solutions. Additionally, we introduce techniques based on the variational formulation of the nuclear norm and Burer-Monteiro decomposition to further accelerate the AltMin method. Extensive numerical experiments confirm the efficiency and robustness of our algorithms.

math.OC↗

arXiv2025

Multiple Regression for Matrix and Vector Predictors: Models, Theory, Algorithms, and Beyond

Matrix regression plays an important role in modern data analysis due to its ability to handle complex relationships involving both matrix and vector variables. We propose a class of regularized regression models capable of predicting both matrix and vector variables, accommodating various regularization techniques tailored to the inherent structures of the data. We establish the consistency of our estimator when penalizing the nuclear norm of the matrix variable and the $\ell_1$ norm of the vector variable. To tackle the general regularized regression model, we propose a unified framework based on an efficient preconditioned proximal point algorithm. Numerical experiments demonstrate the superior estimation and prediction accuracy of our proposed estimator, as well as the efficiency of our algorithm compared to the state-of-the-art solvers.

math.OC↗

arXiv2024

Low Rank Convex Clustering For Matrix-Valued Observations

Common clustering methods, such as $k$-means and convex clustering, group similar vector-valued observations into clusters. However, with the increasing prevalence of matrix-valued observations, which often exhibit low rank characteristics, there is a growing need for specialized clustering techniques for these data types. In this paper, we propose a low rank convex clustering model tailored for matrix-valued observations. Our approach extends the convex clustering model originally designed for vector-valued data to classify matrix-valued observations. Additionally, it serves as a convex relaxation of the low rank $k$-means method proposed by Z. Lyu, and D. Xia (arXiv:2207.04600). Theoretically, we establish exact cluster recovery for finite samples and asymptotic cluster recovery as the sample size approaches infinity. We also give a finite sample bound on prediction error in terms of centroid estimation, and further establish the prediction consistency. To make the model practically useful, we develop an efficient double-loop algorithm for solving it. Extensive numerical experiments are conducted to show the effectiveness of our proposed model.

math.OC↗

arXiv2024

DNNLasso: Scalable Graph Learning for Matrix-Variate Data

We consider the problem of jointly learning row-wise and column-wise dependencies of matrix-variate observations, which are modelled separately by two precision matrices. Due to the complicated structure of Kronecker-product precision matrices in the commonly used matrix-variate Gaussian graphical models, a sparser Kronecker-sum structure was proposed recently based on the Cartesian product of graphs. However, existing methods for estimating Kronecker-sum structured precision matrices do not scale well to large scale datasets. In this paper, we introduce DNNLasso, a diagonally non-negative graphical lasso model for estimating the Kronecker-sum structured precision matrix, which outperforms the state-of-the-art methods by a large margin in both accuracy and computational time. Our code is available at https://github.com/YangjingZhang/DNNLasso.

cs.LG↗

arXiv2023

On Regularized Square-root Regression Problems: Distributionally Robust Interpretation and Fast Computations

Square-root (loss) regularized models have recently become popular in linear regression due to their nice statistical properties. Moreover, some of these models can be interpreted as the distributionally robust optimization counterparts of the traditional least-squares regularized models. In this paper, we give a unified proof to show that any square-root regularized model whose penalty function being the sum of a simple norm and a seminorm can be interpreted as the distributionally robust optimization (DRO) formulation of the corresponding least-squares problem. In particular, the optimal transport cost in the DRO formulation is given by a certain dual form of the penalty. To solve the resulting square-root regularized model whose loss function and penalty function are both nonsmooth, we design a proximal point dual semismooth Newton algorithm and demonstrate its efficiency when the penalty is the sparse group Lasso penalty or the fused Lasso penalty. Extensive experiments demonstrate that our algorithm is highly efficient for solving the square-root sparse group Lasso problems and the square-root fused Lasso problems.

math.OC↗

arXiv2023

Learning Graph Laplacian with MCP

We consider the problem of learning a graph under the Laplacian constraint with a non-convex penalty: minimax concave penalty (MCP). For solving the MCP penalized graphical model, we design an inexact proximal difference-of-convex algorithm (DCA) and prove its convergence to critical points. We note that each subproblem of the proximal DCA enjoys the nice property that the objective function in its dual problem is continuously differentiable with a semismooth gradient. Therefore, we apply an efficient semismooth Newton method to subproblems of the proximal DCA. Numerical experiments on various synthetic and real data sets demonstrate the effectiveness of the non-convex penalty MCP in promoting sparsity. Compared with the existing state-of-the-art method, our method is demonstrated to be more efficient and reliable for learning graph Laplacian with MCP.

cs.LG↗

arXiv2023

Strong Variational Sufficiency for Nonlinear Semidefinite Programming and its Implications

Strong variational sufficiency is a newly proposed property, which turns out to be of great use in the convergence analysis of multiplier methods. However, what this property implies for non-polyhedral problems remains a puzzle. In this paper, we prove the equivalence between the strong variational sufficiency and the strong second order sufficient condition (SOSC) for nonlinear semidefinite programming (NLSDP), without requiring the uniqueness of multiplier or any other constraint qualifications. Based on this characterization, the local convergence property of the augmented Lagrangian method (ALM) for NLSDP can be established under strong SOSC in the absence of constraint qualifications. Moreover, under the strong SOSC, we can apply the semi-smooth Newton method to solve the ALM subproblems of NLSDP as the positive definiteness of the generalized Hessian of augmented Lagrangian function is satisfied.

math.OC↗

arXiv2022

On Efficient and Scalable Computation of the Nonparametric Maximum Likelihood Estimator in Mixture Models

In this paper we study the computation of the nonparametric maximum likelihood estimator (NPMLE) in multivariate mixture models. Our first approach discretizes this infinite dimensional convex optimization problem by fixing the support points of the NPMLE and optimizing over the mixture proportions. In this context we propose, leveraging the sparsity of the solution, an efficient and scalable semismooth Newton based augmented Lagrangian method (ALM). Our algorithm beats the state-of-the-art methods~\cite{koenker2017rebayes, kim2020fast} and can handle $n \approx 10^6$ data points with $m \approx 10^4$ support points. Our second procedure, which combines the expectation-maximization (EM) algorithm with the ALM approach above, allows for joint optimization of both the support points and the probability weights. For both our algorithms we provide formal results on their (superlinear) convergence properties. The computed NPMLE can be immediately used for denoising the observations in the framework of empirical Bayes. We propose new denoising estimands in this context along with their consistent estimates. Extensive numerical experiments are conducted to illustrate the effectiveness of our methods. In particular, we employ our procedures to analyze two astronomy data sets: (i) Gaia-TGAS Catalog~\cite{anderson2018improving} containing $n \approx 1.4 \times 10^6$ data points in two dimensions, and (ii) the $d=19$ dimensional data set from the APOGEE survey~\cite{majewski2017apache} with $n \approx 2.7 \times 10^4$.

stat.ME↗

arXiv2020

A Proximal Point Dual Newton Algorithm for Solving Group Graphical Lasso Problems

Undirected graphical models have been especially popular for learning the conditional independence structure among a large number of variables where the observations are drawn independently and identically from the same distribution. However, many modern statistical problems would involve categorical data or time-varying data, which might follow different but related underlying distributions. In order to learn a collection of related graphical models simultaneously, various joint graphical models inducing sparsity in graphs and similarity across graphs have been proposed. In this paper, we aim to propose an implementable proximal point dual Newton algorithm (PPDNA) for solving the group graphical Lasso model, which encourages a shared pattern of sparsity across graphs. Though the group graphical Lasso regularizer is non-polyhedral, the asymptotic superlinear convergence of our proposed method PPDNA can be obtained by leveraging on the local Lipschitz continuity of the Karush-Kuhn-Tucker solution mapping associated with the group graphical Lasso model. A variety of numerical experiments on real data sets illustrates that the PPDNA for solving the group graphical Lasso model can be highly efficient and robust.

math.OC↗

arXiv2019

An Efficient Linearly Convergent Regularized Proximal Point Algorithm for Fused Multiple Graphical Lasso Problems

Nowadays, analysing data from different classes or over a temporal grid has attracted a great deal of interest. As a result, various multiple graphical models for learning a collection of graphical models simultaneously have been derived by introducing sparsity in graphs and similarity across multiple graphs. This paper focuses on the fused multiple graphical Lasso model which encourages not only shared pattern of sparsity, but also shared values of edges across different graphs. For solving this model, we develop an efficient regularized proximal point algorithm, where the subproblem in each iteration of the algorithm is solved by a superlinearly convergent semismooth Newton method. To implement the semismooth Newton method, we derive an explicit expression for the generalized Jacobian of the proximal mapping of the fused multiple graphical Lasso regularizer. Unlike those widely used first order methods, our approach has heavily exploited the underlying second order information through the semismooth Newton method. This can not only accelerate the convergence of the algorithm, but also improve its robustness. The efficiency and robustness of our proposed algorithm are demonstrated by comparing with some state-of-the-art methods on both synthetic and real data sets. Supplementary materials for this article are available online.

math.OC↗

arXiv2017

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

The sparse group Lasso is a widely used statistical model which encourages the sparsity both on a group and within the group level. In this paper, we develop an efficient augmented Lagrangian method for large-scale non-overlapping sparse group Lasso problems with each subproblem being solved by a superlinearly convergent inexact semismooth Newton method. Theoretically, we prove that, if the penalty parameter is chosen sufficiently large, the augmented Lagrangian method converges globally at an arbitrarily fast linear rate for the primal iterative sequence, the dual infeasibility, and the duality gap of the primal and dual objective functions. Computationally, we derive explicitly the generalized Jacobian of the proximal mapping associated with the sparse group Lasso regularizer and exploit fully the underlying second order sparsity through the semismooth Newton method. The efficiency and robustness of our proposed algorithm are demonstrated by numerical experiments on both the synthetic and real data sets.

math.OC↗