arXiv subjects

Jiachen Jin

Publications and source records attributed to Jiachen Jin.

6 recordsLinked to original sources

arXiv2025

A Single-loop Stochastic Riemannian ADMM for Nonsmooth Optimization

We study a class of nonsmooth stochastic optimization problems on Riemannian manifolds. In this work, we propose MARS-ADMM, the first stochastic Riemannian alternating direction method of multipliers with provable near-optimal complexity guarantees. Our algorithm incorporates a momentum-based variance-reduced gradient estimator applied exclusively to the smooth component of the objective, together with carefully designed penalty parameter and dual stepsize updates. Unlike existing approaches that rely on computationally expensive double-loop frameworks, MARS-ADMM operates in a single-loop fashion and requires only a constant number of stochastic gradient evaluations per iteration. Under mild assumptions, we establish that MARS-ADMM achieves an iteration complexity of $\tilde{\mathcal{O}}(\varepsilon^{-3})$, which improves upon the previously best-known bound of $\mathcal{O}(\varepsilon^{-3.5})$ for stochastic Riemannian operator-splitting methods. As a result, our analysis closes the theoretical complexity gap between stochastic Riemannian operator-splitting algorithms and stochastic methods for nonsmooth optimization with nonlinear constraints. Notably, the obtained complexity also matches the best-known bounds in deterministic nonsmooth Riemannian optimization, demonstrating that deterministic-level accuracy can be achieved using only constant-size stochastic samples.

math.OC↗

arXiv2025

Adaptive Riemannian ADMM for Nonsmooth Optimization: Optimal Complexity without Smoothing

We study the problem of minimizing the sum of a smooth function and a nonsmooth convex regularizer over a compact Riemannian submanifold embedded in Euclidean space. By introducing an auxiliary splitting variable, we propose an adaptive Riemannian alternating direction method of multipliers (ARADMM), which, for the first time, achieves convergence without requiring smoothing of the nonsmooth term. Our approach involves only one Riemannian gradient evaluation and one proximal update per iteration. Through careful and adaptive coordination of the stepsizes and penalty parameters, we establish an optimal iteration complexity of order $\mathcal{O}(ε^{-3})$ for finding an $ε$-approximate KKT point, matching the complexity of existing smoothing technique-based Riemannian ADMM methods. Extensive numerical experiments on sparse PCA and robust subspace recovery demonstrate that our ARADMM consistently outperforms state-of-the-art Riemannian ADMM variants in convergence speed and solution quality.

math.OC↗

arXiv2025

Stochastic ADMM with batch size adaptation for nonconvex nonsmooth optimization

Stochastic alternating direction method of multipliers (SADMM) is a popular method for solving nonconvex nonsmooth optimization in various applications. However, it typically requires an empirical selection of the static batch size for gradient estimation, resulting in a challenging trade-off between variance reduction and computational cost. This paper proposes adaptive batch size SADMM, a practical method that dynamically adjusts the batch size based on accumulated differences along the optimization path. We develop a simple convergence analysis to handle the dependence of batch size adaptation that matches the best-known complexity with flexible parameter choices. We further extend this adaptive scheme to reduce the overall complexity of the popular variance-reduced methods, SVRG-ADMM and SPIDER-ADMM. Numerical results validate the effectiveness of our proposed methods.

math.OC↗

arXiv2025

An accelerated proximal PRS-SQP algorithm with dual ascent-descent procedures for smooth composite optimization

Conventional wisdom in composite optimization suggests augmented Lagrangian dual ascent (ALDA) in Peaceman-Rachford splitting (PRS) methods for dual feasibility. However, ALDA may fail when the primal iterate is a local minimum, a stationary point, or a coordinatewise solution of the highly nonconvex augmented Lagrangian function. Splitting sequential quadratic programming (SQP) methods utilize augmented Lagrangian dual descent (ALDD) to directly minimize the primal residual, circumventing the limitations of ALDA and achieving faster convergence in smooth optimization. This paper aims to present a fairly accessible generalization of two contrasting dual updates, ALDA and ALDD, for smooth composite optimization. A key feature of our PRS-SQP algorithm is its dual ascent-descent procedure, which provides a free direction rule for the dual updates and a new insight to explain the counterintuitive convergence behavior. Furthermore, we incorporate a hybrid acceleration technique that combines inertial extrapolation and back substitution to improve convergence. Theoretically, we establish the feasibility for a wider range of acceleration factors than previously known and derive convergence rates within the Kurdyka- Lojasiewicz framework. Numerical experiments validate the effectiveness and stability of the proposed method in various dual-update scenarios.

math.OC↗

arXiv2025

Anderson acceleration of derivative-free projection methods for constrained monotone nonlinear equations

The derivative-free projection method (DFPM) is an efficient algorithm for solving monotone nonlinear equations. As problems grow larger, there is a strong demand for speeding up the convergence of DFPM. This paper considers the application of Anderson acceleration (AA) to DFPM for constrained monotone nonlinear equations. By employing a nonstationary relaxation parameter and interleaving with slight modifications in each iteration, a globally convergent variant of AA for DFPM named as AA-DFPM is proposed. Further, the linear convergence rate is proved under some mild assumptions. Experiments on both mathematical examples and a real-world application show encouraging results of AA-DFPM and confirm the suitability of AA for accelerating DFPM in solving optimization problems.

math.OC↗

arXiv2025

Stochastic momentum ADMM for nonconvex and nonsmooth optimization with application to PnP algorithm

This paper proposes SMADMM, a single-loop Stochastic Momentum Alternating Direction Method of Multipliers for solving a class of nonconvex and nonsmooth composite optimization problems. SMADMM achieves the optimal oracle complexity of $\mathcal{O}(ε^{-3/2})$ in the online setting. Unlike previous stochastic ADMM algorithms that require large mini-batches or a double-loop structure, SMADMM uses only $\mathcal{O}(1)$ stochastic gradient evaluations per iteration and avoids costly restarts. To further improve practicality, we incorporate dynamic step sizes and penalty parameters, proving that SMADMM maintains its optimal complexity without the need for large initial batches. We also develop PnP-SMADMM by integrating plug-and-play priors, and establish its theoretical convergence under mild assumptions. Extensive experiments on classification, CT image reconstruction, and phase retrieval tasks demonstrate that our approach outperforms existing stochastic ADMM methods both in accuracy and efficiency, validating our theoretical results.

math.OC↗