arXiv subjects

Rujun Jiang

Publications and source records attributed to Rujun Jiang.

At least 19 recordsLinked to original sources

arXiv2026

Accelerated Concatenation of Stepsize Schedules for Smooth Strongly Convex Gradient Descent

This work considers stepsize schedules for gradient descent on smooth strongly convex objectives. We develop a curvature-dependent concatenation framework for constructing stepsize schedules of arbitrary length with sharp worst-case guarantees. Within this framework, we introduce two families of schedules for the function-value gap and gradient norm of the last iterate, respectively. For each fixed condition number $κ=L/μ>1$, both families attain iteration complexity $Θ(κ^ρ\log(1/ε))$ for sufficiently small $ε$, where $ρ=\log_{\sqrt2+1}2\approx0.7864$. This complexity is optimal up to a subpolynomial factor in $κ$ among predetermined nonnegative stepsize schedules. The framework also recovers the strongly convex silver schedule at power-of-two iteration budgets and applies to smooth convex and smooth weakly convex functions.

math.OC↗

arXiv2026

Optimal Recursive Composition and Dyadic Phase Laws for Gradient Descent with Predetermined Stepsizes

Predetermined stepsize schedules featuring carefully chosen long steps have recently been shown to accelerate gradient descent (GD) on smooth convex functions. A prominent class of such schedules is built through recursive composition. In this paper, we characterize the convergence of these optimized recursive schedules, revealing a nonconstant log-periodic modulation across prescribed horizons. Specifically, we prove that for every $N \geq 1$, the optimized ConPP/OBS-S schedules satisfy $$f(x_{N-1})-f_\ast \le \frac{1}{2N^pΦ(\log_2N)-1} \frac{L}{2}\|x_0-x_\ast\|^2, \qquad p=\log_2(1+\sqrt2), $$ where $Φ$ is a positive, Lipschitz, nonconstant $1$-periodic function. We derive this by proving that balanced splitting is optimal at every horizon for these constructions, resolving a conjecture of Zhang and Jiang. Furthermore, for their asymmetric counterparts (ConPD/OBS-F schedules), we show that although optimal splits are not necessarily balanced, the same Silver exponent asymptotically persists alongside a distinct log-periodic modulation.

math.OC↗

arXiv2026

Differentiable Partitioning With Dual-Max 3D Wirelength Model and Hybrid Bonding Terminal Awareness for Optimized 3D Placement

Research on 3D integrated circuits (3D-ICs) physical design has expanded rapidly in recent years, driven by the increasing adoption of hybrid bonding and the demand for higher integration density. Continued scaling of hybrid-bonding pitch supports higher vertical interconnect density and finer-grained circuit partitioning in 3D-ICs. Hybrid bonding-enabled 3D-ICs offer substantial benefits in interconnect scaling and system integration, yet tier assignment remains challenging because it jointly determines 3D wirelength and hybrid bonding terminal (HBT) assignment. This paper presents a differentiable partitioning framework that directly optimizes placement-aware tier assignment for 3D-ICs through gradient-based optimization. Discrete tier assignment is relaxed to continuous probabilities, and a Dual-Max 3D wirelength model is introduced as a placement-aware surrogate for tier-specific half-perimeter wirelength (HPWL). In addition, a terminal-aware cutsize penalty gives higher weights to cross-die nets with predicted HBT overlaps, and a local balance penalty discourages regional area overflow on each tier. On ten OpenROAD benchmarks, our method reduces final die-to-die (D2D) HPWL by 8.05% and 14.81% on average relative to hMETIS and TritonPart, respectively. On the seven benchmarks with available CoPlace results, the average HPWL reduction is 10.33%. The source code for the differentiable partitioner and the integrated 3D placement flow is publicly available to support reproducibility.

cs.ET↗

arXiv2026

A Local Geometric Analysis of Maximal Coding Rate Reduction via Error Bounds

While the maximal coding rate reduction (MCR$^2$) objective has become a useful principle for learning compact and discriminative representations, a theoretical understanding of its optimization geometry remains limited, especially regarding why first-order methods often exhibit fast convergence near its local maximizers. In this work, we show that the MCR$^2$ objective satisfies an error-bound property in a neighborhood of its local maximizers. This property guarantees that the distance to the local maximizer set can be bounded above by the gradient norm at any point in the neighborhood, directly implying regularity conditions such as the Polyak--Łojasiewicz inequality and quadratic growth. Leveraging this property, we prove that first-order methods converge linearly to a local maximizer of the MCR$^2$ objective under mild conditions. Numerical experiments validate the predicted local convergence behavior and provide empirical evidence that optimizing the MCR$^2$ objective improves representation geometry while maintaining competitive downstream accuracy.

math.OC↗

arXiv2026

An Alternating Direction Method of Multipliers for Utility-based Shortfall Risk Portfolio Optimization

Utility-based shortfall risk (UBSR), a convex risk measure sensitive to tail losses, has gained popularity in recent years. However, research on computational methods for UBSR optimization remains relatively scarce. In this paper, we propose a fast and scalable algorithm for the UBSR-based portfolio optimization problem. Leveraging the Sample Average Approximation (SAA) framework, we reformulate the problem as a block-separable convex program and solve it efficiently via the alternating direction method of multipliers (ADMM). In the high-dimensional setting, a key challenge arises in one of the subproblems---a projection onto a nonlinear feasibility set defined by the shortfall-risk constraint. We propose a semismooth Newton algorithm that employs an implicit-function transformation of semismooth functions to reduce the problem to a univariate equation involving the Lagrange multiplier. This algorithm achieves local superlinear convergence under mild regularity conditions and global convergence for the common loss functions considered in our experiments. Theoretical convergence guarantees of the proposed algorithms are established, and numerical experiments demonstrate a substantial speedup over state-of-the-art solvers, particularly in high-dimensional regimes.

math.OC↗

arXiv2026

Value-Function Root-Finding Algorithms for Composite Convex Simple Bilevel Optimization

This paper considers composite convex simple bilevel optimization (SBO), which minimizes a composite convex function over the solution set of another composite convex minimization problem. We use a scalar value-function reformulation under which the bilevel optimal value is characterized as the leftmost root of a scalar equation. To evaluate this value function without level-set proximal access, we construct a first-order oracle based on Lagrangian duality. The resulting feasibility and value-error bounds, together with multiplier information, enable a bisection method and a safeguarded Newton-type method to obtain an $(ε,ε)$-solution. Under the stated assumptions and for fixed problem data and initialization tolerance, both methods achieve an operation complexity of $\widetilde{\mathcal{O}}(ε^{-1/2})$, matching, up to logarithmic factors, the known near-optimal dependence for smooth convex SBO and the accelerated first-order rate for unconstrained composite convex optimization.

math.OC↗

arXiv2026

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks

The optimization foundations of deep linear networks have recently received significant attention. However, due to their inherent non-convexity and hierarchical structure, analyzing the loss functions of deep linear networks remains a challenging task. In this work, we study the local geometry of the regularized squared loss of deep linear networks around each critical point. Specifically, we obtain a closed-form characterization of the critical point set building on existing results and establish an error bound for the regularized loss under mild conditions on network width and regularization parameters. Notably, this error bound quantifies the distance from a point to the critical point set in terms of the current gradient norm, which can be used to derive linear convergence of first-order methods. To support our theoretical findings, we conduct numerical experiments and demonstrate that gradient descent converges linearly to a critical point when optimizing the regularized loss of deep linear networks.

math.OC↗

arXiv2026

A Semismooth Newton Augmented Lagrangian Method for Sparse Spectral Risk Optimization

Empirical risk minimization is a standard and effective paradigm for learning predictive models by minimizing average loss. In high-stakes decision-making, however, an average-loss criterion may underrepresent rare but severe losses. Spectral risk measures (SRMs) provide a principled framework by incorporating weighted order statistics of losses, but the induced nonsmoothness and nonseparability from sorting make the resulting optimization problems challenging. We propose a relative inexact proximal augmented Lagrangian method with a semismooth Newton subproblem solver for solving SRM-based optimization problems. Exploiting a dual reformulation and properties of the Moreau envelope, we reduce the subproblems to structured dual-variable formulations, significantly simplifying computation. We provide explicit generalized Jacobian characterizations and tailor the pool adjacent violators algorithm for their efficient evaluation. Numerical results on synthetic and real-data instances show that the proposed method attains lower running times than the tested ADMM baseline while producing comparable stationarity residuals and sparse solutions.

math.OC↗

arXiv2026

Benign Landscape of Quadratic Programs with Orthogonality Constraints and Its Application to Heteroscedastic Probabilistic PCA

In this work, we study the optimization landscape of homogeneous quadratic programs with orthogonality constraints (QPOC) and apply the resulting theory to heteroscedastic probabilistic PCA (HePPCA). For QPOC, we establish a complete characterization of the benign optimization landscape by showing that every critical point is either a global maximizer or a strict saddle point. Our analysis builds on a closed-form characterization of the critical point set, from which we derive a necessary and sufficient condition for global optimality and show that every non-optimal critical point has a direction of positive curvature. As an application, we show that the population version of HePPCA is a special instance of QPOC and therefore has a benign optimization landscape; moreover, it satisfies local geodesic strong concavity near every global maximizer. We furthermore prove that, when the sample size is sufficiently large, the sample version of HePPCA inherits these favorable properties with high probability. Together with existing theory on avoiding strict saddle points, our results provide a theoretical justification for the observed local linear convergence of retraction-based optimization methods to global solutions for both QPOC and HePPCA. Finally, we present numerical experiments to corroborate our theoretical results.

math.OC↗

arXiv2026

A Complete Loss Landscape Analysis of Regularized Deep Matrix Factorization

Despite its wide range of applications across various domains, the optimization foundations of deep matrix factorization (DMF) remain largely open. In this work, we aim to fill this gap by conducting a comprehensive study of the loss landscape of the regularized DMF problem. Toward this goal, we first provide a closed-form characterization of all critical points of the problem. Building on this, we establish precise conditions under which a critical point is a local minimizer, a global minimizer, a strict saddle point, or a non-strict saddle point. Leveraging these results, we derive a necessary and sufficient condition under which every critical point is either a local minimizer or a strict saddle point. This provides insights into why gradient-based methods almost always converge to a local minimizer of the regularized DMF problem. Finally, we conduct numerical experiments to visualize its loss landscape to support our theory.

math.OC↗

arXiv2026

Solving Chance Constrained Programs via a Penalty based Difference of Convex Approach

We develop two penalty based difference of convex (DC) algorithms for solving chance constrained programs. First, leveraging a rank-based DC decomposition of the chance constraint, we propose a proximal penalty based DC algorithm in the primal space that does not require a feasible initialization. Second, to improve numerical stability in the general nonlinear settings, we derive an equivalent lifted formulation with complementary constraints and show that, after minimizing primal variables, the penalized lifted problem admits a tractable DC structure in the dual space over a simple polyhedron. We then develop a penalty based DC algorithm in the lifted space with a finite termination guarantee. We establish exact penalty and stationarity guarantees under mild constraint qualifications and identify the relationship of the local minimizers between the two formulations. Numerical experiments demonstrate the efficiency and effectiveness of our proposed methods compared with state-of-the-art benchmarks.

math.OC↗

arXiv2026

Accelerated Gradient Descent by Concatenation of Stepsize Schedules

This work considers stepsize schedules for gradient descent on smooth convex objectives. We extend the existing literature and propose a unified technique for constructing stepsizes with analytic bounds for an arbitrary number of iterations. This technique constructs new stepsize schedules by concatenating two stepsize schedules with fewer steps. Using this approach, we introduce two new families of stepsize schedules, achieving a convergence rate of $O(n^{-\log_2(\sqrt 2+1)})$ with state-of-the-art constants for the objective value and gradient norm of the last iterate, respectively. Furthermore, our analytically derived stepsize schedules either match or surpass the existing best numerically computed stepsize schedules.

math.OC↗

arXiv2025

Adaptive Algorithms for Nonconvex Bilevel Optimization under PŁ Conditions

Existing methods for nonconvex bilevel optimization (NBO) require prior knowledge of first- and second-order problem-specific parameters (e.g., Lipschitz constants and the Polyak-Łojasiewicz (PŁ) parameters) to set step sizes, a requirement that poses practical limitations when such parameters are unknown or computationally expensive. We introduce the Adaptive Fully First-order Bilevel Approximation (AF${}^2$BA) algorithm and its accelerated variant, A${}^2$F${}^2$BA, for solving NBO problems under the PŁ conditions. To our knowledge, these are the first methods to employ fully adaptive step size strategies, eliminating the need for any problem-specific parameters in NBO. We prove that both algorithms achieve $\mathcal{O}(1/ε^2)$ iteration complexity for finding an $ε$-stationary point, matching the iteration complexity of existing well-tuned methods. Furthermore, we show that A${}^2$F${}^2$BA enjoys a near-optimal first-order oracle complexity of $\tilde{\mathcal{O}}(1/ε^2)$, matching the oracle complexity of existing well-tuned methods, and aligning with the complexity of gradient descent for smooth nonconvex single-level optimization when ignoring the logarithmic factors.

math.OC↗

arXiv2025

An Adaptive Algorithm for Bilevel Optimization on Riemannian Manifolds

Existing methods for solving Riemannian bilevel optimization (RBO) problems require prior knowledge of the problem's first- and second-order information and curvature parameter of the Riemannian manifold to determine step sizes, which poses practical limitations when these parameters are unknown or computationally infeasible to obtain. In this paper, we introduce the Adaptive Riemannian Hypergradient Descent (AdaRHD) algorithm for solving RBO problems. To our knowledge, AdaRHD is the first method to incorporate a fully adaptive step size strategy that eliminates the need for problem-specific parameters in RBO. We prove that AdaRHD achieves an $\mathcal{O}(1/ε)$ iteration complexity for finding an $ε$-stationary point, thus matching the complexity of existing non-adaptive methods. Furthermore, we demonstrate that substituting exponential mappings with retraction mappings maintains the same complexity bound. Experiments demonstrate that AdaRHD achieves comparable performance to existing non-adaptive approaches while exhibiting greater robustness.

math.OC↗

arXiv2025

Linear Convergence of the Proximal Gradient Method for Composite Optimization Under the Polyak-Łojasiewicz Inequality and Its Variant

We study the linear convergence rates of the proximal gradient method for composite functions satisfying two classes of Polyak-Łojasiewicz (PL) inequality: the PL inequality, the variant of PL inequality defined by the proximal map-based residual. Using the performance estimation problem, we either provide new explicit linear convergence rates or improve existing complexity bounds for minimizing composite functions under the two classes of PL inequality. Finally, we illustrate numerically the effects of our theoretical results.

math.OC↗

arXiv2025

Riemannian Adaptive Regularized Newton Methods with Hölder Continuous Hessians

This paper presents strong worst-case iteration and operation complexity guarantees for Riemannian adaptive regularized Newton methods, a unified framework encompassing both Riemannian adaptive regularization (RAR) methods and Riemannian trust region (RTR) methods. We comprehensively characterize the sources of approximation in second-order manifold optimization methods: the objective function's smoothness, retraction's smoothness, and subproblem solver's inexactness. Specifically, for a function with a $μ$-Hölder continuous Hessian, when equipped with a retraction featuring a $ν$-Hölder continuous differential and a $θ$-inexact subproblem solver, both RTR and RAR with $2+α$ regularization (where $α=\min\{μ,ν,θ\}$) locate an $(ε,ε^{α/(1+α)})$-approximate second-order stationary point within at most $O(ε^{-(2+α)/(1+α)})$ iterations and at most $\tilde{O}(ε^{-(4+3α)/(2(1+α))})$ Hessian-vector products. These complexity results are novel and sharp, and reduce to an iteration complexity of $O(ε^{-3/2})$ and an operation complexity of $\tilde{O}(ε^{-7/4})$ when $α=1$.

math.OC↗

arXiv2025

A Proximal DC Algorithm for Sample Average Approximation of Chance Constrained Programming

Chance constrained programming (CCP) refers to a type of optimization problem with uncertain constraints that are satisfied with at least a prescribed probability level. In this work, we study the sample average approximation (SAA) of chance constraints. This is an important approach to solving CCP, especially in the data-driven setting where only a sample of multiple realizations of the random vector in the chance constraints is available. The SAA is obtained by replacing the underlying distribution with an empirical distribution over the available sample. Assuming that the functions in chance constraints are all convex, we reformulate the SAA of chance constraints into a difference-of-convex (DC) form. Moreover, considering that the objective function is a difference-of-convex function, the resulting formulation becomes a DC constrained DC program. Then, we propose a proximal DC algorithm for solving this reformulation. In particular, we show that the subproblems of the proximal DC are suitable for off-the-shelf solvers in some scenarios. Moreover, we not only prove the subsequential and sequential convergence of the proposed algorithm but also derive the iteration complexity for finding an approximate Karush-Kuhn-Tucker point. To support and complement our theoretical development, we show via numerical experiments that our proposed approach is competitive with a host of existing approaches.

math.OC↗

arXiv2024

Penalty-based Methods for Simple Bilevel Optimization under Hölderian Error Bounds

This paper investigates simple bilevel optimization problems where we minimize an upper-level objective over the optimal solution set of a convex lower-level objective. Existing methods for such problems either only guarantee asymptotic convergence, have slow sublinear rates, or require strong assumptions. To address these challenges, we propose a penalization framework that delineates the relationship between approximate solutions of the original problem and its reformulated counterparts. This framework accommodates varying assumptions regarding smoothness and convexity, enabling the application of specific methods with different complexity results. Specifically, when both upper- and lower-level objectives are composite convex functions, under an $α$-H{ö}lderian error bound condition and certain mild assumptions, our algorithm attains an $(ε,ε^β)$-optimal solution of the original problem for any $β> 0$ within $\mathcal{O}\left(\sqrt{{1}/{ε^{\max\{α,β\}}}}\right)$ iterations. The result can be improved further if the smooth part of the upper-level objective is strongly convex. We also establish complexity results when the upper- and lower-level objectives are general nonsmooth functions. Numerical experiments demonstrate the effectiveness of our algorithms.

math.OC↗