arXiv subjects

Rujun Jiang

Publications and source records attributed to Rujun Jiang.

At least 19 recordsLinked to original sources

arXiv2026

Optimal Recursive Composition and Dyadic Phase Laws for Gradient Descent with Predetermined Stepsizes

Predetermined stepsize schedules featuring carefully chosen long steps have recently been shown to accelerate gradient descent (GD) on smooth convex functions. A prominent class of such schedules is built through recursive composition. In this paper, we characterize the convergence of these optimized recursive schedules, revealing a non-constant log-periodic modulation across prescribed horizons. Specifically, for symmetric recursive frameworks (primitive and OBS-S constructions), we prove that for every $N \geq 1$, the corresponding optimized schedules satisfy $f(x_{N-1})-f^\ast \le \frac{1}{2N^p \Phi(\log_2N)-1} \frac{L}{2}\|x_0-x^\ast\|^2$, $ p=\log_2(1+\sqrt2)$, where $\Phi$ is a positive, Lipschitz, nonconstant $1$-periodic function. We derive this by proving that balanced splitting is optimal at every horizon for these constructions, resolving a conjecture of Zhang and Jiang. Furthermore, for the asymmetric framework (the OBS-F construction), we show that although optimal splits are not necessarily balanced, the same Silver exponent asymptotically persists alongside a distinct log-periodic modulation.

math.OC↗

arXiv2026

Differentiable Partitioning with Placement and Hybrid Bonding Terminal Awareness for Optimized 3D Placement

Research on 3D-ICs physical design has expanded rapidly in recent years. Hybrid bonding-enabled 3D integrated circuits (3D-ICs) offer substantial benefits in interconnect scaling and system integration, yet tier assignment remains challenging because it jointly determines 3D wirelength and hybrid bonding terminal (HBT) assignment. This paper presents a differentiable partitioning framework that directly optimizes placement-aware tier assignment for 3D-ICs through gradient-based optimization. Discrete tier assignment is relaxed to continuous probabilities, and a Dual-Max 3D wirelength model is introduced to capture per-tier half-perimeter wirelength (HPWL). In addition, a terminal-aware cutsize penalty selectively suppresses cross-die nets in HBT-congested regions, and a local balance constraint enforces grid-cell density equilibrium across tiers. Experimental results on OpenROAD benchmarks show that our method reduces D2D HPWL by 2.0% on average over two min-cut baselines and by 12.1% over the state-of-the-art 3D placer. We open-source our partition code with 3D placement flow to support reproducibility.

cs.ET↗

arXiv2026

A Semismooth Newton Augmented Lagrangian Method for Sparse Spectral Risk Optimization

Empirical risk minimization is a standard and effective paradigm for learning predictive models by minimizing average loss. In high-stakes decision-making, however, an average-loss criterion may underrepresent rare but severe losses. Spectral risk measures (SRMs) provide a principled framework by incorporating weighted order statistics of losses, but the induced nonsmoothness and nonseparability from sorting make the resulting optimization problems challenging. We propose a relative inexact proximal augmented Lagrangian method with a semismooth Newton subproblem solver for solving SRM-based optimization problems. Exploiting a dual reformulation and properties of the Moreau envelope, we reduce the subproblems to structured dual-variable formulations, significantly simplifying computation. We provide explicit generalized Jacobian characterizations and tailor the pool adjacent violators algorithm for their efficient evaluation. Numerical results on synthetic and real-data instances show that the proposed method attains lower running times than the tested ADMM baseline while producing comparable stationarity residuals and sparse solutions.

math.OC↗

arXiv2026

Benign Landscape of Quadratic Programs with Orthogonality Constraints and Its Application to Heteroscedastic Probabilistic PCA

In this work, we study the optimization landscape of homogeneous quadratic programs with orthogonality constraints (QPOC) and apply the resulting theory to heteroscedastic probabilistic PCA (HePPCA). For QPOC, we establish a complete characterization of the benign optimization landscape by showing that every critical point is either a global maximizer or a strict saddle point. Our analysis builds on a closed-form characterization of the critical point set, from which we derive a necessary and sufficient condition for global optimality and show that every non-optimal critical point has a direction of positive curvature. As an application, we show that the population version of HePPCA is a special instance of QPOC and therefore has a benign optimization landscape; moreover, it satisfies local geodesic strong concavity near every global maximizer. We furthermore prove that, when the sample size is sufficiently large, the sample version of HePPCA inherits these favorable properties with high probability. Together with existing theory on avoiding strict saddle points, our results provide a theoretical justification for the observed local linear convergence of retraction-based optimization methods to global solutions for both QPOC and HePPCA. Finally, we present numerical experiments to corroborate our theoretical results.

math.OC↗

arXiv2026

Solving Chance Constrained Programs via a Penalty based Difference of Convex Approach

We develop two penalty based difference of convex (DC) algorithms for solving chance constrained programs. First, leveraging a rank-based DC decomposition of the chance constraint, we propose a proximal penalty based DC algorithm in the primal space that does not require a feasible initialization. Second, to improve numerical stability in the general nonlinear settings, we derive an equivalent lifted formulation with complementary constraints and show that, after minimizing primal variables, the penalized lifted problem admits a tractable DC structure in the dual space over a simple polyhedron. We then develop a penalty based DC algorithm in the lifted space with a finite termination guarantee. We establish exact penalty and stationarity guarantees under mild constraint qualifications and identify the relationship of the local minimizers between the two formulations. Numerical experiments demonstrate the efficiency and effectiveness of our proposed methods compared with state-of-the-art benchmarks.

math.OC↗

arXiv2025

Adaptive Algorithms for Nonconvex Bilevel Optimization under P{\L} Conditions

Existing methods for nonconvex bilevel optimization (NBO) require prior knowledge of first- and second-order problem-specific parameters (e.g., Lipschitz constants and the Polyak-{\L}ojasiewicz (P{\L}) parameters) to set step sizes, a requirement that poses practical limitations when such parameters are unknown or computationally expensive. We introduce the Adaptive Fully First-order Bilevel Approximation (AF${}^2$BA) algorithm and its accelerated variant, A${}^2$F${}^2$BA, for solving NBO problems under the P{\L} conditions. To our knowledge, these are the first methods to employ fully adaptive step size strategies, eliminating the need for any problem-specific parameters in NBO. We prove that both algorithms achieve $\mathcal{O}(1/\epsilon^2)$ iteration complexity for finding an $\epsilon$-stationary point, matching the iteration complexity of existing well-tuned methods. Furthermore, we show that A${}^2$F${}^2$BA enjoys a near-optimal first-order oracle complexity of $\tilde{\mathcal{O}}(1/\epsilon^2)$, matching the oracle complexity of existing well-tuned methods, and aligning with the complexity of gradient descent for smooth nonconvex single-level optimization when ignoring the logarithmic factors.

math.OC↗

arXiv2025

An Alternating Direction Method of Multipliers for Utility-based Shortfall Risk Portfolio Optimization

Utility-based shortfall risk (UBSR), a convex risk measure sensitive to tail losses, has gained popularity in recent years. However, research on computational methods for UBSR optimization remains relatively scarce. In this paper, we propose a fast and scalable algorithm for the UBSR-based portfolio optimization problem. Leveraging the Sample Average Approximation (SAA) framework, we reformulate the problem as a block-separable convex program and solve it efficiently via the alternating direction method of multipliers (ADMM). In the high-dimensional setting, a key challenge arises in one of the subproblems -- a projection onto a nonlinear feasibility set defined by the shortfall-risk constraint. We propose two semismooth Newton algorithms to solve this projection subproblem. The first algorithm directly applies a semismooth Newton iteration to the Karush-Kuhn-Tucker (KKT) system of the projection problem. The second algorithm employs an implicit function transformation of semismooth functions to reduce the problem to a univariate equation involving the Lagrange multiplier and achieves global superlinear convergence with enhanced numerical stability under mild regularity conditions. Theoretical convergence guarantees of the proposed algorithms are established, and numerical experiments demonstrate a substantial speedup over state-of-the-art solvers, particularly in high-dimensional regimes.

math.OC↗

arXiv2025

A Complete Loss Landscape Analysis of Regularized Deep Matrix Factorization

Despite its wide range of applications across various domains, the optimization foundations of deep matrix factorization (DMF) remain largely open. In this work, we aim to fill this gap by conducting a comprehensive study of the loss landscape of the regularized DMF problem. Toward this goal, we first provide a closed-form characterization of all critical points of the problem. Building on this, we establish precise conditions under which a critical point is a local minimizer, a global minimizer, a strict saddle point, or a non-strict saddle point. Leveraging these results, we derive a necessary and sufficient condition under which every critical point is either a local minimizer or a strict saddle point. This provides insights into why gradient-based methods almost always converge to a local minimizer of the regularized DMF problem. Finally, we conduct numerical experiments to visualize its loss landscape to support our theory.

math.OC↗

arXiv2025

An Adaptive Algorithm for Bilevel Optimization on Riemannian Manifolds

Existing methods for solving Riemannian bilevel optimization (RBO) problems require prior knowledge of the problem's first- and second-order information and curvature parameter of the Riemannian manifold to determine step sizes, which poses practical limitations when these parameters are unknown or computationally infeasible to obtain. In this paper, we introduce the Adaptive Riemannian Hypergradient Descent (AdaRHD) algorithm for solving RBO problems. To our knowledge, AdaRHD is the first method to incorporate a fully adaptive step size strategy that eliminates the need for problem-specific parameters in RBO. We prove that AdaRHD achieves an $\mathcal{O}(1/\epsilon)$ iteration complexity for finding an $\epsilon$-stationary point, thus matching the complexity of existing non-adaptive methods. Furthermore, we demonstrate that substituting exponential mappings with retraction mappings maintains the same complexity bound. Experiments demonstrate that AdaRHD achieves comparable performance to existing non-adaptive approaches while exhibiting greater robustness.

math.OC↗

arXiv2025

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks

The optimization foundations of deep linear networks have recently received significant attention. However, due to their inherent non-convexity and hierarchical structure, analyzing the loss functions of deep linear networks remains a challenging task. In this work, we study the local geometry of the regularized squared loss of deep linear networks around each critical point. Specifically, we obtain a closed-form characterization of the critical point set building on existing results and establish an error bound for the regularized loss under mild conditions on network width and regularization parameters. Notably, this error bound quantifies the distance from a point to the critical point set in terms of the current gradient norm, which can be used to derive linear convergence of first-order methods. To support our theoretical findings, we conduct numerical experiments and demonstrate that gradient descent converges linearly to a critical point when optimizing the regularized loss of deep linear networks.

math.OC↗

arXiv2024

Linear Convergence of the Proximal Gradient Method for Composite Optimization Under the Polyak-{\L}ojasiewicz Inequality and Its Variant

We study the linear convergence rates of the proximal gradient method for composite functions satisfying two classes of Polyak-{\L}ojasiewicz (PL) inequality: the PL inequality, the variant of PL inequality defined by the proximal map-based residual. Using the performance estimation problem, we either provide new explicit linear convergence rates or improve existing complexity bounds for minimizing composite functions under the two classes of PL inequality. Finally, we illustrate numerically the effects of our theoretical results.

math.OC↗

arXiv2024

Accelerated Gradient Descent by Concatenation of Stepsize Schedules

This work considers stepsize schedules for gradient descent on smooth convex objectives. We extend the existing literature and propose a unified technique for constructing stepsizes with analytic bounds for an arbitrary number of iterations. This technique constructs new stepsize schedules by concatenating two stepsize schedules with fewer steps. Using this approach, we introduce two new families of stepsize schedules, achieving a convergence rate of $O(n^{-\log_2(\sqrt 2+1)})$ with state-of-the-art constants for the objective value and gradient norm of the last iterate, respectively. Furthermore, our analytically derived stepsize schedules either match or surpass the existing best numerically computed stepsize schedules.

math.OC↗

arXiv2024

Value-Function Root-Finding Algorithms for Composite Convex Simple Bilevel Optimization

This paper considers composite convex simple bilevel optimization (SBO), which minimizes a composite convex function over the solution set of another composite convex minimization problem. We use a scalar value-function reformulation under which the bilevel optimal value is characterized as the leftmost root of a scalar equation. To evaluate this value function without level-set proximal access, we construct a first-order oracle based on Lagrangian duality. The resulting feasibility and value-error bounds, together with multiplier information, enable a bisection method and a safeguarded Newton-type method to obtain an $(\epsilon,\epsilon)$-solution. Under the stated assumptions and for fixed problem data and initialization tolerance, both methods achieve an operation complexity of $\widetilde{\mathcal{O}}(\epsilon^{-1/2})$, matching, up to logarithmic factors, the known near-optimal dependence for smooth convex SBO and the accelerated first-order rate for unconstrained composite convex optimization.

math.OC↗

arXiv2024

SOREL: A Stochastic Algorithm for Spectral Risks Minimization

The spectral risk has wide applications in machine learning, especially in real-world decision-making, where people are not only concerned with models' average performance. By assigning different weights to the losses of different sample points, rather than the same weights as in the empirical risk, it allows the model's performance to lie between the average performance and the worst-case performance. In this paper, we propose SOREL, the first stochastic gradient-based algorithm with convergence guarantees for the spectral risk minimization. Previous algorithms often consider adding a strongly concave function to smooth the spectral risk, thus lacking convergence guarantees for the original spectral risk. We theoretically prove that our algorithm achieves a near-optimal rate of $\widetilde{O}(1/\sqrtε)$ in terms of $ε$. Experiments on real datasets show that our algorithm outperforms existing algorithms in most cases, both in terms of runtime and sample complexity.

math.OC↗

arXiv2024

New notions of simultaneous diagonalizability of quadratic forms with applications to QCQPs

A set of quadratic forms is simultaneously diagonalizable via congruence (SDC) if there exists a basis under which each of the quadratic forms is diagonal. This property appears naturally when analyzing quadratically constrained quadratic programs (QCQPs) and has important implications in this context. This paper extends the reach of the SDC property by studying two new related but weaker notions of simultaneous diagonalizability. Specifically, we say that a set of quadratic forms is almost SDC (ASDC) if it is the limit of SDC sets and d-restricted SDC (d-RSDC) if it is the restriction of an SDC set in up to d-many additional dimensions. Our main contributions are a complete characterization of the ASDC pairs and the nonsingular ASDC triples, as well as a sufficient condition for the 1-RSDC property for pairs of quadratic forms. Surprisingly, we show that every singular pair is ASDC and that almost every pair is 1-RSDC. We accompany our theoretical results with preliminary numerical experiments applying the RSDC property to QCQPs with a single quadratic constraint.

math.OC↗

arXiv2024

Riemannian Trust Region Methods for SC$^1$ Minimization

Manifold optimization has recently gained significant attention due to its wide range of applications in various areas. This paper introduces the first Riemannian trust region method for minimizing an SC$^1$ function, which is a differentiable function that has a semismooth gradient vector field, on manifolds with convergence guarantee. We provide proof of both global and local convergence results, along with demonstrating the local superlinear convergence rate of our proposed method. As an application and to demonstrate our motivation, we utilize our trust region method as a subproblem solver within an augmented Lagrangian method for minimizing nonsmooth nonconvex functions over manifolds. This represents the first approach that fully explores the second-order information of the subproblem in the context of augmented Lagrangian methods on manifolds. Numerical experiments confirm that our method outperforms existing methods.

math.OC↗

arXiv2024

Decision Making under Cumulative Prospect Theory: An Alternating Direction Method of Multipliers

This paper proposes a novel numerical method for solving the problem of decision making under cumulative prospect theory (CPT), where the goal is to maximize utility subject to practical constraints, assuming only finite realizations of the associated distribution are available. Existing methods for CPT optimization rely on particular assumptions that may not hold in practice. To overcome this limitation, we present the first numerical method with a theoretical guarantee for solving CPT optimization using an alternating direction method of multipliers (ADMM). One of its subproblems involves optimization with the CPT utility subject to a chain constraint, which presents a significant challenge. To address this, we develop two methods for solving this subproblem. The first method uses dynamic programming, while the second method is a modified version of the pooling-adjacent-violators algorithm that incorporates the CPT utility function. Moreover, we prove the theoretical convergence of our proposed ADMM method and the two subproblem-solving methods. Finally, we conduct numerical experiments to validate our proposed approach and demonstrate how CPT's parameters influence investor behavior using real-world data.

math.OC↗

arXiv2024

A Riemannian Proximal Newton Method

In recent years, the proximal gradient method and its variants have been generalized to Riemannian manifolds for solving optimization problems with an additively separable structure, i.e., $f + h$, where $f$ is continuously differentiable, and $h$ may be nonsmooth but convex with computationally reasonable proximal mapping. In this paper, we generalize the proximal Newton method to embedded submanifolds for solving the type of problem with $h(x) = μ\|x\|_1$. The generalization relies on the Weingarten and semismooth analysis. It is shown that the Riemannian proximal Newton method has a local quadratic convergence rate under certain reasonable assumptions. Moreover, a hybrid version is given by concatenating a Riemannian proximal gradient method and the Riemannian proximal Newton method. It is shown that if the switch parameter is chosen appropriately, then the hybrid method converges globally and also has a local quadratic convergence rate. Numerical experiments on random and synthetic data are used to demonstrate the performance of the proposed methods.

math.OC↗