arXiv subjects

Chuwen Zhang

Publications and source records attributed to Chuwen Zhang.

14 recordsLinked to original sources

arXiv2026

Accelerating Trust-Region Methods: An Attempt to Balance Global and Local Efficiency

Balancing global efficiency and local convergence remains a central challenge in second-order methods for unconstrained convex optimization problems. Newton's method enjoys fast local convergence but may diverge when initialized far from the solution. In contrast, accelerated second-order methods provide global guarantees but typically suffer from slower local convergence. This raises the fundamental question of to what extent global acceleration can be achieved without sacrificing strong local convergence. In this paper, we tackle this challenge by proposing the first accelerated trust-region-type methods and leveraging their inherent primal-dual information. Our primary contribution is the Accelerated Trust-Region method with Local Detection, which utilizes the Lagrange multiplier to detect local regions and achieves a global oracle complexity of \tilde{O}(ε^{-1/3}), while maintaining quadratic local convergence. We further examine the trade-off that arises when global convergence is pushed to the limit. Specifically, we introduce the Accelerated Trust-Region Extragradient Method, which achieves a global oracle complexity of \tilde{O}(ε^{-2/7}) but no longer enjoys quadratic local convergence. This reveals a phase-transition-like phenomenon in accelerated trust-region-type methods: quadratic local convergence is preserved under moderate global acceleration, but it breaks down when pursuing extreme global efficiency. Numerical experiments are consistent with the theoretical predictions and illustrate the global-local trade-off.

math.OC↗

arXiv2026

Homogeneous second-order descent framework: a fast alternative to Newton-type methods

This paper proposes a homogeneous second-order descent framework (HSODF) for nonconvex and convex optimization based on the generalized homogeneous model (GHM). In comparison to the Newton steps, the GHM can be solved by extremal symmetric eigenvalue procedures and thus grant an advantage in ill-conditioned problems. Moreover, GHM extends the ordinary homogeneous model (OHM) (Zhang et al. 2022) to allow adaptiveness in the construction of the aggregated matrix. Consequently, HSODF is able to recover some well-known second-order methods, such as trust-region methods and gradient regularized methods, while maintaining comparable iteration complexity bounds. We also study two specific realizations of HSODF. One is adaptive HSODM, which has a parameter-free $O(ε^{-3/2})$ global complexity bound for nonconvex second-order Lipschitz continuous objective functions. The other one is homotopy HSODM, which is proven to have a global linear rate of convergence without strong convexity. The efficiency of our approach to ill-conditioned and high-dimensional problems is justified by some preliminary numerical results.

math.OC↗

arXiv2026

A homogeneous second-order descent method for nonconvex optimization

In this paper, we introduce a Homogeneous Second-Order Descent Method (HSODM) using the homogenized quadratic approximation to the original function. The merit of homogenization is that only the leftmost eigenvector of a gradient-Hessian integrated matrix is computed at each iteration. Therefore, the algorithm is a single-loop method that does not need to switch to other sophisticated algorithms and is easy to implement. We show that HSODM has a global convergence rate of $O(ε^{-3/2})$ to find an $ε$-approximate second-order stationary point, and has a local quadratic convergence rate under the standard assumptions. The numerical results demonstrate the advantage of the proposed method over other second-order methods.

math.OC↗

arXiv2026

Rationalizing collective revealed preferences with an application in fair resource allocation

This paper presents a revealed preference approach for rationalizing collective consumption behavior. We introduce the Constructive Rationalization Method (CRM), which approximates the real market via a surrogate market of artificial consumers, called androids, with easy-to-compute demand functions. CRM uses observed aggregate demand and adds artificial consumers on the fly, while redistributing wealth under an empirical risk minimization principle. Unlike classical revealed preference approaches, CRM provides guarantees on the generalization risk for learning the aggregate demand function, while respecting the privacy of the underlying consumers in the real market. As an application, CRM can be used to provide reliable predictions for collective consumption behavior. Specifically, we show how to apply CRM to approximate allocations that are proportionally fair without requiring the knowledge of individual utilities.

cs.GT↗

arXiv2026

The Dynamic and Endogenous Behavior of Re-Offense Risk: An Agent-Based Simulation Study of Treatment Allocation in Incarceration Diversion Programs

Incarceration-diversion treatment programs aim to improve societal reintegration and reduce recidivism, but limited capacity forces policymakers to make prioritization decisions that often rely on risk assessment tools. While predictive, these tools typically treat risk as a static, individual attribute, which overlooks how risk evolves over time and how treatment decisions shape outcomes through social interactions. In this paper, we develop a new framework that models reoffending risk as a human-system interaction, linking individual behavior with system-level dynamics and endogenous community feedback. Using an agent-based simulation calibrated to U.S. probation data, we evaluate treatment allocation policies under different capacity constraints and incarceration settings. Our results show that no single prioritization policy dominates. Instead, policy effectiveness depends on temporal windows and system parameters: prioritizing low-risk individuals performs better when long-term trajectories matter, while prioritizing high-risk individuals becomes more effective in the short term or when incarceration leads to shorter monitoring periods. These findings highlight the need to evaluate risk-based decision systems as sociotechnical systems with long-term accountability, rather than as isolated predictive tools.

cs.CY↗

arXiv2026

Beyond Nonconvexity: A Universal Trust-Region Method with New Analyses

The trust-region (TR) method is renowned historically for its robustness in nonconvex problems and extraordinary numerical performance, but the study of its performance in convex optimization is somehow limited. This paper complements the existing literature by presenting a universal trust-region method that simultaneously incorporates the quadratic regularization and ball constraint. In particular, we introduce a novel descent property tailored for trust-region-type algorithms, enabling us to unify and streamline the analysis for both convex and nonconvex optimization. Our method exhibits an iteration complexity of $\tilde O(ε^{-3/2})$ to find an $ε$-approximate second-order stationary point for nonconvex optimization. Meanwhile, the analysis reveals that the universal method attains an $O(ε^{-1/2})$ complexity bound for convex optimization. Finally, we develop an adaptive universal method to address practical implementations. The numerical results show the effectiveness of our method in both nonconvex and convex problems.

math.OC↗

arXiv2025

History-Aware Adaptive High-Order Tensor Regularization

In this paper, we develop a new adaptive regularization method for minimizing a composite function, which is the sum of a $p$th-order ($p \ge 1$) Lipschitz continuous function and a simple, convex, and possibly nonsmooth function. We use a history of local Lipschitz estimates to adaptively select the current regularization parameter, an approach we shall term the {\it history-aware adaptive regularization method}. We explore how the selection of an appropriate volume of historical information affects both the theoretical and practical performance. By using all the historical information, our method matches the complexity guarantees of the standard $p$th-order tensor methods that require a known Lipschitz constant, for both convex and nonconvex objectives. In the nonconvex case, the number of iterations required to find an $(ε_g,ε_H)$-approximate second-order stationary point is bounded by $\mathcal{O}(\max\{ε_g^{-(p+1)/p}, ε_H^{-(p+1)/(p-1)}\})$. For convex functions, we establish an $\mathcal{O}(ε^{-1/p})$ iteration complexity for finding an $ε$-approximate optimal point and further propose an accelerated variant attaining an iteration complexity of $\mathcal{O}(ε^{-1/(p+1)})$. For practical consideration, we propose several variants of this method with only part of historical information. We introduce cyclic and sliding-window strategies for choosing historical Lipschitz estimates, which mitigate the limitation of overly conservative updates. As long as a rough upper bound of the Lipschitz constant is known, these two variants achieve the same iteration complexity guarantees in terms of the input accuracy as the method using full historical information. Finally, extensive numerical experiments are conducted to demonstrate the effectiveness of our adaptive approach.

math.OC↗

arXiv2025

The Second-Order Tâtonnement: Decentralized Interior-Point Methods for Market Equilibrium

The tâtonnement process and Smale's process are two classical approaches to compute market equilibrium in exchange economies. While the tâtonnement process can be seen as a first-order method, Smale's process, being second-order, is less popular due to its reliance on additional information from the players and expensive Newton steps. In this paper, we study Fisher exchange market for a broad class of utility functions, where we show that all high-order information required by Smale's process is readily available from players' best responses. Motivated by this observation, we develop two second-order tâtonnement processes, constructed as decentralized interior-point methods, which are traditionally known to work in a centralized manner. The methods here bear the name "tâtonnement", since, in spirit, they demand no more information than the classical tâtonnement process. To address the Newton systems involved, we introduce an explicitly invertible approximation with high-probability guarantees and a scaling matrix that optimally minimizes the condition number, both of which rely solely on best responses as the methods themselves. Using these tools, the first second-order tâtonnement process has O(log(1/$ε$))complexity rate. Under mild conditions, the other method achieves a non-asymptotic superlinear convergence rate. Preliminary experiments are presented to justify the capability of the proposed methods for large-scale problems. Extensions of our approach are also discussed.

math.OC↗

arXiv2025

Trust Region Methods For Nonconvex Stochastic Optimization Beyond Lipschitz Smoothness

In many important machine learning applications, the standard assumption of having a globally Lipschitz continuous gradient may fail to hold. This paper delves into a more general $(L_0, L_1)$-smoothness setting, which gains particular significance within the realms of deep neural networks and distributionally robust optimization (DRO). We demonstrate the significant advantage of trust region methods for stochastic nonconvex optimization under such generalized smoothness assumption. We show that first-order trust region methods can recover the normalized and clipped stochastic gradient as special cases and then provide a unified analysis to show their convergence to first-order stationary conditions. Motivated by the important application of DRO, we propose a generalized high-order smoothness condition, under which second-order trust region methods can achieve a complexity of $\mathcal{O}(ε^{-3.5})$ for convergence to second-order stationary points. By incorporating variance reduction, the second-order trust region method obtains an even better complexity of $\mathcal{O}(ε^{-3})$, matching the optimal bound for standard smooth optimization. To our best knowledge, this is the first work to show convergence beyond the first-order stationary condition for generalized smooth optimization. Preliminary experiments show that our proposed algorithms perform favorably compared with existing methods.

math.OC↗

arXiv2024

A Customized Augmented Lagrangian Method for Block-Structured Integer Programming

Integer programming with block structures has received considerable attention recently and is widely used in many practical applications such as train timetabling and vehicle routing problems. It is known to be NP-hard due to the presence of integer variables. We define a novel augmented Lagrangian function by directly penalizing the inequality constraints and establish the strong duality between the primal problem and the augmented Lagrangian dual problem. Then, a customized augmented Lagrangian method is proposed to address the block-structures. In particular, the minimization of the augmented Lagrangian function is decomposed into multiple subproblems by decoupling the linking constraints and these subproblems can be efficiently solved using the block coordinate descent method. We also establish the convergence property of the proposed method. To make the algorithm more practical, we further introduce several refinement techniques to identify high-quality feasible solutions. Numerical experiments on a few interesting scenarios show that our proposed algorithm often achieves a satisfactory solution and is quite effective.

math.OC↗

arXiv2024

A Homogenization Approach for Gradient-Dominated Stochastic Optimization

Gradient dominance property is a condition weaker than strong convexity, yet sufficiently ensures global convergence even in non-convex optimization. This property finds wide applications in machine learning, reinforcement learning (RL), and operations management. In this paper, we propose the stochastic homogeneous second-order descent method (SHSODM) for stochastic functions enjoying gradient dominance property based on a recently proposed homogenization approach. Theoretically, we provide its sample complexity analysis, and further present an enhanced result by incorporating variance reduction techniques. Our findings show that SHSODM matches the best-known sample complexity achieved by other second-order methods for gradient-dominated stochastic optimization but without cubic regularization. Empirically, since the homogenization approach only relies on solving extremal eigenvector problem at each iteration instead of Newton-type system, our methods gain the advantage of cheaper computational cost and robustness in ill-conditioned problems. Numerical experiments on several RL tasks demonstrate the better performance of SHSODM compared to other off-the-shelf methods.

math.OC↗

arXiv2024

An Enhanced ADMM-based Interior Point Method for Linear and Conic Optimization

The ADMM-based interior point (ABIP, Lin et al. 2021) method is a hybrid algorithm that effectively combines interior point method (IPM) and first-order methods to achieve a performance boost in large-scale linear optimization. Different from traditional IPM that relies on computationally intensive Newton steps, the ABIP method applies the alternating direction method of multipliers (ADMM) to approximately solve the barrier penalized problem. However, similar to other first-order methods, this technique remains sensitive to condition number and inverse precision. In this paper, we provide an enhanced ABIP method with multiple improvements. Firstly, we develop an ABIP method to solve the general linear conic optimization and establish the associated iteration complexity. Secondly, inspired by some existing methods, we develop different implementation strategies for ABIP method, which substantially improve its performance in linear optimization. Finally, we conduct extensive numerical experiments in both synthetic and real-world datasets to demonstrate the empirical advantage of our developments. In particular, the enhanced ABIP method achieves a 5.8x reduction in the geometric mean of run time on $105$ selected LP instances from Netlib, and it exhibits advantages in certain structured problems such as SVM and PageRank. However, the enhanced ABIP method still falls behind commercial solvers in many benchmarks, especially when high accuracy is desired. We posit that it can serve as a complementary tool alongside well-established solvers.

math.OC↗

arXiv2024

cuPDLP-C: A Strengthened Implementation of cuPDLP for Linear Programming by C language

A recent GPU implementation of the Restarted Primal-Dual Hybrid Gradient Method for Linear Programming was proposed in Lu and Yang (2023). Its computational results demonstrate the significant computational advantages of the GPU-based first-order algorithm on certain large-scale problems. The average performance also achieves a level close to commercial solvers for the first time in history. However, due to limitations in experimental hardware and the disadvantage of implementing the algorithm in Julia compared to C language, neither the commercial solver nor cuPDLP reached their maximum efficiency. Therefore, in this report, we have re-implemented and optimized cuPDLP in C language. Utilizing state-of-the-art CPU and GPU hardware, we extensively compare cuPDLP with the best commercial solvers. The experiments further highlight its substantial computational advantages and potential for solving large-scale linear programming problems. We also discuss the profound impact this breakthrough may have on mathematical programming research and the entire operations research community.

math.OC↗

arXiv2023

DRSOM: A Dimension Reduced Second-Order Method

In this paper, we propose a Dimension-Reduced Second-Order Method (DRSOM) for convex and nonconvex (unconstrained) optimization. Under a trust-region-like framework, our method preserves the convergence of the second-order method while using only curvature information in a few directions. Consequently, the computational overhead of our method remains comparable to the first-order such as the gradient descent method. Theoretically, we show that the method has a local quadratic convergence and a global convergence rate of $O(ε^{-3/2})$ to satisfy the first-order and second-order conditions if the subspace satisfies a commonly adopted approximated Hessian assumption. We further show that this assumption can be removed if we perform a corrector step using a Krylov-like method periodically at the end stage of the algorithm. The applicability and performance of DRSOM are exhibited by various computational experiments, including $L_2 - L_p$ minimization, CUTEst problems, and sensor network localization.

math.OC↗