SearcharxivSearch

arXiv subjects

Ya-Xiang Yuan

Publications and source records attributed to Ya-Xiang Yuan.

9 recordsLinked to original sources

The Sharp Worst-Case Asymptotic Rate of the Barzilai--Borwein Method in $\mathbb R^d$ and Hilbert Spaces

We establish sharp asymptotic rates for the two Barzilai--Borwein (BB) rules on uniformly positive quadratics and local nonlinear problems. In finite dimensions, for either fixed rule and an arbitrary positive first step, the gradient root factor is bounded by $(b_0-a_0)/(b_0+a_0)$, where $[a_0,b_0]$ is the initially active spectral interval. Hence the worst trajectory factor is $c_H=(κ(H)-1)/(κ(H)+1)$. When $H$ has at least two distinct eigenvalues, matched initialization and a balanced endpoint trajectory attain this value. Under matched initialization, the same constant is the optimal uniform-envelope threshold. For bounded, self-adjoint, uniformly positive operators on Hilbert space, scalar spectral measures yield the corresponding active-support bound and optimal matched uniform-envelope threshold, including continuous endpoint spectrum. Finally, if the gradient is strictly Fréchet differentiable at a stationary point and its derivative is self-adjoint and uniformly positive, every $γ\in(c_*,1)$, where $c_*=(κ(A_*)-1)/(κ(A_*)+1)$, is a uniform local envelope rate for either pure BB rule. Every well-defined trajectory converging to the stationary point has error and gradient root factors at most $c_*$ and objective-gap root factor at most $c_*^2$. Over the class of objectives with prescribed distinct derivative endpoints $m_*<M_*$, matched endpoint trajectories for quadratic and $C^\infty$ genuinely nonquadratic examples in $\mathbb R^2$ attain these factors.

math.NA

BUP-TR: Bayesian Underdetermined Projection Trust-Region Methods for Derivative-Free Optimization

Underdetermined quadratic interpolation is a central model-construction tool in model-based derivative-free trust-region methods: it limits sampling costs but leaves an affine family of interpolating quadratics. Classical solvers select one element of this family by prescribing a fixed norm or model-change measure, such as the least-Frobenius-change Hessian update in Powell-type methods. We introduce BUP-TR (Bayesian Underdetermined Projection Trust-Region), which instead completes the model by projecting a prior quadratic onto the affine interpolation set in the precision norm supplied by the prior. The same precision matrix defines a spectral geometry certificate, MAP-poisedness, and a repair mechanism for interpolation sets. Under standard smoothness assumptions, uniform precision bounds, MAP-poisedness, and a trust-region-scale prior-accuracy condition, the hard-MAP models are fully linear. Consequently, BUP-TR attains global first-order convergence and O(epsilon^{-2}) evaluation complexity, with geometry-repair evaluations included. A NEWUOA-style implementation, BUP-NEWUOA, improves fixed-budget performance on the reported benchmark suite at moderate and stringent accuracy targets while retaining the computational structure of a Powell-type trust-region method.

math.OC

MATRO: Metric-Aware Trust-Region Optimization with Fully Quadratic Models

Model-based derivative-free trust-region methods build local interpolation models and restrict trial steps to regions where those models are reliable. This paper studies the shape of that region. When an objective is poorly scaled or locally anisotropic, a Euclidean ball can be governed by the steepest local direction and can restrict progress along directions of slow variation. We propose MATRO (Metric-Aware Trust-Region Optimization), a fully quadratic interpolation framework in which the trust region is the ellipsoid s^T M_k s <= Delta_k^2. For any positive definite metric M_k, the induced variable y = M_k^{1/2}s converts the ellipsoidal subproblem into a standard Euclidean trust-region subproblem, so model decrease, ratio tests, radius updates, poisedness, and fully quadratic error bounds can be stated in induced coordinates under a uniform metric contract. The metric is selected from the interpolation Hessian: positive definite quadratics yield a unique volume-normalized curvature metric that isotropizes the induced Hessian and gives a truncated Newton step, while indefinite fitted Hessians motivate an absolute-curvature metric that balances curvature magnitudes without changing curvature signs. Under the standard fully quadratic assumptions and the metric contract, MATRO retains the first-order evaluation-complexity order O(n^2 epsilon^{-2}). Experiments on More-Wild benchmarks, controlled anisotropy tests, and two-dimensional trajectories show that curvature-shaped regions are most effective when the interpolation Hessian captures stable local anisotropy, while dense linear algebra is most visible at loose accuracies or on inexpensive analytic tests.

math.OC

Distributed Gradient-Regularized Newton Method: Scheduled Consensus and O(epsilon^{-1}) Global Iteration Complexity

We propose DisGrem, a fully decentralized second-order method for convex consensus optimization over networks. Each agent solves a local Newton system with vanishing gradient-norm regularization and an eigenvalue-shift stabilizer, communicating through a two-stage gossip-mixing mechanism. We introduce a reference-step framework that reduces the network-wide update to an inexact centralized regularized Newton step, replacing the static Hessian-heterogeneity assumptions of prior work with an increment-based dispersion analysis that imposes no irreducible accuracy floor. Under a bounded-iterates assumption, after a burn-in phase whose order is controlled by the scheduled consensus accuracy, the post-burn-in phase achieves an O(epsilon^{-1}) iteration complexity for driving the gradient norm below epsilon, matching the centralized regularized Newton rate without line search or stepsize tuning. For a logarithmic schedule with p >= 3, the total iteration complexity remains O(epsilon^{-1}). For a fixed connected network, this yields O(epsilon^{-1} log(1/epsilon)) neighbor communication rounds, with explicit spectral-gap dependence O((1-rho)^{-1} epsilon^{-1} log(1/epsilon)) as rho approaches 1. Under strong convexity and a relative tracking-accuracy condition, we further establish conditional local superlinear convergence of order 3/2. In our nine-problem benchmark suite, the DisGrem family attains relF <= 10^{-6} on every test instance, while the tested baselines stagnate or diverge on at least one problem.

math.OC

New gradient methods with 3 dimensional quadratic termination

A new stepsize for gradient method is proposed. Combining it with the exact line search stepsizes, the gradient method achieves the optimal solution in 5 steps for 3 dimensional quadratic function minimization problem. The new stepsize is plugged in the cyclic stepsize update strategy, and a new gradient method is proposed. By applying the quadratic interpolation for Cauchy approximation, the proposed gradient method is extended to solve general unconstrained problem. With the improved GLL line search, the global convergence of the proposed method is proved. Furthermore, its sublinear convergence rate for convex problems and R-linear convergence rate for problems with quadratic functional growth property are analyzed. Numerical results show that our proposed algorithm enjoys good performances in terms of computational cost, and line search requires very few trial stepsizes.

math.OC

Finite convergence of Moment-SOS relaxations with non-real radical ideals

We consider the linear conic optimization problem with the cone of nonnegative polynomials. Its dual optimization problem is the generalized moment problem. Moment-SOS relaxations are powerful for solving them. This paper studies finite convergence of the Moment-SOS hierarchy when the constraining set is defined by equations whose ideal may not be real radical. Under the archimedeanness, we show that the Moment-SOS hierarchy has finite convergence if some classical optimality conditions hold at every minimizer of the optimal nonnegative polynomial for the linear conic optimization problem. When the archimedeanness fails (this is the case for unbounded sets), we propose a homogenized Moment-SOS hierarchy and prove its finite convergence under similar assumptions. Furthermore, we also prove the finite convergence of the Moment-SOS hierarchy with denominators. In particular, this paper resolves a conjecture posed in the earlier work.

math.OC

Homogenization for polynomial optimization with unbounded sets

This paper considers polynomial optimization with unbounded sets. We give a homogenization formulation and propose a hierarchy of Moment-SOS relaxations to solve it. Under the assumptions that the feasible set is closed at infinity and the ideal of homogenized equality constraining polynomials is real radical, we show that this hierarchy of Moment-SOS relaxations has finite convergence, if some optimality conditions (i.e., the linear independence constraint qualification, strict complementarity and second order sufficiency conditions) hold at every minimizer, including the one at infinity. Moreover, we prove extended versions of Putinar-Vasilescu type Positivstellensatz for polynomials that are nonnegative on unbounded sets. The classical Moment-SOS hierarchy with denominators is also studied. In particular, we give a positive answer to a conjecture of Mai, Lasserre and Magron in their recent work.

math.OC

Generalized truncated moment problems with unbounded sets

This paper studies generalized truncated moment problems with unbounded sets. First, we study geometric properties of the truncated moment cone and its dual cone of nonnegative polynomials. By the technique of homogenization, we give a convergent hierarchy of Moment-SOS relaxations for approximating these cones. With them, we give a Moment-SOS method for solving generalized truncated moment problems with unbounded sets. Finitely atomic representing measures, or certificates for their nonexistence, can be obtained by the proposed method. Numerical experiments and applications are also given.

math.NA

Algorithm XXX: SC-SR1: Matlab software for solving shape-changing L-SR1 trust-region subproblems

We present a MATLAB implementation of the symmetric rank-one (SC-SR1) method that solves trust-region. subproblems when a limited-memory symmetric rank-one (L-SR1) matrix is used in place of the true Hessian matrix, which can be used for large-scale optimization. The method takes advantage of two shape-changing norms[Burdakov and Yuan 2002; Burdakov et al. 2017] to decompose the trust-region subproblem into two separate problems. Using one of the proposed norms, the resulting subproblems have closed-form solutions. Meanwhile, using the other proposed norm, one of the resulting subproblems has a closed-form solution while the other is easily solvable using techniques that exploit the structure of L-SR1 matrices. Numerical results suggest that the SC-SR1 method is able to solve trust-region subproblems to high accuracy even in the so-called "hard case". When integrated into a trust-region algorithm, extensive numerical experiments suggest that the proposed algorithms perform well, when compared with widely used solvers, such as truncated CG.

math.OC