arXiv subjects

Nuozhou Wang

Publications and source records attributed to Nuozhou Wang.

5 recordsLinked to original sources

arXiv2026

Complexity Analysis of Convex Majorization Schemes for Nonconvex Constrained Optimization

In this paper, we introduce and study various algorithms for solving nonconvex minimization with inequality constraints, based on the construction of convex surrogate envelopes that majorize the objective and the constraints. In the case where the objective and constraint functions are gradient Hölderian continuous, the surrogate functions can be readily constructed and the solution method can be efficiently implemented. The surrogate envelopes are extended to the settings where the second-order information is available, and the convex subproblems are further represented by Dikin ellipsoids using the self-concordance of the convex surrogate constraints. Iteration complexities have been developed for both convex and nonconvex optimization models. The numerical results show promising potential of the proposed approaches.

math.OC↗

arXiv2025

An Accelerated Variance Reduced Extra-Point Approach to Finite-Sum VI and Optimization

In this paper, we develop stochastic variance reduced algorithms for solving a class of finite-sum hemivariational inequality (HVI) problem. In this HVI problem, the associated function is assumed to be differentiable, and both the vector mapping and the function are of finite-sum structure. We propose two algorithms to solve the cases when the vector mapping is either merely monotone or strongly monotone, while the function is assumed to be convex. We show how to apply variance reduction in the proposed algorithms when such an HVI problem has a finite-sum structure, and the resulting accelerated gradient complexities can match the best bound established for finite-sum VI problem, as well as the bound given by the direct Katyusha for finite-sum optimization respectively, in terms of the corresponding parameters such as (gradient) Lipschitz constants and the sizes of the finite-sums. We demonstrate the application of our algorithms through solving a finite-sum constrained finite-sum optimization problem and provide preliminary numerical results.

math.OC↗

arXiv2024

Efficient First Order Method for Saddle Point Problems with Higher Order Smoothness

This paper studies the complexity of finding approximate stationary points for the smooth nonconvex-strongly-concave (NC-SC) saddle point problem: $\min_x\max_yf(x,y)$. Under the standard first-order smoothness conditions where $f$ is $\ell$-smooth in both arguments and $μ_y$-strongly concave in $y$, existing literature shows that the optimal complexity for first-order methods to obtain an $ε$-stationary point is $\tilde{O}\big(\sqrt{κ_y}\ellε^{-2}\big)$, where $κ_y=\ell/μ_y$ is the condition number. However, when $Φ(x):=\max_y f(x,y)$ has $L_2$-Lipschitz continuous Hessian in addition, we derive a first-order algorithm with an $\tilde{O}\big(\sqrt{κ_y}\ell^{1/2}L_2^{1/4}ε^{-7/4}\big)$ complexity by designing an accelerated proximal point algorithm enhanced with the "Convex Until Proven Guilty" technique. Moreover, an improved $Ω\big(\sqrt{κ_y}\ell^{3/7}L_2^{2/7}ε^{-12/7}\big)$ lower bound for first-order method is also derived for sufficiently small $ε$. As a result, given the second-order smoothness of the problem, the complexity of our method improves the state-of-the-art result by a factor of $\tilde{O}\big(\big(\frac{\ell^2}{L_2ε}\big)^{1/4}\big)$, while almost matching the lower bound except for a small $\tilde{O}\big(\big(\frac{\ell^2}{L_2ε}\big)^{1/28}\big)$ factor.

math.OC↗

arXiv2024

Novel Optimization Techniques for Parameter Estimation

In this paper, we introduce a new optimization algorithm that is well suited for solving parameter estimation problems. We call our new method cubic regularized Newton with affine scaling (CRNAS). In contrast to so-called first-order methods which rely solely on the gradient of the objective function, our method utilizes the Hessian of the objective. As a result it is able to focus on points satisfying the second-order optimality conditions, as opposed to first-order methods that simply converge to critical points. This is an important feature in parameter estimation problems where the objective function is often non-convex and as a result there can be many critical points making it is near impossible to identify the global minimum. An important feature of parameter estimation in mathematical models of biological systems is that the parameters are constrained by either physical constraints or prior knowledge. We use an affine scaling approach to handle a wide class of constraints. We establish that CRNAS identifies a point satisfying $ε$-approximate second-order optimality conditions within $O(ε^{-3/2})$ iterations. Finally, we compare CRNAS with MATLAB's optimization solver fmincon on three different test problems. These test problems all feature mixtures of heterogeneous populations, a problem setting that CRNAS is particularly well-suited for. Our numerical simulations show CRNAS has favorable performance, performing comparable if not better than fmincon in accuracy and computational cost for most of our examples.

math.OC↗

arXiv2022

A Gradient Complexity Analysis for Minimizing the Sum of Strongly Convex Functions with Varying Condition Numbers

A popular approach to minimize a finite-sum of convex functions is stochastic gradient descent (SGD) and its variants. Fundamental research questions associated with SGD include: (i) To find a lower bound on the number of times that the gradient oracle of each individual function must be assessed in order to find an $ε$-minimizer of the overall objective; (ii) To design algorithms which guarantee to find an $ε$-minimizer of the overall objective in expectation at no more than a certain number of times (in terms of $1/ε$) that the gradient oracle of each functions needs to be assessed (i.e., upper bound). If these two bounds are at the same order of magnitude, then the algorithms may be called optimal. Most existing results along this line of research typically assume that the functions in the objective share the same condition number. In this paper, the first model we study is the problem of minimizing the sum of finitely many strongly convex functions whose condition numbers are all different. We propose an SGD method for this model and show that it is optimal in gradient computations, up to a logarithmic factor. We then consider a constrained separate block optimization model, and present lower and upper bounds for its gradient computation complexity. Next, we propose to solve the Fenchel dual of the constrained block optimization model via the SGD we introduced earlier, and show that it yields a lower iteration complexity than solving the original model by the ADMM-type approach. Finally, we extend the analysis to the general composite convex optimization model, and obtain gradient-computation complexity results under certain conditions.

math.OC↗