arXiv subjects

Akiko Takeda

Publications and source records attributed to Akiko Takeda.

At least 19 recordsLinked to original sources

arXiv2026

A Splitting Framework for Composite Semimonotone Inclusions

We introduce a general framework for composite inclusion problems with affine constraints, covering both monotone and semimonotone regimes. The central idea is a new interpretation of the constrained inclusion through an operator-vector pair that separates the implicit inclusion from the affine constraint: the former is handled through possibly preconditioned resolvent evaluations, while the latter is handled through an explicit forward step in an auxiliary variable. This yields a single abstract iteration applicable to multioperator inclusions, linearly coupled inclusions, and block-separable inclusions with affine constraints. The freedom in choosing the operator-vector pair enables the systematic construction of problem-adapted splitting algorithms, including new schemes for several important problem classes. Exploiting the orthogonal decomposition induced by the constraint subspace, we develop a unified and streamlined convergence analysis and establish weak and strong convergence guarantees under semimonotonicity assumptions. When specialized to multioperator inclusions, the framework permits general bounded linear operator coefficients, rather than only scalar coefficients, and therefore accommodates preconditioned resolvents. The resulting schemes recover several existing methods while extending them to previously uncovered regimes, and in several important cases, require weaker assumptions and admit provably larger admissible parameter ranges.

math.OC↗

arXiv2026

Adaptive Gradient-Based Methods for a Broader Class of Optimization Problems under Performative Prediction

We study optimization under performative prediction, where deploying a model affects the future data distribution. For this setting, several gradient-based approaches have been proposed. However, they typically assume specific data distributions or loss functions, which limit their practical applicability. To overcome these limitations, we propose a gradient-based optimization method with convergence guarantees under substantially weaker assumptions. Our method explicitly estimates the induced distribution shift through finite differences. It enables higher-dimensional optimization across broader classes of loss functions and data distributions. We also propose a practical variant that reduces the number of samples required. Numerical experiments demonstrate that our proposed algorithms converge faster and more consistently than existing ones.

math.OC↗

arXiv2026

Parameter-Free Cubic-Regularized Newton Method: Sharp Complexity and Generalized Smoothness

We analyze a variant of the cubic-regularized Newton method for nonconvex optimization. This variant is parameter-free in that it requires no prior knowledge of problem-dependent parameters. Under the generalized smoothness condition $\|\nabla^3 f(x)\| \leq L_0 + L_1 \|\nabla f(x)\|$, we derive an oracle complexity bound for finding an $(\varepsilon, \delta)$-second-order stationary point. This assumption is weaker than the generalized smoothness conditions used in existing analyses of second-order methods, while the complexity bound improves upon existing guarantees for parameter-free second-order methods. In particular, when $L_1 = 0$, the bound matches the optimal dependence on $L_0$ as well as on $\varepsilon$, $\delta$, and the initial function value gap, up to additive logarithmic terms. To establish this bound, we derive Taylor-type inequalities and prove their equivalence to the generalized smoothness condition.

math.OC↗

arXiv2026

A General Recipe for Parameter-Free Nonconvex Optimization via Higher-Order Regularization

We develop a systematic framework for constructing parameter-free algorithms for smooth nonconvex optimization. The framework is based on higher-order regularization: each step is computed from a regularized local model whose regularization exponent exceeds the order of the model error. This design makes the resulting method robust to misspecification of the regularization parameter and yields complexity bounds without backtracking or other acceptance tests. We apply the framework to gradient descent, Newton's method, the Gauss--Newton method, stochastic gradient descent, and PAGE. Without prior knowledge of problem-dependent parameters, the resulting algorithms achieve complexity bounds with optimal or best-known dependence on the target accuracy. When the problem-dependent parameters are known up to constant factors, suitable tuning also recovers the optimal or best-known dependence on those parameters.

math.OC↗

arXiv2026

Random-Subspace Frank--Wolfe over Strongly Convex Sets

Frank--Wolfe methods avoid projections, but over curved feasible regions the full-space linear minimization oracle (LMO) can itself become the computational bottleneck. We introduce random-subspace Frank--Wolfe (RSFW), the first Frank--Wolfe framework, to our knowledge, that replaces the ambient LMO by exact LMOs over random low-dimensional affine sections of a general feasible set, while preserving feasibility in the original space. For smooth convex objectives over compact strongly convex feasible sets, we prove a dimension-explicit approximate-oracle inequality and derive the standard $O(1/k)$ open-loop rate, with high-probability and almost-sure counterparts. Under short steps and a gradient lower bound, the same geometric control yields linear convergence, and we extend the sublinear theory to finite-sum stochastic gradients. We also show that random sections can improve the local curvature model controlling short steps: for smooth objectives, the quadratic model along a sampled section is governed by the compressed Hessian, yielding computable $d\times d$ curvature constants for quadratic objectives over balls and ellipsoids. These results provide a geometric theory of oracle-side randomization in projection-free optimization.

math.OC↗

arXiv2026

Randomized Subspace Nesterov Accelerated Gradient

Randomized-subspace methods reduce the cost of first-order optimization by using only low-dimensional projected-gradient information, a feature that is attractive in forward-mode automatic differentiation and communication-limited settings. While Nesterov acceleration is well understood for full-gradient and coordinate-based methods, obtaining accelerated methods for general subspace sketches that use only projected-gradient information and can improve over full-dimensional Nesterov acceleration in oracle complexity is technically nontrivial. We develop randomized-subspace Nesterov accelerated gradient methods for smooth convex and smooth strongly convex optimization under matrix smoothness and generic sketch moment assumptions. The key technical ingredient is a three-sequence formulation tailored to matrix smoothness, which recovers the corresponding classical Nesterov methods in the full-dimensional case. The resulting theory establishes accelerated oracle-complexity guarantees and makes explicit how matrix smoothness and the sketch distribution enter the complexity. It also provides a unified basis for comparing sketch families and identifying when randomized-subspace acceleration improves over full-dimensional Nesterov acceleration in oracle complexity.

math.OC↗

arXiv2026

Practical Regularized Quasi-Newton Methods with Inexact Function Values

Many practical optimization problems involve objective function values that are corrupted by unavoidable numerical errors. In smooth nonconvex optimization, quasi-Newton methods combined with line search are widely used due to their efficiency and scalability. These methods implicitly assume accurate function evaluations and thus may fail to converge in noisy settings. Developing fast and robust quasi-Newton methods for such scenarios is therefore crucial. To address this issue, we propose a noise-tolerant regularized quasi-Newton method equipped with a relaxed Armijo-type line search, designed to remain stable under inaccurate function evaluations. By combining a regularization parameter update rule inspired by Objective-Function-Free Optimization and the AdaGrad-Norm method, we establish a global convergence rate of $\mathcal{O}(1/\varepsilon^2)$ for reaching a first-order stationary point under the assumed error model. We performed extensive experiments on the CUTEst benchmark collection with artificially noisy objective function evaluations, as well as with low-precision floating-point arithmetic (64-, 32-, and 16-bit). The results demonstrate that the proposed method is substantially more robust than several existing methods, while maintaining competitive practical convergence speed and computational cost.

math.OC↗

arXiv2026

Fairness in Robust Unit Commitment Problem Considering Suppression of Renewable Energy

Power company operators make power generation plans one day in advance, in what is known as the Unit Commitment (UC) problem. UC is exposed to uncertainties, such as unknown electricity load and disturbances caused by renewable energy sources, especially PVs. In previous research, we proposed the Renewable Energy Robust Optimization Problem (RE-RP), which solves these uncertainties by considering suppression. In this paper, we propose a new model called RE-RP with fairness (RE-RPfair), which aims to achieve fair allocation among PVs allocation. This model is an expansion of the original RE-RP, and we prove its effectiveness through simulation. To measure the degree of fairness, we use the Gini Index, which is well-known in social science.

math.OC↗

arXiv2026

Convergence Analysis of Randomized Subspace Normalized SGD under Heavy-Tailed Noise

Randomized subspace methods reduce per-iteration cost; however, in nonconvex optimization, most analyses are expectation-based, and high-probability bounds remain scarce even under sub-Gaussian noise. We first prove that randomized subspace SGD (RS-SGD) admits a high-probability convergence bound under sub-Gaussian noise, achieving the same order of oracle complexity as prior in-expectation results. Motivated by the prevalence of heavy-tailed gradients in modern machine learning, we then propose randomized subspace normalized SGD (RS-NSGD), which integrates direction normalization into subspace updates. Assuming the noise has bounded $p$-th moments, we establish both in-expectation and high-probability convergence guarantees, and show that RS-NSGD can achieve better oracle complexity than full-dimensional normalized SGD.

math.OC↗

arXiv2025

Complexity and convergence analysis of a single-loop SDCAM for Lipschitz composite optimization and beyond

We develop and analyze a single-loop algorithm for minimizing the sum of a Lipschitz differentiable function $f$, a prox-friendly proper closed function $g$ (with a closed domain on which $g$ is continuous) and the composition of another prox-friendly proper closed function $h$ (whose domain is closed on which $h$ is continuous) with a continuously differentiable mapping $c$ (that is Lipschitz continuous and Lipschitz differentiable on the convex closure of the domain of $g$). Such models arise naturally in many contemporary applications, where $f$ is the loss function for data misfit, and $g$ and $h$ are nonsmooth functions for inducing desirable structures in $x$ and $c(x)$. Existing single-loop algorithms mainly focus either on the case where $h$ is Lipschitz continuous or the case where $h$ is an indicator function of a closed convex set. In this paper, we develop a single-loop algorithm for more general possibly non-Lipschitz $h$. Our algorithm is a single-loop variant of the successive difference-of-convex approximation method (SDCAM) proposed in [22]. We show that when $h$ is Lipschitz, our algorithm exhibits an iteration complexity that matches the best known complexity result for obtaining an $(\epsilon_1,\epsilon_2,0)$-stationary point. Moreover, we show that, by assuming additionally that dom $g$ is compact, our algorithm exhibits an iteration complexity of $\tilde{O}(\epsilon^{-4})$ for obtaining an $(\epsilon,\epsilon,\epsilon)$-stationary point when $h$ is merely continuous and real-valued. Furthermore, we consider a scenario where $h$ does not have full domain and establish vanishing bounds on successive changes of iterates. Finally, in all three cases mentioned above, we show that one can construct a subsequence such that any accumulation point $x^*$ satisfies $c(x^*)\in$ dom $h$, and if a standard constraint qualification holds at $x^*$, then $x^*$ is a stationary point.

math.OC↗

arXiv2025

A Regression-Based Prediction-Correction Method for Stochastic Time-Varying Optimization Problems

In many real-world applications, optimization problems evolve continuously over time and are often subject to stochastic noise. We consider a stochastic time-varying optimization problem in which the objective function $f(x;t)$ changes continuously and only noisy gradient observations are available. In deterministic settings, the prediction-correction method that exploits the time derivative of the solution is effective for accurately tracking the solution trajectory. However, a straightforward extension to stochastic problems requires an estimate of $\nabla_{xt} f(x;t)$ and the computation of a Hessian inverse at each step--requirements that are difficult or costly in practice. To address these issues, we propose a prediction-correction algorithm that uses a regression-based prediction step: the prediction is formed as a linear combination of recent iterates, which can be computed efficiently without estimating $\nabla_{xt}f(x;t)$ or computing Hessian inversions. We prove a tracking-error bound for the proposed method under standard smoothness and stochastic assumptions. Numerical experiments show that the regression-based prediction improves tracking accuracy while reducing computational cost compared with existing methods.

math.OC↗

arXiv2025

Inexact subgradient algorithm with a non-asymptotic convergence guarantee for copositive programming problems

In this paper, we propose a subgradient algorithm with a non-asymptotic convergence guarantee to solve copositive programming problems. The subproblem to be solved at each iteration is a standard quadratic programming problem, which is NP-hard in general. However, the proposed algorithm allows this subproblem to be solved inexactly. For a prescribed accuracy $\epsilon > 0$ for both the objective function and the constraint arising from the copositivity condition, the proposed algorithm yields an approximate solution after $O(\epsilon^{-2})$ iterations, even when the subproblems are solved inexactly. We also discuss exact and inexact approaches for solving standard quadratic programming problems and compare their performance through numerical experiments. In addition, we apply the proposed algorithm to the problem of testing complete positivity of a matrix and derive a sufficient condition for certifying that a matrix is not completely positive. Experimental results demonstrate that we can detect the lack of complete positivity in various doubly nonnegative matrices that are not completely positive.

math.OC↗

arXiv2025

Zeroth-order gradient estimators for stochastic problems with decision-dependent distributions

Stochastic optimization problems with unknown decision-dependent distributions have attracted increasing attention in recent years due to its importance in applications. Since the gradient of the objective function is inaccessible as a result of the unknown distribution, various zeroth-order methods have been developed to solve the problem. However, it remains unclear which search direction to construct a gradient estimator is more appropriate and how to set the algorithmic parameters. In this paper, we conduct a unified sample complexity analysis of zeroth-order methods across gradient estimators with different search directions. As a result, we show that gradient estimators that average over multiple directions, either uniformly from the unit sphere or from a Gaussian distribution, achieve the lowest sample complexity. The attained sample complexities improve those of existing zeroth-order methods in the problem setting that allows nonconvexity and unboundedness of the objective function. Moreover, by simulation experiments on multiple products pricing and strategic classification applications, we show practical performance of zeroth-order methods with various gradient estimators.

math.OC↗

arXiv2025

Approximate Bregman proximal gradient algorithm with variable metric Armijo--Wolfe line search

We propose a variant of the approximate Bregman proximal gradient (ABPG) algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function. ABPG is known to converge globally to a stationary point even when the smooth part of the objective function does not have a globally Lipschitz continuous gradient, and its iterates can often be expressed in closed form. However, ABPG relies on an Armijo line search to guarantee global convergence, which can slow down its practical performance. To address this issue, we propose a variant of ABPG with a variable metric Armijo--Wolfe line search. Under the variable metric Armijo--Wolfe condition, we establish global subsequential convergence of the algorithm. Moreover, assuming the Kurdyka--\L{}ojasiewicz property, we also prove that the algorithm globally converges to a stationary point. Numerical experiments on $\ell_p$-regularized least squares problems and nonnegative linear inverse problems demonstrate that the proposed algorithm outperforms existing algorithms.

math.OC↗

arXiv2025

Douglas--Rachford for multioperator comonotone inclusions with applications to multiblock optimization

We study the convergence of the adaptive Douglas--Rachford (aDR) algorithm for solving a multioperator inclusion problem involving the sum of maximally comonotone operators. To address such problems, we adopt a product space reformulation that accommodates nonconvex-valued operators, which is essential when dealing with comonotone mappings. We establish convergence of the aDR method under comonotonicity assumptions, subject to suitable conditions on the algorithm parameters and comonotonicity moduli of the operators. Our analysis leverages the Attouch--Th\'{e}ra duality framework, which allows us to study the convergence of the aDR algorithm via its application to the dual inclusion problem. As an application, we derive a multiblock ADMM-type algorithm for structured convex and nonconvex optimization problems by applying the aDR algorithm to the operator inclusion formulation of the KKT system. The resulting method extends to multiblock and nonconvex settings the classical duality between the Douglas--Rachford algorithm and the alternating direction method of multipliers in the convex two-block case. Moreover, we establish convergence guarantees for both the fully convex and strongly convex-weakly convex regimes.

math.OC↗

arXiv2025

Modified K-means Algorithm with Local Optimality Guarantees

The K-means algorithm is one of the most widely studied clustering algorithms in machine learning. While extensive research has focused on its ability to achieve a globally optimal solution, there still lacks a rigorous analysis of its local optimality guarantees. In this paper, we first present conditions under which the K-means algorithm converges to a locally optimal solution. Based on this, we propose simple modifications to the K-means algorithm which ensure local optimality in both the continuous and discrete sense, with the same computational complexity as the original K-means algorithm. As the dissimilarity measure, we consider a general Bregman divergence, which is an extension of the squared Euclidean distance often used in the K-means algorithm. Numerical experiments confirm that the K-means algorithm does not always find a locally optimal solution in practice, while our proposed methods provide improved locally optimal solutions with reduced clustering loss. Our code is available at https://github.com/lmingyi/LO-K-means.

cs.LG↗

arXiv2025

Local near-quadratic convergence of Riemannian interior point methods

We consider Riemannian optimization problems with inequality and equality constraints and analyze a class of Riemannian interior point methods for solving them. The algorithm of interest consists of outer and inner iterations. We show that, under standard assumptions, the algorithm achieves local superlinear convergence by solving a linear system at each outer iteration, removing the need for further computations in the inner iterations. We also provide a specific update for the barrier parameter that achieves local near-quadratic convergence of the algorithm. We apply our results to the method proposed by Obara, Okuno, and Takeda (2026) and show its local superlinear and near-quadratic convergence with an analysis of the second-order stationarity. To our knowledge, this is the first algorithm for constrained optimization on Riemannian manifolds that achieves both local convergence and global convergence to a second-order stationary point. Numerical results support the theoretical analyses of the proposed methods.

math.OC↗

arXiv2025

On the Role of Label Noise in the Feature Learning Process

Deep learning with noisy labels presents significant challenges. In this work, we theoretically characterize the role of label noise from a feature learning perspective. Specifically, we consider a signal-noise data distribution, where each sample comprises a label-dependent signal and label-independent noise, and rigorously analyze the training dynamics of a two-layer convolutional neural network under this data setup, along with the presence of label noise. Our analysis identifies two key stages. In Stage I, the model perfectly fits all the clean samples (i.e., samples without label noise) while ignoring the noisy ones (i.e., samples with noisy labels). During this stage, the model learns the signal from the clean samples, which generalizes well on unseen data. In Stage II, as the training loss converges, the gradient in the direction of noise surpasses that of the signal, leading to overfitting on noisy samples. Eventually, the model memorizes the noise present in the noisy samples and degrades its generalization ability. Furthermore, our analysis provides a theoretical basis for two widely used techniques for tackling label noise: early stopping and sample selection. Experiments on both synthetic and real-world setups validate our theory.

stat.ML↗