arXiv subjects

Shotaro Yagishita

Publications and source records attributed to Shotaro Yagishita.

12 recordsLinked to original sources

arXiv2026

A proximal gradient method with adaptive backtracking for weakly smooth multiobjective optimization

In this paper, we propose a proximal gradient method with adaptive linesearch for multiobjective optimization problems whose objective functions are weakly smooth, i.e., they have Hölder continuous gradients. The proposed method is parameter-free as we do not require prior knowledge of parameters related to the weak smoothness of the objective function; the proposed linesearch finds an appropriate step-size that adapts to the weak smoothness. The complexity guarantee analyzed in this paper for the non-convex case is compatible with related works and our algorithm accepts coercer stationarity measure compared to existing methods. We also establish a novel complexity result for the convex case which improves the one in non-convex case.

math.OC↗

arXiv2026

Proximal gradient-type method with generalized distance and convergence analysis without global descent lemma

We consider solving nonconvex composite optimization problems in which the sum of a smooth function and a nonsmooth function is minimized. Many of convergence analyses of proximal gradient-type methods rely on global descent property between the smooth term and its proximal term. On the other hand, the ability to efficiently solve the subproblem depends on the compatibility between the nonsmooth term and the proximal term. Selecting an appropriate proximal term by considering both factors simultaneously is generally difficult. We overcome this issue by providing convergence analyses for proximal gradient-type methods with general proximal terms, without requiring global descent property of the smooth term. As a byproduct, new convergence results of the interior gradient methods for conic optimization are also provided.

math.OC↗

arXiv2025

Simple linesearch-free first-order methods for nonconvex optimization

This paper presents an auto-conditioned proximal gradient method for nonconvex optimization. The method determines the stepsize using an estimation of local curvature and does not require any prior knowledge of problem parameters and any linesearch procedures. Its convergence analysis is carried out in a simple manner without assuming the convexity, unlike previous studies. We also provide convergence analysis in the presence of the Kurdyka--Łojasiewicz property, adaptivity to the weak smoothness, and the extension to the Bregman proximal gradient method. Furthermore, the auto-conditioned stepsize strategy is also applied to the conditional gradient (Frank--Wolfe) method and the Riemannian gradient method.

math.OC↗

arXiv2025

Outlier-robust neural network training: variation regularization meets trimmed loss to prevent functional breakdown

In this study, we tackle the challenge of outlier-robust predictive modeling using highly expressive neural networks. Our approach integrates two key components: (1) a transformed trimmed loss (TTL), a computationally efficient variant of the classical trimmed loss, and (2) higher-order variation regularization (HOVR), which imposes smoothness constraints on the prediction function. While traditional robust statistics typically assume low-complexity models such as linear and kernel models, applying TTL alone to modern neural networks may fail to ensure robustness, as their high expressive power allows them to fit both inliers and outliers, even when a robust loss is used. To address this, we revisit the traditional notion of breakdown point and adapt it to the nonlinear function setting, introducing a regularization scheme via HOVR that controls the model's capacity and suppresses overfitting to outliers. We theoretically establish that our training procedure retains a high functional breakdown point, thereby ensuring robustness to outlier contamination. We develop a stochastic optimization algorithm tailored to this framework and provide a theoretical guarantee of its convergence.

stat.ME↗

arXiv2025

Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions

The generalized conditional gradient method is a popular algorithm for solving composite problems whose objective function is the sum of a smooth function and a nonsmooth convex function. Many convergence analyses of the algorithm rely on smoothness assumptions, such as the Lipschitz continuity of the gradient of the smooth part. This paper provides convergence results of linesearch-based generalized conditional gradient methods without smoothness assumptions. In particular, we show that a parameter-free variant, which automatically adapts to the Hölder exponent, guarantees convergence even when the gradient of the smooth part of the objective is not Hölder continuous.

math.OC↗

arXiv2024

Optimal reinsurance and investment via stochastic projected gradient method based on Malliavin calculus

This paper proposes a new approach using the stochastic projected gradient method and Malliavin calculus for optimal reinsurance and investment strategies. Unlike traditional methodologies, we aim to optimize static investment and reinsurance strategies by directly minimizing the ruin probability. Furthermore, we provide a convergence analysis of the stochastic projected gradient method for general constrained optimization problems whose objective function has Hölder continuous gradient. Numerical experiments show the effectiveness of our proposed method.

q-fin.MF↗

arXiv2024

Fast algorithm for sparse least trimmed squares via trimmed-regularized reformulation

The least trimmed squares (LTS) is a reasonable formulation of robust regression whereas it suffers from high computational cost due to the nonconvexity and nonsmoothness of its objective function. The most frequently used FAST-LTS algorithm is particularly slow when a sparsity-inducing penalty such as the $\ell_1$ norm is added. This paper proposes a computationally inexpensive algorithm for the sparse LTS, which is based on the proximal gradient method with a reformulation technique. Proposed method is equipped with theoretical convergence preferred over existing methods. Numerical experiments show that our method efficiently yields small objective value.

stat.CO↗

arXiv2024

Exact Penalization at D-Stationary Points of Cardinality- or Rank-Constrained Problem

This paper studies the properties of d-stationary points of the trimmed lasso (Luo et al., 2013, Huang et al., 2015, and Gotoh et al., 2018) and the composite optimization problem with the truncated nuclear norm (Gao and Sun, 2010, and Zhang et al., 2012), which are known as tight relaxations of nonconvex optimization problems that have either cardinality or rank constraints, respectively. First, we extend the trimmed lasso for broader applications and for conducting a unified analysis of the property of the generalized trimmed lasso. Next, the equivalence between local optimality and d-stationarity of the generalized trimmed lasso is shown under a suitable assumption. More generally, the equivalence is shown for problems that minimize a function defined by the pointwise minimum of finitely many convex functions. Then, we present new results of the exact penalty at d-stationary points of the generalized trimmed lasso and the problem with the truncated nuclear norm penalty under mild assumptions. Our exact penalty results are not only new, but also intuitive, so that we can see what properties of the problems play a role for establishing the exact penalization. Lastly, matters relating to algorithms are also discussed.

math.OC↗

arXiv2023

Proximal Diagonal Newton Methods for Composite Optimization Problems

This paper proposes new proximal Newton-type methods with a diagonal metric for solving composite optimization problems whose objective function is the sum of a twice continuously differentiable function and a proper closed directionally differentiable function. Although proximal Newton-type methods using diagonal metrics have been shown to be superior to the proximal gradient method numerically, no theoretical results have been obtained to suggest this superiority. Even though our proposed method is based on a simple idea, its convergence rate suggests an advantage over the proximal gradient method in certain situations. Numerical experiments show that our proposed algorithms are effective, especially in the nonconvex case.

math.OC↗

arXiv2023

Exact Penalty Method for Knot Selection of B-Spline Regression

This paper presents a new approach to selecting knots at the same time as estimating the B-spline regression model. Such simultaneous selection of knots and model is not trivial, but our strategy can make it possible by employing a nonconvex regularization on the least square method that is usually applied. More specifically, motivated by the constraint that directly designates (the upper bound of) the number of knots to be used, we present an (unconstrained) regularized least square reformulation, which is later shown to be equivalent to the motivating cardinality-constrained formulation. The obtained formulation is further modified so that we can employ a proximal gradient-type algorithm, known as GIST, for a class of non-smooth non-convex optimization problems. We show that under a mild technical assumption, the algorithm is shown to reach a local minimum of the problem. Since it is shown that any local minimum of the problem satisfies the cardinality constraint, the proposed algorithm can be used to obtain a spline regression model that depends only on a designated number of knots at most. Numerical experiments demonstrate how our approach performs on synthetic and real data sets.

math.OC↗

arXiv2022

Some Martingale Properties of Simple Random Walk and Its Maximum Process

In this paper, martingales related to simple random walks and their maximum process are investigated. First, a sufficient condition under which a function with three arguments, time, the random walk, and its maximum process becomes a martingale is presented, and as an application, an alternative way of deriving the Kennedy martingale is provided. Then, a complete characterization of a function with two arguments, the random walk and its maximum, being a martingale is presented. This martingale can be regarded as a discrete version of the Azéma--Yor martingale. As applications of discrete Azéma--Yor martingale, a proof of the Doob's inequalities is provided and a discrete Azéma--Yor solution for the Skorokhod embedding problem for the simple random walk is formulated and examined in detail.

math.PR↗

arXiv2021

Pursuit of the Cluster Structure of Network Lasso: Recovery Condition and Non-convex Extension

Network Lasso (NL for short) is a methodology for estimating models by simultaneously clustering data samples and fitting the models to the samples. It often succeeds in forming clusters thanks to the geometry of the $\ell_1$-regularizer employed therein, but there might be limitations because of the convexity of the regularizer. This paper focuses on the cluster structure that NL yields and reinforces it by developing a non-convex extension, which we call Network Trimmed Lasso (NTL for short). Specifically, we first study a sufficient condition that guarantees the recovery of the latent cluster structure of NL on the basis of the result of Sun et al. (2021) for Convex Clustering, which is a special case of NL for clustering. Second, we extend NL to NTL to incorporate a cardinality (or, $\ell_0$-)constraint and rewrite the constrained optimization problem defined with the $\ell_0$ norm, a discontinuous function, into an equivalent unconstrained continuous optimization problem. We develop ADMM algorithms to solve NTL and provide its convergence results. Numerical illustrations demonstrate that the non-convex extension provides a more clear-cut cluster structure when NL fails to form clusters without incorporating prior knowledge of the associated parameters.

math.OC↗