arXiv subjects

Shun Sato

Publications and source records attributed to Shun Sato.

At least 19 recordsLinked to original sources

arXiv2026

A Restart-Free Accelerated Algorithm for Non-Convex Minimization: Continuous and Discrete Analysis

We propose two novel first-order methods for minimizing nonconvex functions with Lipschitz-continuous gradients and Hessians. These algorithms attain an $\varepsilon$-approximate first-order stationary point in $\mathrm{O}(\varepsilon^{-7/4})$ function and gradient evaluations, without using $\varepsilon$ as an input parameter. While existing methods rely on restart mechanisms to achieve this complexity, our methods do not. Consequently, the first algorithm enjoys a simple implementation, making its last iterate differentiable with respect to the initial point. By estimating the Lipschitz constants adaptively, we develop the second algorithm that does not require prior knowledge of the constants. This algorithm exhibits better numerical performance than existing parameter-free methods for certain problems, which can be attributed to its restart-free design. Both algorithms are derived by discretizing a newly introduced continuous-time model represented by an ordinary differential equation, and their continuous- and discrete-time convergence analyses proceed in a parallel manner under the Performance Estimation Problem framework.

math.OC↗

arXiv2026

Structural Inconsistency and Stability Classification of Multi-symplectic Diamond Schemes

Multi-symplectic diamond schemes proposed by McLachlan and Wilkins (2015) provide a framework for the numerical integration of Hamiltonian partial differential equations, combining local implicitness with high-order accuracy and discrete multi-symplectic conservation laws. Despite these advantages, their behavior beyond a limited class of model equations remains poorly understood, and numerical difficulties may arise depending on the underlying multi-symplectic formulation. In this paper, we present a systematic stability analysis framework for diamond schemes applied to general multi-symplectic PDEs. The approach consists of three stages. First, we identify structural inconsistency of the local diamond update using Dulmage--Mendelsohn decomposition, revealing cases in which the scheme is intrinsically unsolvable. Second, we introduce a graph-based error-propagation analysis that yields a necessary stability condition by detecting negative cycles in a weighted directed graph. Third, for equations that pass the preliminary tests, we derive eigenvalue-based timestep restrictions providing sufficient conditions for stability. The analysis leads to a comprehensive classification of multi-symplectic PDEs according to whether diamond schemes are structurally inconsistent, unconditionally unstable, or conditionally stable. In particular, we show that benchmark equations such as the Korteweg--de Vries equation are intrinsically incompatible with the diamond update, while systems including the nonlinear Dirac and ``good'' Boussinesq equations admit stability regimes under mild timestep scaling. Extensive numerical experiments confirm the theoretical predictions and demonstrate the practical implications of the proposed framework. Our results clarify fundamental limitations of diamond schemes and provide practical guidelines for their reliable application to new PDE models.

math.NA↗

arXiv2026

A Mathematical Analysis of a Smooth-Convex-Concave Splitting Scheme for the Swift--Hohenberg Equation

The Swift--Hohenberg equation is a widely studied fourth-order model, originally proposed to describe hydrodynamic fluctuations. It admits an energy-dissipation law and, under suitable assumptions, bounded solutions. Many structure-preserving numerical schemes have been proposed to retain such properties; however, existing approaches are often fully implicit and therefore computationally expensive. We introduce a simple design principle for constructing dissipation-preserving finite difference schemes and apply it to the Swift--Hohenberg equation in three spatial dimensions. Our analysis relies on discrete inequalities for the underlying energy, assuming a Lipschitz continuous gradient and either convexity or $\mu$-strong convexity of the relevant terms. The resulting method is linearly implicit, yet it preserves the original energy-dissipation law, guarantees unique solvability, ensures boundedness of numerical solutions, and admits an a priori error estimate, provided that the time step is sufficiently small. To the best of our knowledge, this is the first linearly implicit finite difference scheme for the Swift--Hohenberg equation for which all of these properties are established.

math.NA↗

arXiv2025

Essential Convergence Rates of Continuous-Time Models for Optimization Methods

Designing and analyzing optimization methods via continuous-time models expressed as ordinary differential equations (ODEs) is a promising approach for its intuitiveness and simplicity. A key concern, however, is that the convergence rates of such models can be arbitrarily modified by time rescaling, rendering the task of seeking ODEs with ``fast'' convergence meaningless. To eliminate this ambiguity of the rates, we introduce the notion of the essential convergence rate. We justify this notion by proving that, under appropriate assumptions on discretization, no method obtained by discretizing an ODE can achieve a faster rate than its essential convergence rate.

math.OC↗

arXiv2025

Can Test-time Computation Mitigate Reproduction Bias in Neural Symbolic Regression?

Mathematical expressions play a central role in scientific discovery. Symbolic regression aims to automatically discover such expressions from given numerical data. Recently, Neural symbolic regression (NSR) methods that involve Transformers pre-trained on synthetic datasets have gained attention for their fast inference, but they often perform poorly, especially with many input variables. In this study, we analyze NSR from both theoretical and empirical perspectives and show that (1) ordinary token-by-token generation is ill-suited for NSR, as Transformers cannot compositionally generate tokens while validating numerical consistency, and (2) the search space of NSR methods is greatly restricted due to reproduction bias, where the majority of generated expressions are merely copied from the training data. We further examine whether tailored test-time strategies can reduce reproduction bias and show that providing additional information at test time effectively mitigates it. These findings contribute to a deeper understanding of the limitation of NSR approaches and provide guidance for designing more robust and generalizable methods. Code is available at https://github.com/Shun-0922/Mem-Bias-NSR .

cs.LG↗

arXiv2024

Analysis of nonquadratic energy-conservative schemes for KdV type-equations

Numerical schemes that conserve invariants have demonstrated superior performance in various contexts, and several unified methods have been developed for constructing such schemes. However, the mathematical properties of these schemes remain poorly understood, except in norm-preserving cases. This study introduces a novel analytical framework applicable to general energy-preserving schemes. The proposed framework is applied to Korteweg-de Vries (KdV)-type equations, establishing global existence and convergence estimates for the numerical solutions.

math.NA↗

arXiv2024

On the B-series composition theorem

The B-series composition theorem has been an important topic in numerical analysis of ordinary differential equations for the past-half century. Traditional proofs of this theorem rely on labelled trees, whereas recent developments in B-series analysis favour the use of unlabelled trees. In this paper, we present a new proof of the B-series composition theorem that does not depend on labelled trees. A key challenge in this approach is accurately counting combinations related to ``pruning.'' This challenge is overcome by introducing the concept of ``assignment.''

math.NA↗

arXiv2024

A novel interpretation of Nesterov's acceleration via variable step-size linear multistep methods

Nesterov's acceleration in continuous optimization can be understood in a novel way when Nesterov's accelerated gradient (NAG) method is considered as a linear multistep (LM) method for gradient flow. Although the NAG method for strongly convex functions (NAG-sc) has been fully discussed, the NAG method for $L$-smooth convex functions (NAG-c) has not. To fill this gap, we show that the existing NAG-c method can be interpreted as a variable step size LM (VLM) for the gradient flow. Surprisingly, the VLM allows linearly increasing step sizes, which explains the acceleration in the convex case. Here, we introduce a novel technique for analyzing the absolute stability of VLMs. Subsequently, we prove that NAG-c is optimal in a certain natural class of VLMs. Finally, we construct a new broader class of VLMs by optimizing the parameters in the VLM for ill-conditioned problems. According to numerical experiments, the proposed method outperforms the NAG-c method in ill-conditioned cases. These results imply that the numerical analysis perspective of the NAG is a promising working environment, and considering a broader class of VLMs could further reveal novel methods.

math.NA↗

arXiv2024

Mathematical analysis and numerical comparison of energy-conservative schemes for the Zakharov equations

Furihata and Matsuo proposed in 2010 an energy-conserving scheme for the Zakharov equations, as an application of the discrete variational derivative method (DVDM). This scheme is distinguished from conventional methods (in particular the one devised by Glassey in 1992) in that the invariants are consistent with respect to time, but it has not been sufficiently studied both theoretically and numerically. In this study, we theoretically prove the solvability under the loosest possible assumptions. We also prove the convergence of this DVDM scheme by improving the argument by Glassey. Furthermore, we perform intensive numerical experiments for comparing the above two schemes. It is found that the DVDM scheme is superior in terms of accuracy, but since it is fully-implicit, the linearly-implicit Glassey scheme is better for practical efficiency. In addition, we proposed a way to choose a solution for the first step that would allow Glassey's scheme to work more efficiently.

math.NA↗

arXiv2023

A new unified framework for designing convex optimization methods with prescribed theoretical convergence estimates: A numerical analysis approach

We propose a new unified framework for describing and designing gradient-based convex optimization methods from a numerical analysis perspective. There the key is the new concept of weak discrete gradients (weak DGs), which is a generalization of DGs standard in numerical analysis. Via weak DG, we consider abstract optimization methods, and prove unified convergence rate estimates that hold independent of the choice of weak DGs except for some constants in the final estimate. With some choices of weak DGs, we can reproduce many popular existing methods, such as the steepest descent and Nesterov's accelerated gradient method, and also some recent variants from numerical analysis community. By considering new weak DGs, we can easily explore new theoretically-guaranteed optimization methods; we show some examples. We believe this work is the first attempt to fully integrate research branches in optimization and numerical analysis areas, so far independently developed.

math.OC↗

arXiv2022

Existence results on Lagrange multiplier approach for gradient flows and application to optimization

This paper deals with the geometric numerical integration of gradient flow and its application to optimization. Gradient flows often appear as model equations of various physical phenomena, and their dissipation laws are essential. Therefore, dissipative numerical methods, which are numerical methods replicating the dissipation law, have been studied in the literature. Recently, Cheng, Liu, and Shen proposed a novel dissipative method, the Lagrange multiplier approach, for gradient flows, which is computationally cheaper than existing dissipative methods. Although their efficacy is numerically confirmed in existing studies, the existence results of the Lagrange multiplier approach are not known in the literature. In this paper, we establish some existence results. We prove the existence of the solution under a relatively mild assumption. In addition, by restricting ourselves to a special case, we show some existence and uniqueness results with concrete bounds. As gradient flows also appear in optimization, we further apply the latter results to optimization problems.

math.OC↗

arXiv2022

Essential convergence rate of ordinary differential equations appearing in optimization

Some continuous optimization methods can be connected to ordinary differential equations (ODEs) by taking continuous limits, and their convergence rates can be explained by the ODEs. However, since such ODEs can achieve any convergence rate by time scaling, the correspondence is not as straightforward as usually expected, and deriving new methods through ODEs is not quite direct. In this letter, we pay attention to stability restriction in discretizing ODEs and show that acceleration by time scaling basically implies deceleration in discretization; they balance out so that we can define an attainable unique convergence rate which we call an "essential convergence rate".

math.NA↗

arXiv2022

High-order linearly implicit schemes conserving quadratic invariants

In this paper, we propose linearly implicit and arbitrary high-order conservative numerical schemes for ordinary differential equations with a quadratic invariant. Many differential equations have invariants, and numerical schemes for preserving them have been extensively studied. Since linear invariants can be easily kept after discretisation, quadratic invariants are essentially the simplest ones. Quadratic invariants are important objects that appear not only in many physical examples but also in the computationally efficient conservative schemes for general invariants such as scalar auxiliary variable approach, which have been studied in recent years. It is known that quadratic invariants can be kept relatively easily compared to general invariants, and indeed can be preserved by canonical Runge--Kutta methods. However, there is no unified method for constructing linearly implicit and high order conservative schemes. In this paper, we construct such schemes based on canonical Runge--Kutta methods and prove some properties involving accuracy.

math.NA↗

arXiv2021

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim of the competition was to build systems that can predict correct answers while also satisfying strict on-disk memory budgets. These memory budgets were designed to encourage contestants to explore the trade-off between storing retrieval corpora or the parameters of learned models. In this report, we describe the motivation and organization of the competition, review the best submissions, and analyze system predictions to inform a discussion of evaluation for open-domain QA.

cs.CL↗

arXiv2021

Scalar auxiliary variable approach for conservative/dissipative partial differential equations with unbounded energy

In this paper, we present a novel investigation of the so-called SAV approach, which is a framework to construct linearly implicit geometric numerical integrators for partial differential equations with variational structure. SAV approach was originally proposed for the gradient flows that have lower-bounded nonlinear potentials such as the Allen-Cahn and Cahn-Hilliard equations, and this assumption on the energy was essential. In this paper, we propose a novel approach to address gradient flows with unbounded energy such as the KdV equation by a decomposition of energy functionals. Further, we will show that the equation of the SAV approach, which is a system of equations with scalar auxiliary variables, is expressed as another gradient system that inherits the variational structure of the original system. This expression allows us to construct novel higher-order integrators by a certain class of Runge-Kutta methods. We will propose second and fourth order schemes for conservative systems in our framework and present several numerical examples.

math.NA↗

arXiv2020

Linearly implicit structure-preserving schemes for Hamiltonian systems

Kahan's method and a two-step generalization of the discrete gradient method are both linearly implicit methods that can preserve a modified energy for Hamiltonian systems with a cubic Hamiltonian. These methods are here investigated and compared. The schemes are applied to the Korteweg-de Vries equation and the Camassa-Holm equation, and the numerical results are presented and analysed.

math.NA↗

arXiv2018

Linear gradient structures and discrete gradient methods for conservative/dissipative differential-algebraic equations

In this paper, we consider the use of discrete gradients for differential-algebraic equations (DAEs) with a conservation/dissipation law. As one of the most popular numerical methods for conservative/dissipative ordinary differential equations, the framework of discrete gradient methods has been intensively developed over recent decades. Although discrete gradients have been applied to several specific conservative/dissipative DAEs, no unified framework for DAEs has yet been constructed. In this paper, we move toward the establishment of such a framework, and introduce concepts including an appropriate linear gradient structure for DAEs. Then, we reveal that the simple use of discrete gradients does not imply the discrete conservation/dissipation laws. Fortunately, however, we can successfully construct a new discrete gradient method for the case of index-1 DAEs. We believe this first attempt provides an indispensable basis for constructing a unified framework of discrete gradient methods for DAEs.

math.NA↗

arXiv2018

Stability and convergence of a conservative finite difference scheme for the modified Hunter--Saxton equation

The modified Hunter--Saxton equation models the propagation of short capillary-gravity waves. As it involves a mixed derivative, its initial value problem on the periodic domain is much more complicated than the standard evolutionary equations. Although its local well-posedness has recently been proved, the behavior of its solution is yet to be investigated. In this paper, to develop a reliable numerical method for this problem, we derive a conservative finite difference scheme. Then, we rigorously prove not only its stability in the sense of the uniform norm but also its uniform convergence to sufficiently smooth exact solutions. Discrete conservation laws are used to overcome the difficulty due to the mixed derivative.

math.NA↗