arXiv subjects

Yuto Miyatake

Publications and source records attributed to Yuto Miyatake.

At least 37 records · Page 2Linked to original sources

arXiv2022

Composing a surrogate observation operator for sequential data assimilation

In data assimilation, state estimation is not straightforward when the observation operator is unknown. This study proposes a method for composing a surrogate operator when the true operator is unknown. A neural network is used to improve the surrogate model iteratively to decrease the difference between the observations and the results of the surrogate model. A twin experiment suggests that the proposed method outperforms approaches that tentatively use a specific operator throughout the data assimilation process.

cs.LG↗

arXiv2022

High-order linearly implicit schemes conserving quadratic invariants

In this paper, we propose linearly implicit and arbitrary high-order conservative numerical schemes for ordinary differential equations with a quadratic invariant. Many differential equations have invariants, and numerical schemes for preserving them have been extensively studied. Since linear invariants can be easily kept after discretisation, quadratic invariants are essentially the simplest ones. Quadratic invariants are important objects that appear not only in many physical examples but also in the computationally efficient conservative schemes for general invariants such as scalar auxiliary variable approach, which have been studied in recent years. It is known that quadratic invariants can be kept relatively easily compared to general invariants, and indeed can be preserved by canonical Runge--Kutta methods. However, there is no unified method for constructing linearly implicit and high order conservative schemes. In this paper, we construct such schemes based on canonical Runge--Kutta methods and prove some properties involving accuracy.

math.NA↗

arXiv2021

Symplectic Adjoint Method for Exact Gradient of Neural ODE with Minimal Memory

A neural network model of a differential equation, namely neural ODE, has enabled the learning of continuous-time dynamical systems and probabilistic distributions with high accuracy. The neural ODE uses the same network repeatedly during a numerical integration. The memory consumption of the backpropagation algorithm is proportional to the number of uses times the network size. This is true even if a checkpointing scheme divides the computation graph into sub-graphs. Otherwise, the adjoint method obtains a gradient by a numerical integration backward in time. Although this method consumes memory only for a single network use, it requires high computational cost to suppress numerical errors. This study proposes the symplectic adjoint method, which is an adjoint method solved by a symplectic integrator. The symplectic adjoint method obtains the exact gradient (up to rounding error) with memory proportional to the number of uses plus the network size. The experimental results demonstrate that the symplectic adjoint method consumes much less memory than the naive backpropagation algorithm and checkpointing schemes, performs faster than the adjoint method, and is more robust to rounding errors.

cs.LG↗

arXiv2021

Computing the matrix fractional power with the double exponential formula

Two quadrature-based algorithms for computing the matrix fractional power $A^α$ are presented in this paper. These algorithms are based on the double exponential (DE) formula, which is well-known for its effectiveness in computing improper integrals as well as in treating nearly arbitrary endpoint singularities. The DE formula transforms a given integral into another integral that is suited for the trapezoidal rule; in this process, the integral interval is transformed to the infinite interval. Therefore, it is necessary to truncate the infinite interval into an appropriate finite interval. In this paper, a truncation method, which is based on a truncation error analysis specialized to the computation of $A^α$, is proposed. Then, two algorithms are presented -- one computes $A^α$ with a fixed number of abscissas, and the other computes $A^α$ adaptively. Subsequently, the convergence rate of the DE formula for Hermitian positive definite matrices is analyzed. The convergence rate analysis shows that the DE formula converges faster than the Gaussian quadrature when $A$ is ill-conditioned and $α$ is a non-unit fraction. Numerical results show that our algorithms achieved the required accuracy and were faster than other algorithms in several situations.

math.NA↗

arXiv2021

Estimation of ordinary differential equation models with discretization error quantification

We consider parameter estimation of ordinary differential equation (ODE) models from noisy observations. For this problem, one conventional approach is to fit numerical solutions (e.g., Euler, Runge--Kutta) of ODEs to data. However, such a method does not account for the discretization error in numerical solutions and has limited estimation accuracy. In this study, we develop an estimation method that quantifies the discretization error based on data. The key idea is to model the discretization error as random variables and estimate their variance simultaneously with the ODE parameter. The proposed method has the form of iteratively reweighted least squares, where the discretization error variance is updated with the isotonic regression algorithm and the ODE parameter is updated by solving a weighted least squares problem using the adjoint system. Experimental results demonstrate that the proposed method attains robust estimation with at least comparable accuracy to the conventional method by successfully quantifying the reliability of the numerical solutions.

stat.ME↗

arXiv2020

A fully discrete curve-shortening polygonal evolution law for moving boundary problems

We consider the numerical integration of moving boundary problems with the curve-shortening property, such as the mean curvature flow and Hele-Shaw flow. We propose a fully discrete curve-shortening polygonal evolution law. The proposed evolution law is fully implicit, and the key to the derivation is to devise the definitions of tangent and normal vectors and tangential and normal velocities at each vertex in an implicit manner. Numerical experiments show that the proposed method allows the use of relatively large time step sizes and also captures the area-preserving or dissipative property in good accuracy.

math.NA↗

arXiv2020

Adjoint-based exact Hessian computation

We consider a scalar function depending on a numerical solution of an initial value problem, and its second-derivative (Hessian) matrix for the initial value. The need to extract the information of the Hessian or to solve a linear system having the Hessian as a coefficient matrix arises in many research fields such as optimization, Bayesian estimation, and uncertainty quantification. From the perspective of memory efficiency, these tasks often employ a Krylov subspace method that does not need to hold the Hessian matrix explicitly and only requires computing the multiplication of the Hessian and a given vector. One of the ways to obtain an approximation of such Hessian-vector multiplication is to integrate the so-called second-order adjoint system numerically. However, the error in the approximation could be significant even if the numerical integration to the second-order adjoint system is sufficiently accurate. This paper presents a novel algorithm that computes the intended Hessian-vector multiplication exactly and efficiently. For this aim, we give a new concise derivation of the second-order adjoint system and show that the intended multiplication can be computed exactly by applying a particular numerical method to the second-order adjoint system. In the discussion, symplectic partitioned Runge--Kutta methods play an essential role.

math.NA↗

arXiv2020

Generalization of partitioned Runge--Kutta methods for adjoint systems

This study computes the gradient of a function of numerical solutions of ordinary differential equations (ODEs) with respect to the initial condition. The adjoint method computes the gradient approximately by solving the corresponding adjoint system numerically. In this context, Sanz-Serna [SIAM Rev., 58 (2016), pp. 3--33] showed that when the initial value problem is solved by a Runge--Kutta (RK) method, the gradient can be exactly computed by applying an appropriate RK method to the adjoint system. Focusing on the case where the initial value problem is solved by a partitioned RK (PRK) method, this paper presents a numerical method, which can be seen as a generalization of PRK methods, for the adjoint system that gives the exact gradient.

math.NA↗

arXiv2020

A Parallelizable Energy-Preserving Integrator MB4 and Its Application to Quantum-Mechanical Wavepacket Dynamics

In simulating physical systems, conservation of the total energy is often essential, especially when energy conversion between different forms of energy occurs frequently. Recently, a new fourth order energy-preserving integrator named MB4 was proposed based on the so-called continuous stage Runge--Kutta methods (Y.~Miyatake and J.~C.~Butcher, SIAM J.~Numer.~Anal., 54(3), 1993-2013). A salient feature of this method is that it is parallelizable, which makes its computational time for one time step comparable to that of second order methods. In this paper, we illustrate how to apply the MB4 method to a concrete ordinary differential equation using the nonlinear Schrödinger-type equation on a two-dimensional grid as an example. This system is a prototypical model of two-dimensional disordered organic material and is difficult to solve with standard methods like the classical Runge--Kutta methods due to the nonlinearity and the $δ$-function like potential coming from defects. Numerical tests show that the method can solve the equation stably and preserves the total energy to 16-digit accuracy throughout the simulation. It is also shown that parallelization of the method yields up to 2.8 times speedup using 3 computational nodes.

math.NA↗

arXiv2019

Modified Strang splitting for semilinear parabolic problems

We consider applying the Strang splitting to semilinear parabolic problems. The key ingredients of the Strang splitting are the decomposition of the equation into several parts and the computation of approximate solutions by combining the time evolution of each split equation. However, when the Dirichlet boundary condition is imposed, order reduction could occur due to the incompatibility of the split equations with the boundary condition. In this paper, to overcome the order reduction, a modified Strang splitting procedure is presented for the one-dimensional semilinear parabolic equation with first-order spatial derivatives, like the Burgers equation.

math.NA↗

arXiv2019

A structure-preserving Fourier pseudo-spectral linearly implicit scheme for the space-fractional nonlinear Schrödinger equation

We propose a Fourier pseudo-spectral scheme for the space-fractional nonlinear Schrödinger equation. The proposed scheme has the following features: it is linearly implicit, it preserves two invariants of the equation, its unique solvability is guaranteed without any restrictions on space and time step sizes. The scheme requires solving a complex symmetric linear system per time step. To solve the system efficiently, we also present a certain variable transformation and preconditioner.

math.NA↗

arXiv2019

Algorithms for the computation of the matrix logarithm based on the double exponential formula

We consider the computation of the matrix logarithm by using numerical quadrature. The efficiency of numerical quadrature depends on the integrand and the choice of quadrature formula. The Gauss--Legendre quadrature has been conventionally employed; however, the convergence could be slow for ill-conditioned matrices. This effect may stem from the rapid change of the integrand values. To avoid such situations, we focus on the double exponential formula, which has been developed to address integrands with endpoint singularity. In order to utilize the double exponential formula, we must determine a suitable finite integration interval, which provides the required accuracy and efficiency. In this paper, we present a method for selecting a suitable finite interval based on an error analysis as well as two algorithms, and one of these algorithms addresses error control.

math.NA↗

arXiv2019

Adaptive SOR methods based on the Wolfe conditions

Because the expense of estimating the optimal value of the relaxation parameter in the successive over-relaxation (SOR) method is usually prohibitive, the parameter is often adaptively controlled. In this paper, new adaptive SOR methods are presented that are applicable to a variety of symmetric positive definite linear systems and do not require additional matrix-vector products when updating the parameter. To this end, we regard the SOR method as an algorithm for minimising a certain objective function, which yields an interpretation of the relaxation parameter as the step size following a certain change of variables. This interpretation enables us to adaptively control the step size based on some line search techniques, such as the Wolfe conditions. Numerical examples demonstrate the favourable behaviour of the proposed methods.

math.NA↗

arXiv2019

Relation between the T-congruence Sylvester equation and the generalized Sylvester equation

The T-congruence Sylvester equation is the matrix equation $AX+X^{\mathrm{T}}B=C$, where $A\in\mathbb{R}^{m\times n}$, $B\in\mathbb{R}^{n\times m}$, and $C\in\mathbb{R}^{m\times m}$ are given, and $X\in\mathbb{R}^{n\times m}$ is to be determined. Recently, Oozawa et al. discovered a transformation that the matrix equation is equivalent to one of the well-studied matrix equations (the Lyapunov equation); however, the condition of the transformation seems to be too limited because matrices $A$ and $B$ are assumed to be square matrices ($m=n$). In this paper, two transformations are provided for rectangular matrices $A$ and $B$. One of them is an extension of the result of Oozawa et al. for the case $m\ge n$, and the other is a novel transformation for the case $m\le n$.

math.NA↗

arXiv2018

Structure-preserving model reduction for dynamical systems with a first integral

Since the expense of the numerical integration of large scale dynamical systems is often computationally prohibitive, model reduction methods, which approximate such systems by simpler and much lower order ones, are often employed to reduce the computational effort. In this paper, for dynamical systems with a first integral, new structure-preserving model reduction approaches are presented that yield reduced-order systems while preserving the first integral. We apply energy-preserving integrators to the reduced-order systems and show some numerical experiments that demonstrate the favourable behaviour of the proposed approaches.

math.NA↗

arXiv2018

Solution of the $k$-th eigenvalue problem in large-scale electronic structure calculations

We consider computing the $k$-th eigenvalue and its corresponding eigenvector of a generalized Hermitian eigenvalue problem of $n\times n$ large sparse matrices. In electronic structure calculations, several properties of materials, such as those of optoelectronic device materials, are governed by the eigenpair with a material-specific index $k.$ We present a three-stage algorithm for computing the $k$-th eigenpair with validation of its index. In the first stage of the algorithm, we propose an efficient way of finding an interval containing the $k$-th eigenvalue $(1 \ll k \ll n)$ with a non-standard application of the Lanczos method. In the second stage, spectral bisection for large-scale problems is realized using a sparse direct linear solver to narrow down the interval of the $k$-th eigenvalue. In the third stage, we switch to a modified shift-and-invert Lanczos method to reduce bisection iterations and compute the $k$-th eigenpair with validation. Numerical results with problem sizes up to 1.5 million are reported, and the results demonstrate the accuracy and efficiency of the three-stage algorithm.

math.NA↗

arXiv2017

On the equivalence between SOR-type methods for linear systems and discrete gradient methods for gradient systems

The iterative nature of many discretisation methods for continuous dynamical systems has led to the study of the connections between iterative numerical methods in numerical linear algebra and continuous dynamical systems. Certain researchers have used the explicit Euler method to understand this connection, but this method has its limitation. In this study, we present a new connection between successive over-relaxation (SOR)-type methods and gradient systems; this connection is based on discrete gradient methods. The focus of the discussion is the equivalence between SOR-type methods and discrete gradient methods applied to gradient systems. The discussion leads to new interpretations for SOR-type methods. For example, we found a new way to derive these methods; these methods monotonically decrease a certain quadratic function and obtain a new interpretation of the relaxation parameter. We also obtained a new discrete gradient while studying the new connection.

math.NA↗

arXiv2017

Geometric numerical integrators for Hunter-Saxton-like equations

We present novel geometric numerical integrators for Hunter--Saxton-like equations by means of new multi-symplectic formulations and known Hamiltonian structures of the problems. We consider the Hunter--Saxton equation, the modified Hunter--Saxton equation, and the two-component Hunter--Saxton equation. Multi-symplectic discretisations based on these new formulations of the problems are exemplified by means of the explicit Euler box scheme, and Hamiltonian-preserving discretisations are exemplified by means of the discrete variational derivative method. We explain and justify the correct treatment of boundary conditions in a unified manner. This is necessary for a proper numerical implementation of these equations and was never explicitly clarified in the literature before, to the best of our knowledge. Finally, numerical experiments demonstrate the favourable behaviour of the proposed numerical integrators.

math.NA↗