arXiv subjects

Tomoya Takeuchi

Publications and source records attributed to Tomoya Takeuchi.

9 recordsLinked to original sources

arXiv2026

Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization

Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware. However, training deep continuous-time SNNs is severely constrained by the memory required for exact spike-time computation, which evaluates and retains candidate firing times over intervals determined by presynaptic spike ordering. Here we introduce a memory-efficient training framework based on differentiable spike-time discretization (DSTD) for leaky integrate-and-fire neurons with general membrane and synaptic time constants. DSTD maps irregular presynaptic spikes onto differentiable weighted events at fixed time points, replacing the input-dependent candidate dimension with $M$ fixed time intervals while accurately approximating continuous-time membrane-potential dynamics. This reduces candidate-related activation memory from $O(N_{\mathrm{out}}N_{\mathrm{in}})$ to $O(N_{\mathrm{out}}M)$ in the case of time-to-first-spike (TTFS) coding, where $N_{\mathrm{in}}$ and $N_{\mathrm{out}}$ denote the numbers of presynaptic and postsynaptic neurons, respectively. We further introduce synfire-chain-inspired temporal regularization that organizes layer-wise firing windows, mitigates dead-neuron failures, and enables pipeline-like processing. In dense LIF layers, DSTD reduced peak memory consumption by up to approximately 100-fold and training time by up to approximately 20-fold compared with exact spike-time computation. Together, these methods allowed us to train 9-layer convolutional SNNs on CIFAR-10 and 20-layer convolutional SNNs on Fashion-MNIST on a single GPU.

cs.LG↗

arXiv2025

Learning the Simplest Neural ODE

Since the advent of the ``Neural Ordinary Differential Equation (Neural ODE)'' paper, learning ODEs with deep learning has been applied to system identification, time-series forecasting, and related areas. Exploiting the diffeomorphic nature of ODE solution maps, neural ODEs has also enabled their use in generative modeling. Despite the rich potential to incorporate various kinds of physical information, training Neural ODEs remains challenging in practice. This study demonstrates, through the simplest one-dimensional linear model, why training Neural ODEs is difficult. We then propose a new stabilization method and provide an analytical convergence analysis. The insights and techniques presented here serve as a concise tutorial for researchers beginning work on Neural ODEs.

stat.ML↗

arXiv2020

The Proximal Method of Multipliers for a Class of Nonsmooth Convex Optimization

This paper develops the proximal method of multipliers for a class of nonsmooth convex optimization. The method generates a sequence of minimization problems (subproblems). We show that the sequence of approximations to the solutions of the subproblems converges to a saddle point of the Lagrangian even if the original optimization problem may possess multiple solutions. The augmented Lagrangian due to Fortin appears in the subproblem. The remarkable property of the augmented Lagrangian over the standard Lagrangian is that it is always differentiable, and it is often semismoothly differentiable. This fact allows us to employ a nonsmooth Newton method for computing an approximation to the subproblem. The proximal term serves as the regularization of the objective function and guarantees the solvability of the Newton system without assuming strong convexity on the objective function. We exploit the theory of the nonsmooth Newton method to provide a rigorous proof for the global convergence of the proposed algorithm.

math.NA↗

arXiv2015

Lagrange optimality system for a class of nonsmooth convex optimization

In this paper, we revisit the augmented Lagrangian method for a class of nonsmooth convex optimization. We present the Lagrange optimality system of the augmented Lagrangian associated with the problems, and establish its connections with the standard optimality condition and the saddle point condition of the augmented Lagrangian, which provides a powerful tool for developing numerical algorithms. We apply a linear Newton method to the Lagrange optimality system to obtain a novel algorithm applicable to a variety of nonsmooth convex optimization problems arising in practical applications. Under suitable conditions, we prove the nonsingularity of the Newton system and the local convergence of the algorithm.

math.OC↗

arXiv2013

Multi-Parameter Tikhonov Regularization -- An Augmented Approach

We study multi-parameter regularization (multiple penalties) for solving linear inverse problems to promote simultaneously distinct features of the sought-for objects. We revisit a balancing principle for choosing regularization parameters from the viewpoint of augmented Tikhonov regularization, and derive a new parameter choice strategy called the \textit{balanced discrepancy principle}. A priori and a posteriori error estimates are provided to theoretically justify the principles, and numerical algorithms for efficiently implementing the principles are also provided. Numerical results on denoising are presented to illustrate the feasibility of the balanced discrepancy principle.

math.NA↗

arXiv2012

Nonsmooth method for constrained optimization

We propose an implicit iterative algorithm for an exact penalty method arising from inequality constrained optimization problems. A rapidly convergent fixed point method is developed for a regularized penalty functional. The applicability and feasibility of the proposed method is demonstrated using large scale inequality constrained problems.

math.OC↗

arXiv2011

A multi-moment scheme for the two dimensional Maxwell's equations

We develop a numerical scheme for solving time-domain Maxwell's equation. The method is motivated by CIP method which uses function values and its derivatives as unknown variables. The proposed scheme is developed by using the Poisson formula for the wave equation. It is fully explicit space and time integration method with higher order accuracy and CFL number being one. The bi-cubic interpolation is used for the solution profile to attain the resolution. It preserves sharp profiles very accurately without any smearing and distortion due to the exact time integration and high resolution approximation. The stability and numerical accuracy are investigated.

math.NA↗

arXiv2011

CIP methods for hyperbolic system with variable and discontinuous coefficient

We propose a multi-moment method for one-dimensional hyperbolic equations with smooth coefficient and piecewise constant coefficient. The method is entirely based on the backward characteristic method and uses the solution and its derivative as unknowns and cubic Hermite interpolation for each computational cell. The exact update formula for solution and its derivative is derived and used for an efficient time integration. At points of discontinuity of wave speed we define a piecewise cubic Hermite interpolation based on immersed interface method. The method is extended to the one-dimensional Maxwell's equations with variable material properties.

math.NA↗

arXiv2011

Multi-Parameter Tikhonov Regularization

We study multi-parameter Tikhonov regularization, i.e., with multiple penalties. Such models are useful when the sought-for solution exhibits several distinct features simultaneously. Two choice rules, i.e., discrepancy principle and balancing principle, are studied for choosing an appropriate (vector-valued) regularization parameter, and some theoretical results are presented. In particular, the consistency of the discrepancy principle as well as convergence rate are established, and an a posteriori error estimate for the balancing principle is established. Also two fixed point algorithms are proposed for computing the regularization parameter by the latter rule. Numerical results for several nonsmooth multi-parameter models are presented, which show clearly their superior performance over their single-parameter counterparts.

math.NA↗