arXiv subjects
Publications and source records attributed to Masahiro Ikeda.
We identify hidden layers inside a deep neural network (DNN) with group actions on the data domain, and formulate a formal deep network as a dual voice transform with respect to the Koopman operator, a linear representation of the group action. Based on the group theoretic arguments, particularly by using Schur's lemma, we show a simple proof of the universality of DNNs.
The symmetry and geometry of input data are considered to be encoded in the internal data representation inside the neural network, but the specific encoding rule has been less investigated. In this study, we present a systematic method to induce a generalized neural network and its right inverse operator, called the ridgelet transform, from a joint group invariant function on the data-parameter domain. Since the ridgelet transform is an inverse, (1) it can describe the arrangement of parameters for the network to represent a target function, which is understood as the encoding rule, and (2) it implies the universality of the network. Based on the group representation theory, we present a new simple proof of the universality by using Schur's lemma in a unified manner covering a wide class of networks, for example, the original ridgelet transform, formal deep networks, and the dual voice transform. Since traditional universality theorems were demonstrated based on functional analysis, this study sheds light on the group theoretic aspect of the approximation theory, connecting geometric deep learning to abstract harmonic analysis.
Reproducing kernel Hilbert $C^*$-module (RKHM) is a generalization of reproducing kernel Hilbert space (RKHS) by means of $C^*$-algebra, and the Perron-Frobenius operator is a linear operator related to the composition of functions. Combining these two concepts, we present deep RKHM, a deep learning framework for kernel methods. We derive a new Rademacher generalization bound in this setting and provide a theoretical interpretation of benign overfitting by means of Perron-Frobenius operators. By virtue of $C^*$-algebra, the dependency of the bound on output dimension is milder than existing bounds. We show that $C^*$-algebra is a suitable tool for deep learning with kernels, enabling us to take advantage of the product structure of operators and to provide a clear connection with convolutional neural networks. Our theoretical analysis provides a new lens through which one can design and analyze deep kernel methods.
An algebraic upper bound for the decay rate of solutions to the Navier-Stokes and Navier-Stokes-Coriolis equations in the critical space $\dot{H} ^{\frac{1}{2}} (\mathbb{R} ^3)$ is derived using the Fourier Splitting Method. Estimates are framed in terms of the decay character of initial data, leading to solutions with algebraic decay and showing in detail the roles played by the linear and nonlinear parts.
We introduce a linear operator on a Hilbert $C^*$-module for analyzing skew-product dynamical systems. The operator is defined by composition and multiplication. We show that it admits a decomposition in the Hilbert $C^*$-module, called eigenoperator decomposition, that generalizes the concept of the eigenvalue decomposition. This decomposition reconstructs the Koopman operator of the system in a manner that represents the continuous spectrum through eigenoperators. In addition, it is related to the notions of cocycle and Oseledets subspaces and it is useful for characterizing coherent structures under skew-product dynamics. We present numerical applications to simple systems on two-dimensional domains.
We consider the Cauchy problem of the system of nonlinear Schrödinger equations with derivative nonlinearlity. This system was introduced by Colin-Colin (2004) as a model of laser-plasma interactions. We study existence of ground state solutions and the global well-posedness of this system by using the variational methods. We also consider the stability of traveling waves for this system. These problems are proposed by Colin-Colin as the open problems. We give a subset of the ground-states set which satisfies the condition of stability. In particular, we prove the stability of the set of traveling waves with small speed for $1$-dimension.
We consider the large time behavior of the solutions to the Cauchy problem for the BBM-Burgers equation. We prove that the solution to this problem goes to the self-similar solution to the Burgers equation called the nonlinear diffusion wave. Moreover, we construct the appropriate second asymptotic profiles of the solutions depending on the initial data. Based on that discussion, we investigate the effect of the initial data on the large time behavior of the solution, and derive the optimal asymptotic rate to the nonlinear diffusion wave. Especially, the important point of this study is that the second asymptotic profiles of the solutions with slowly decaying data, whose case has not been studied, are obtained.
Lifespan estimates for semilinear damped wave equations of the form $\partial_t^2u-Δu+\partial_tu=|u|^p$ in a two dimensional exterior domain endowed with the Dirichlet boundary condition are dealt with. For the critical case of the semilinear heat equation $\partial_tv-Δv=v^2$ with the Dirichlet boundary condition and the initial condition $v(0)=\varepsilon f$, the corresponding lifespan can be estimated from below and above by $\exp(\exp(C\varepsilon^{-1}))$ with different constants $C$. This paper clarifies that the same estimates hold even for the critical semilinear damped wave equation in the exterior of the unit ball under the restriction of radial symmetry. To achieve this result, a new technique to control $L^1$-type norm and a new Gagliardo--Nirenberg type estimate with logarithmic weight are introduced.
This paper aims to characterize boundedness of composition operators on Besov spaces $B^s_{p,q}$ of higher order derivatives $s>1+1/p$ on the one-dimensional Euclidean space. In contrast to the lower order case $0 1$. We prove a relation between the composition operators and pointwise multipliers of Besov spaces, and effectively use the characterizations of the pointwise multipliers. As a result, we obtain necessary and sufficient conditions for the boundedness of composition operators for general $p$, $q$, and $s$ such that $1 1+1/p$. In this paper, we treat, as a map that induces the composition operator, not only a homeomorphism on the real line but also a continuous map whose number of elements of inverse images at any one point is bounded above. We also show a similar characterization of the boundedness of composition operators on Sobolev spaces.
In this paper, we consider the energy critical nonlinear Schrödinger equation with a repulsive inverse square potential. In particular, we deal with radial initial data, whose energy is equal to the energy of static solution to the corresponding nonlinear Schrödinger equation without a potential. We investigate time behavior of the radial solutions with such initial data.
We consider the nonlinear Schrödinger equations on the star graph with the Kirchhoff boundary and the repulsive Dirac delta boundary at the origin. In the present paper, we show the scattering-blowup dichotomy result below the mass-energy of the ground state on the real line. The proof of the scattering part is based on a concentration compactness and rigidity argument. Our main contribution is to give a linear profile decomposition on the star graph by using a symmetrical decomposition.
This paper is concerned with the Cauchy problem of a multivalued ordinary differential equation governed by the hypergraph Laplacian, which describes the diffusion of ``heat'' or ``particles'' on the vertices of hypergraph. We consider the case where the heat on several vertices are manipulated internally by the observer, namely, are fixed by some given functions. This situation can be reduced to a nonlinear evolution equation associated with a time-dependent subdifferential operator, whose solvability has been investigated in numerous previous researches. In this paper, however, we give an alternative proof of the solvability in order to avoid some complicated calculations arising from the chain rule for the time-dependent subdifferential. As for results which cannot be assured by the known abstract theory, we also discuss the continuous dependence of solution on the given data and the time-global behavior of solution.
We show the universality of depth-2 group convolutional neural networks (GCNNs) in a unified and constructive manner based on the ridgelet theory. Despite widespread use in applications, the approximation property of (G)CNNs has not been well investigated. The universality of (G)CNNs has been shown since the late 2010s. Yet, our understanding on how (G)CNNs represent functions is incomplete because the past universality theorems have been shown in a case-by-case manner by manually/carefully assigning the network parameters depending on the variety of convolution layers, and in an indirect manner by converting/modifying the (G)CNNs into other universal approximators such as invariant polynomials and fully-connected networks. In this study, we formulate a versatile depth-2 continuous GCNN $S[γ]$ as a nonlinear mapping between group representations, and directly obtain an analysis operator, called the ridgelet trasform, that maps a given function $f$ to the network parameter $γ$ so that $S[γ]=f$. The proposed GCNN covers typical GCNNs such as the cyclic convolution on multi-channel images, networks on permutation-invariant inputs (Deep Sets), and $\mathrm{E}(n)$-equivariant networks. The closed-form expression of the ridgelet transform can describe how the network parameters are organized to represent a function. While it has been known only for fully-connected networks, this study is the first to obtain the ridgelet transform for GCNNs. By discretizing the closed-form expression, we can systematically generate a constructive proof of the $cc$-universality of finite GCNNs. In other words, our universality proofs are more unified and constructive than previous proofs.
Neural network on Riemannian symmetric space such as hyperbolic space and the manifold of symmetric positive definite (SPD) matrices is an emerging subject of research in geometric deep learning. Based on the well-established framework of the Helgason-Fourier transform on the noncompact symmetric space, we present a fully-connected network and its associated ridgelet transform on the noncompact symmetric space, covering the hyperbolic neural network (HNN) and the SPDNet as special cases. The ridgelet transform is an analysis operator of a depth-2 continuous network spanned by neurons, namely, it maps an arbitrary given function to the weights of a network. Thanks to the coordinate-free reformulation, the role of nonlinear activation functions is revealed to be a wavelet function, and the reconstruction formula directly yields the universality of the proposed networks.
Let $V$ be a finite set, $E \subset 2^{V} $ be a set of hyperedges, and $w : E \to (0, \infty)$ be an edge weight. On the (wighted) hypergraph $G = (V ,E ,w )$, we can define a multivalued nonlinear operator $L_{G,p}$ ($p \in [1 ,\infty )$) as the subdifferential of a convex function on $\mathbb{R} ^V $, which is called "hypergraph $p$-Laplacian." In this article, we first introduce an inequality for this operator $L_{G,p}$ which resembles the Poincaré-Wirtinger inequality in PDEs. Next we consider an ordinary differential equation on $\mathbb{R} ^V $ governed by $L_{G,p}$, which is referred as "heat" equation on the hypergraph and used to study the geometric structure of graph in recent researches. With the aid of the Poincaré-Wirtinger type inequality, we can discuss the existence and the large time behavior of solutions to the ODE by procedures similar to those for the standard heat equation in PDEs with the zero Neumann boundary condition.
Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning. However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry. The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs. We apply the framework to two representative classes of INNs, namely Coupling-Flow-based INNs (CF-INNs) and Neural Ordinary Differential Equations (NODEs), and elucidate their high representation power despite the restrictions on their architectures.
In this paper, we consider the mass-critical nonlinear Schrödinger equation in one dimension. Ogawa--Tsutsumi [19] proved a blow-up result for negative energy solution by using a scaling argument for initial data. By the reason, the method cannot be used to an equation with a linear potential. So, we modify the proof and get that for the equation with the linear potential.
In this paper we study long time dynamics (i.e., scattering and blow-up) of solutions for the focusing NLS with a repulsive inverse-power potential and with initial data lying exactly at the mass-energy threshold, namely, when $E_{V}(u_{0})M(u_{0})=E_{0}(Q)M(Q)$. Moreover, we prove failure of the uniform space-time bounds at the mass-energy threshold.