arXiv subjects

Kenichi Bannai

Publications and source records attributed to Kenichi Bannai.

At least 19 recordsLinked to original sources

arXiv2025

Log-Normal Multiplicative Dynamics for Stable Low-Precision Training of Large Networks

Studies in neuroscience have shown that biological synapses follow a log-normal distribution whose transitioning can be explained by noisy multiplicative dynamics. Biological networks can function stably even under dynamically fluctuating conditions arising due to unreliable synaptic transmissions. Here we ask: Is it possible to design similar multiplicative training in artificial neural networks? To answer this question, we derive a Bayesian learning rule that assumes log-normal posterior distributions over weights which gives rise to a new Log-Normal Multiplicative Dynamics (LMD) algorithm. The algorithm uses multiplicative updates with both noise and regularization applied multiplicatively. The method is as easy to implement as Adam and only requires one additional vector to store. Our results show that LMD achieves stable and accurate training-from-scratch under low-precision forward operations for Vision Transformer and GPT-2. These results suggest that multiplicative dynamics, a biological feature, may enable stable low-precision inference and learning on future energy-efficient hardware.

cs.LG↗

arXiv2024

On Interactions for Large Scale Interacting Systems

Statistical mechanics explains the properties of macroscopic phenomena based on the movements of microscopic particles such as atoms and molecules. Movements of microscopic particles can be represented by large-scale interacting systems. In this article, we study a combinatorial object which we call interactions, given as a symmetric directed graph representing the possible transition of states on adjacent sites of large-scale interacting systems. Such interactions underlie various standard processes such as the exclusion processes, generalized exclusion processes, multi-species exclusion processes, lattice-gas processes with energy, and the multi-lane particle processes. We introduce the notion of equivalences of interactions using their space of conserved quantities. This allows for the classification of interactions reflecting corresponding macroscopic properties. In particular, we prove that when the set of local states consists of two, three or four elements, then the number of equivalence classes of separable interactions are respectively one, two and five. We also define the wedge sums and box products of interactions, which give systematic methods for constructing new interactions from existing ones. Furthermore, we prove that the irreducibly quantified condition for interactions, which has implicitly played an important role in the theory of hydrodynamic limits, is preserved by wedge sums and box products. Our results provide a systematic method to construct and classify interactions, offering abundant examples suitable for considering hydrodynamic limits.

math.PR↗

arXiv2024

Topological Structures of Large Scale Interacting Systems via Uniform Functions and Forms

In this article, we investigate the topological structure of large scale interacting systems on infinite graphs, by constructing a suitable cohomology which we call the uniform cohomology. The central idea for the construction is the introduction of a class of functions called uniform functions. Uniform cohomology provides a new perspective for the identification of macroscopic observables from the microscopic system. As a straightforward application of our theory when the underlying graph has a free action of a group, we prove a certain decomposition theorem for shift-invariant closed uniform forms. This result is a uniform version in a very general setting of the decomposition result for shift-invariant closed $L^2$-forms originally proposed by Varadhan, which has repeatedly played a key role in the proof of the hydrodynamic limits of nongradient large scale interacting systems. In a subsequent article, we use this result as a key to prove Varadhan's decomposition theorem for a general class of large scale interacting systems.

math.PR↗

arXiv2024

On Uniform Functions on Configuration Spaces of Large Scale Interacting Systems

Stochastic large scale interacting systems can be studied via the observables, i.e. functions on the underlying configuration space. In our previous article, we introduced the concept of uniform functions, which are suitable class of functions on configuration spaces underlying stochastic systems on infinite graphs. An important consequence is the successful characterization of conserved quantities without introducing the notion of stationary distributions. In this article, we further develop the theory of uniform functions and construct the theory independent of any choice of a base state. Furthermore, we generalize the notion of interactions given in our previous article to accommodate the case where there are multiple possible state transitions on adjacent vertices. We then prove that if the interaction is exchangeable, then any uniform function which gives a global conserved quantity can be expressed as a sum of local conserved quantities of the interaction. Contrary to our previous article, we do not need to assume that the interaction is irreducibly quantified. This shows that our theory of uniform functions on configuration spaces over infinite graphs with transition structure given by an exchangeable interaction is a natural framework to study general stochastic large scale interacting systems. While some of the ideas in this article are based on our previous article, the article is logically independent and self-contained.

math.PR↗

arXiv2024

Varadhan's Decomposition of Shift-Invariant Closed $L^2$-forms for Large Scale Interacting Systems on the Euclidean Lattice

We rigorously formulate and prove for a relatively general class of interactions Varadhan's Decomposition of shift-invariant closed $L^2$-forms for a large scale interacting system on the Euclidean lattice with finite range. Such decomposition of closed forms has played an essential role in proving the diffusive scaling limit of nongradient systems. A general expression in terms of conserved quantities was sought from observations for specific models, but a precise formulation or rigorous proof up until now had been elusive. Our result is based on a general decomposition theorem of shift-invariant closed uniform forms studied in our previous article (arXiv:2009.04699). In the present article, we show that the same universal structure also appears for $L^2$-forms. The essential assumptions are: (i) the set of states on each vertex is a finite set, (ii) the measure on the configuration space is the product measure, and (iii) there is a certain uniform spectral gap estimate for the mean field version of the interaction. As a special case, our result gives Varadhan's decomposition for the case of the multi-species exclusion process, which is a new result that could not be proved by existing methods. Our result also gives complete proofs of Varadhan's decompositions for finite range interactions, whose detailed proofs have been missing in the literature - presumably because there exists an obstruction to the standard method of proof.

math.PR↗

arXiv2023

The Hodge Realization of the Polylogarithm and the Shintani Generating Class for Totally Real Fields

In this article, we construct the Hodge realization of the polylogarithm class in the equivariant Deligne-Beilinson cohomology of a certain algebraic torus associated to a totally real field. We then prove that the de Rham realization of this polylogarithm gives the Shintani generating class, a cohomology class generating the values of the Lerch zeta functions of the totally real field at nonpositive integers. Inspired by this result, we give a conjecture concerning the specialization of this polylogarithm class at torsion points, and discuss its relation to the Beilinson conjecture for Hecke characters of totally real fields.

math.NT↗

arXiv2022

Varadhan's decomposition of shift-invariant closed uniform forms for large scale interacting systems on general crystal lattices

We prove a uniform version of Varadhan decomposition for shift-invariant closed uniform forms associated to large scale interacting systems on general crystal lattices. In particular, this result includes the case of translation invariant processes on Euclidean lattices $\mathbf{Z}^d$ with finite range. Our result generalizes the result of arXiv:2009.04699 which was valid for systems on transferable graphs. In subsequent research, we will use the result of this article to prove Varadhan's decomposition of closed $L^2$-forms for large scale interacting systems on general crystal lattices.

math.PR↗

arXiv2022

$p$-adic Polylogarithms and $p$-adic Hecke $L$-functions for Totally Real Fields

The purpose of this article is to newly define the $p$-adic polylogarithm as an equivariant class in the cohomology of a certain infinite disjoint union of algebraic tori associated to a totally real field. We will then express the special values of $p$-adic $L$-functions interpolating nonpositive values of Hecke $L$-functions of the totally real field in terms of special values of these $p$-adic polylogarithms.

math.NT↗

arXiv2022

A Decomposition Theorem of Varadhan Type for Co-local Forms on Large Scale Interacting Systems

In our previous article with Yukio Kametani, we investigated the geometric structure underlying a large scale interacting system on infinite graphs, via constructing a suitable cohomology theory called uniformly local cohomology, which reflects the geometric property of the microscopic model, using a class of functions called the uniformly local functions. In this article, we introduce the co-local functions on the geometric structure associated to a large scale interacting system. We may similarly define the notion of uniformly local functions for co-local functions. However, contrary to the functions appearing in our previous article, the co-local functions reflect the stochastic property of the model, namely the probability measure on the configuration space. We then prove a decomposition theorem of Varadhan type for closed co-local forms. The space of co-local functions and forms contain the space of $L^2$-functions and forms. In the last section, we state a conjecture concerning the decomposition theorem for the $L^2$-case.

math.PR↗

arXiv2020

Category of mixed plectic Hodge structures

The purpose of this article is to investigate the properties of the category of mixed plectic Hodge structures defined by Nekovář and Scholl. We give an equivalent description of mixed plectic Hodge structures in terms of the weight and partial Hodge filtrations. We also construct an explicit complex calculating the extension groups in this category.

math.AG↗

arXiv2020

Theory and Algorithms for Shapelet-based Multiple-Instance Learning

We propose a new formulation of Multiple-Instance Learning (MIL), in which a unit of data consists of a set of instances called a bag. The goal is to find a good classifier of bags based on the similarity with a "shapelet" (or pattern), where the similarity of a bag with a shapelet is the maximum similarity of instances in the bag. In previous work, some of the training instances are chosen as shapelets with no theoretical justification. In our formulation, we use all possible, and thus infinitely many shapelets, resulting in a richer class of classifiers. We show that the formulation is tractable, that is, it can be reduced through Linear Programming Boosting (LPBoost) to Difference of Convex (DC) programs of finite (actually polynomial) size. Our theoretical result also gives justification to the heuristics of some of the previous work. The time complexity of the proposed algorithm highly depends on the size of the set of all instances in the training sample. To apply to the data containing a large number of instances, we also propose a heuristic option of the algorithm without the loss of the theoretical guarantee. Our empirical study demonstrates that our algorithm uniformly works for Shapelet Learning tasks on time-series classification and various MIL tasks with comparable accuracy to the existing methods. Moreover, we show that the proposed heuristics allow us to achieve the result with reasonable computational time.

cs.LG↗

arXiv2020

Canonical Equivariant Cohomology Classes Generating Zeta Values of Totally Real Fields

It is known that the special values at nonpositive integers of a Dirichlet $L$-function may be expressed using the generalized Bernoulli numbers, which are defined by a canonical generating function. The purpose of this article is to consider the generalization of this classical result to the case of Hecke $L$-functions of totally real fields. Hecke $L$-functions may be expressed canonically as a finite sum of zeta functions of Lerch type. By combining the non-canonical multivariable generating functions constructed by Shintani, we newly construct a canonical class, which we call the Shintani generating class, in the equivariant cohomology of an algebraic torus associated to the totally real field. Our main result states that the specializations at torsion points of the derivatives of the Shintani generating class give values at nonpositive integers of the zeta functions of Lerch type. This result gives the insight that the correct framework in the higher dimensional case is to consider higher equivariant cohomology classes instead of functions.

math.NT↗

arXiv2020

The Hodge realization of the polylogarithm on the product of multiplicative groups

The purpose of this article is to describe explicitly the polylogarithm class in absolute Hodge cohomology of a product of multiplicative groups, in terms of the Bloch-Wigner-Ramakrishnan polylogarithm functions. We will use the logarithmic Dolbeault complex defined by Burgos to calculate the corresponding absolute Hodge cohomology groups.

math.AG↗

arXiv2018

Bezier Simplex Fitting: Describing Pareto Fronts of Simplicial Problems with Small Samples in Multi-objective Optimization

Multi-objective optimization problems require simultaneously optimizing two or more objective functions. Many studies have reported that the solution set of an M-objective optimization problem often forms an (M-1)-dimensional topological simplex (a curved line for M=2, a curved triangle for M=3, a curved tetrahedron for M=4, etc.). Since the dimensionality of the solution set increases as the number of objectives grows, an exponentially large sample size is needed to cover the solution set. To reduce the required sample size, this paper proposes a Bezier simplex model and its fitting algorithm. These techniques can exploit the simplex structure of the solution set and decompose a high-dimensional surface fitting task into a sequence of low-dimensional ones. An approximation theorem of Bezier simplices is proven. Numerical experiments with synthetic and real-world optimization problems demonstrate that the proposed method achieves an accurate approximation of high-dimensional solution sets with small samples. In practice, such an approximation will be conducted in the post-optimization process and enable a better trade-off analysis.

math.OC↗

arXiv2018

Multiple-Instance Learning by Boosting Infinitely Many Shapelet-based Classifiers

We propose a new formulation of Multiple-Instance Learning (MIL). In typical MIL settings, a unit of data is given as a set of instances called a bag and the goal is to find a good classifier of bags based on similarity from a single or finitely many "shapelets" (or patterns), where the similarity of the bag from a shapelet is the maximum similarity of instances in the bag. Classifiers based on a single shapelet are not sufficiently strong for certain applications. Additionally, previous work with multiple shapelets has heuristically chosen some of the instances as shapelets with no theoretical guarantee of its generalization ability. Our formulation provides a richer class of the final classifiers based on infinitely many shapelets. We provide an efficient algorithm for the new formulation, in addition to generalization bound. Our empirical study demonstrates that our approach is effective not only for MIL tasks but also for Shapelet Learning for time-series classification.

cs.LG↗

arXiv2017

Boosting the kernelized shapelets: Theory and algorithms for local features

We consider binary classification problems using local features of objects. One of motivating applications is time-series classification, where features reflecting some local closeness measure between a time series and a pattern sequence called shapelet are useful. Despite the empirical success of such approaches using local features, the generalization ability of resulting hypotheses is not fully understood and previous work relies on a bunch of heuristics. In this paper, we formulate a class of hypotheses using local features, where the richness of features is controlled by kernels. We derive generalization bounds of sparse ensembles over the class which is exponentially better than a standard analysis in terms of the number of possible local features. The resulting optimization problem is well suited to the boosting approach and the weak learning problem is formulated as a DC program, for which practical algorithms exist. In preliminary experiments on time-series data sets, our method achieves competitive accuracy with the state-of-the-art algorithms with small parameter-tuning cost.

cs.LG↗

arXiv2015

Integral structures on $p$-adic Fourier theory

In this article, we give an explicit construction of the $p$-adic Fourier transform by Schneider and Teitelbaum, which allows for the investigation of the integral property. As an application, we give a certain integral basis of the space of $K$-locally analytic functions on the ring of integers $\mathcal{O}_K$ for any finite extension $K$ of $\mathbb{Q}_p$, generalizing the basis constructed by Amice for locally analytic functions on $\mathbb{Z}_p$. We also use our result to prove congruences of Bernoulli-Hurwitz numbers at non-ordinary (i.e. supersingular) primes originally investigated by Katz and Chellali.

math.NT↗

arXiv2014

$p$-adic Eisenstein-Kronecker series for CM elliptic curves and the Kronecker limit formulas

Consider an elliptic curve defined over an imaginary quadratic field $K$ with good reduction at the primes above $p\geq 5$ and has complex multiplication by the full ring of integers $\mathcal{O}_K$ of $K$. In this paper, we construct $p$-adic analogues of the Eisenstein-Kronecker series for such elliptic curve as Coleman functions on the elliptic curve. We then prove $p$-adic analogues of the first and second Kronecker limit formulas by using the distribution relation of the Kronecker theta function.

math.NT↗