Searcharxiv⌕ Search

arXiv subjects

Akiyoshi Sannai

Publications and source records attributed to Akiyoshi Sannai.

33 records · Page 2Linked to original sources
arXiv2019

Universal approximations of permutation invariant/equivariant functions by deep neural networks

In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine transformations are $G$-equivariant/invariant. Due to representation theory, we can show that this approximator has exponentially fewer free parameters than usual models.

cs.LG↗
arXiv2019

Asymptotic Risk of Bezier Simplex Fitting

The Bezier simplex fitting is a novel data modeling technique which exploits geometric structures of data to approximate the Pareto front of multi-objective optimization problems. There are two fitting methods based on different sampling strategies. The inductive skeleton fitting employs a stratified subsampling from each skeleton of a simplex, whereas the all-at-once fitting uses a non-stratified sampling which treats a simplex as a whole. In this paper, we analyze the asymptotic risks of those Bézier simplex fitting methods and derive the optimal subsample ratio for the inductive skeleton fitting. It is shown that the inductive skeleton fitting with the optimal ratio has a smaller risk when the degree of a Bezier simplex is less than three. Those results are verified numerically under small to moderate sample sizes. In addition, we provide two complementary applications of our theory: a generalized location problem and a multi-objective hyper-parameter tuning of the group lasso. The former can be represented by a Bezier simplex of degree two where the inductive skeleton fitting outperforms. The latter can be represented by a Bezier simplex of degree three where the all-at-once fitting gets an advantage.

cs.LG↗
arXiv2018

Bezier Simplex Fitting: Describing Pareto Fronts of Simplicial Problems with Small Samples in Multi-objective Optimization

Multi-objective optimization problems require simultaneously optimizing two or more objective functions. Many studies have reported that the solution set of an M-objective optimization problem often forms an (M-1)-dimensional topological simplex (a curved line for M=2, a curved triangle for M=3, a curved tetrahedron for M=4, etc.). Since the dimensionality of the solution set increases as the number of objectives grows, an exponentially large sample size is needed to cover the solution set. To reduce the required sample size, this paper proposes a Bezier simplex model and its fitting algorithm. These techniques can exploit the simplex structure of the solution set and decompose a high-dimensional surface fitting task into a sequence of low-dimensional ones. An approximation theorem of Bezier simplices is proven. Numerical experiments with synthetic and real-world optimization problems demonstrate that the proposed method achieves an accurate approximation of high-dimensional solution sets with small samples. In practice, such an approximation will be conducted in the post-optimization process and enable a better trade-off analysis.

math.OC↗
arXiv2018

Reconstruction of training samples from loss functions

This paper presents a new mathematical framework to analyze the loss functions of deep neural networks with ReLU functions. Furthermore, as as application of this theory, we prove that the loss functions can reconstruct the inputs of the training samples up to scalar multiplication (as vectors) and can provide the number of layers and nodes of the deep neural network. Namely, if we have all input and output of a loss function (or equivalently all possible learning process), for all input of each training sample $x_i \in \mathbb{R}^n$, we can obtain vectors $x'_i\in \mathbb{R}^n$ satisfying $x_i=c_ix'_i$ for some $c_i \neq 0$. To prove theorem, we introduce the notion of virtual polynomials, which are polynomials written as the output of a node in a deep neural network. Using virtual polynomials, we find an algebraic structure for the loss surfaces, called semi-algebraic sets. We analyze these loss surfaces from the algebro-geometric point of view. Factorization of polynomials is one of the most standard ideas in algebra. Hence, we express the factorization of the virtual polynomials in terms of their active paths. This framework can be applied to the leakage problem in the training of deep neural networks. The main theorem in this paper indicates that there are many risks associated with the training of deep neural networks. For example, if we have N (the dimension of weight space) + 1 nonsmooth points on the loss surface, which are sufficiently close to each other, we can obtain the input of training sample up to scalar multiplication. We also point out that the structures of the loss surfaces depend on the shape of the deep neural network and not on the training samples.

stat.ML↗
arXiv2014

Homotopy invariance of higher K-theory for abelian categories

The main theorem in this paper is that the base change functor from a noetherian abelian category to its noetherian polynomial category induces an isomorphism on K-theory. The main theorem implies the well-known fact that A^1-homotopy invariance of K'-theory for noetherian schemes.

math.AG↗
arXiv2013

Dual F-signature

We define the dual F-signature of modules, which is equivalent to the F-signature if the module is the base ring. By using this invariant, We give characterizations of regular, F-regular, F-rational, and Gorenstein singularities.

math.AC↗
arXiv2011

Jet schemes of homogeneous hypersurfaces

This paper studies the singularities of jet schemes of homogeneous hypersurfaces of general type. We obtain the condition of the degree and the dimension for the singularities of the jet schemes to be of dense $F$-regular type. This provides us with examples of singular varieties whose $m$-jet schemes have rational singularities for every $m$.

math.AG↗
arXiv2011

F-signature of graded Gorenstein rings

For a commutative ring $R$, the $F$-signature was defined by Huneke and Leuschke \cite{H-L}. It is an invariant that measures the order of the rank of the free direct summand of $R^{(e)}$. Here, $R^{(e)}$ is $R$ itself, regarded as an $R$-module through $e$-times Frobenius action $F^e$.In this paper, we show a connection of the F-signature of a graded ring with other invariants. More precisely, for a graded $F$-finite Gorenstein ring $R$ of dimension $d$, we give an inequality among the $F$-signature $s(R)$, $a$-invariant $a(R)$ and Poincaré polynomial $P(R,t)$. \[ s(R)\le\frac{(-a(R))^d}{2^{d-1}d!}\lim_{t\rightarrow 1}(1-t)^dP(R,t) \]Moreover, we show that $R^{(e)}$ has only one free direct summand for any $e$, if and only if $R$ is $F$-pure and $a(R)=0$. This gives a characterization of such rings.

math.AC↗
arXiv2011

Higher K-theory of polynomial categories

The main theorem in this paper is that the base change functor from an abelian category $\cA$ to its polynomial category in the sense of Schlichting $-\otimes_{\cA}\bbZ[t]:\cA \to \cA[t]$ induces an isomorphism on their $K$-theories if $\cA$ is noetherian and has enough projective objects. The main theorem implies the well-known fact that $\mathbb{A}^1$-homotopy invariance of $K'$-theory for noetherian schemes.

math.AC↗
arXiv2011

Generalized Koszul resolutions

The main objective of this paper is to generalize a notion of Koszul resolutions and charcterizing modules which admits such a resolution. We turn out that for a noetherian ring $A$ and a coherent $A$ module $M$, $M$ has a two dimensional generalized Koszul resolution if and only if $M$ is a pure weight two module in the sense of \cite{HM09}.

math.AC↗
arXiv2011

Galois extensions, plus closure, and maps on local cohomology

Given a local domain $(R,m)$ of prime characteristic that is a homomorphic image of a Gorenstein ring, Huneke and Lyubeznik proved that there exists a module-finite extension domain $S$ such that the induced map on local cohomology modules $H^i_m(R)\to H^i_m(S)$ is zero for each $i<\dim R$. We prove that the extension $S$ may be chosen to be generically Galois, and analyze the Galois groups that arise.

math.AC↗