arXiv subjects

Daiji Tsutsui

Publications and source records attributed to Daiji Tsutsui.

2 recordsLinked to original sources

arXiv2021

Noise-induced degeneration in online learning

In order to elucidate the plateau phenomena caused by vanishing gradient, we herein analyse stability of stochastic gradient descent near degenerated subspaces in a multi-layer perceptron. In stochastic gradient descent for Fukumizu-Amari model, which is the minimal multi-layer perceptron showing non-trivial plateau phenomena, we show that (1) attracting regions exist in multiply degenerated subspaces, (2) a strong plateau phenomenon emerges as a noise-induced synchronisation, which is not observed in deterministic gradient descent, (3) an optimal fluctuation exists to minimise the escape time from the degenerated subspace. The noise-induced degeneration observed herein is expected to be found in a broad class of machine learning via neural networks.

nlin.AO↗

arXiv2020

Optimal transport problems regularized by generic convex functions: A geometric and algorithmic approach

In order to circumvent the difficulties in solving numerically the discrete optimal transport problem, in which one minimizes the linear target function $P\mapsto\langle C,P\rangle:=\sum_{i,j}C_{ij}P_{ij}$, Cuturi introduced a variant of the problem in which the target function is altered by a convex one $Φ(P)=\langle C,P\rangle-λ\mathcal{H}(P)$, where $\mathcal{H}$ is the Shannon entropy and $λ$ is a positive constant. We herein generalize their formulation to a target function of the form $Φ(P)=\langle C,P\rangle+λf(P)$, where $f$ is a generic strictly convex smooth function. We also propose an iterative method for finding a numerical solution, and clarify that the proposed method is particularly efficient when $f(P)=\frac{1}{2}\|P\|^2$.

math.OC↗