arXiv subjects

Yuki Kurumadani

Publications and source records attributed to Yuki Kurumadani.

3 recordsLinked to original sources

arXiv2026

Upper Bounds for Local Learning Coefficients of Three-Layer Neural Networks

Three-layer neural networks are known to form singular learning models, and their Bayesian asymptotic behavior is governed by the learning coefficient, or real log canonical threshold. Although this quantity has been clarified for regular models and for some special singular models, broadly applicable methods for evaluating it in neural networks remain limited. Recently, a formula for the local learning coefficient of semiregular models was proposed, yielding an upper bound on the learning coefficient. However, this formula applies only to nonsingular points in the set of realization parameters and cannot be used at singular points. In particular, for three-layer neural networks, the resulting upper bound has been shown to differ substantially from learning coefficient values already known in some cases. In this paper, we derive a formula for an upper bound on local learning coefficients at a class of singular realization parameters in three-layer neural networks. This formula can be interpreted as a counting rule under budget, demand, and supply constraints. In the non-polynomial real-analytic case, the formula applies in general settings, whereas in the polynomial case it applies under the restriction that the true distribution has no hidden units. In particular, our result covers activation functions such as the swish function and also includes polynomial activation functions under the above restriction, thereby extending previous results to a broader class of activation functions. We further show that, when the input dimension is one, the numerical value given by the right-hand side of our upper-bound formula agrees with the previously known learning coefficient, thereby providing a useful comparison with known exact results. Our result also provides a systematic perspective on how the weight parameters of three-layer neural networks affect the learning coefficient.

cs.LG↗

arXiv2024

Real Log Canonical Thresholds at Non-singular Points

Recent advances have clarified theoretical learning accuracy in Bayesian inference, revealing that the asymptotic behavior of metrics such as generalization loss and free energy, assessing predictive accuracy, is dictated by a rational number unique to each statistical model, termed the learning coefficient (real log canonical threshold). For models meeting regularity conditions, their learning coefficients are known. However, for singular models not meeting these conditions, exact values of learning coefficients are provided for specific models like reduced-rank regression, but a broadly applicable calculation method for these learning coefficients in singular models remains elusive. This paper extends the application range of the previous work and provides an approach that can be applied to many points within the set of realizable parameters. Specifically, it provides a formula for calculating the real log canonical threshold at many non-singular points within the set of realizable parameters. If this calculation can be performed, it is possible to obtain an upper bound for the learning coefficient of the statistical model. Thus, this approach can also be used to easily obtain an upper bound for the learning coefficients of statistical models. As an application example, it provides an upper bound for the learning coefficient of a mixed binomial model, and calculates the learning coefficient for a specific case of reduced-rank regression, confirming that the results are consistent with previous research.

math.ST↗

arXiv2024

Learning Coefficients in Semi-Regular Models

Recent advances have clarified theoretical learning accuracy in Bayesian inference, revealing that the asymptotic behavior of metrics such as generalization loss and free energy, assessing predictive accuracy, is dictated by a rational number unique to each statistical model, termed the learning coefficient (real log canonical threshold) . For models meeting regularity conditions, their learning coefficients are known. However, for singular models not meeting these conditions, exact values of learning coefficients are provided for specific models like reduced-rank regression, but a broadly applicable calculation method for these learning coefficients in singular models remains elusive. The problem of determining learning coefficients relates to finding normal crossings of Kullback-Leibler divergence in algebraic geometry. In this context, it is crucial to perform appropriate coordinate transformations and blow-ups. This paper introduces an approach that utilizes properties of the log-likelihood ratio function for constructing specific variable transformations and blow-ups to uniformly calculate the real log canonical threshold. It was found that linear independence in the differential structure of the log-likelihood ratio function significantly influences the real log canonical threshold. This approach has not been considered in previous research. In this approach, the paper presents cases and methods for calculating the exact values of learning coefficients in statistical models that satisfy a simple condition next to the regularity conditions (semi-regular models), offering examples of learning coefficients for two-parameter semi-regular models and mixture distribution models with a constant mixing ratio.

math.ST↗