Searcharxiv⌕ Search

arXiv subjects

Yuto Omae

Publications and source records attributed to Yuto Omae.

12 recordsLinked to original sources
arXiv2026

Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks

The flatness hypothesis suggests that flatness of the loss landscape, as measured by the eigenvalues of the loss Hessian, correlates with better neural network generalization. While various algorithms reduce these eigenvalues, most focus on procedural design, leaving it unclear how data distributions and NN parameters structurally determine directions toward flat minima. Characterizing these directions analytically is generally intractable. To overcome this mathematical difficulty, recent studies derived the Wolkowicz-Styan (WS) upper bound on the maximum eigenvalue of the cross-entropy loss Hessian in three-layer NNs. Although this upper bound is differentiable, its gradient was not derived. Therefore, we analytically derive the gradient of the WS upper bound to characterize directions leading to flat minima. Based on this, we propose Hessian Spectral Range (HSR) Regularization, which updates parameters along the steepest descent direction of the WS bound. Experiments demonstrate that HSR Regularization narrows the Hessian eigenvalue spectrum, avoids sharp minima and saddle points, and promotes convergence to flat minima. Although the applicability of this method is currently limited to cross-entropy loss and three-layer architectures, to the best of the authors' knowledge, this is the first study to report a closed-form gradient that promotes convergence to flat minima without numerical approximations. Therefore, the theoretical analysis of this gradient is expected to contribute to the further development of NNs.

cs.LG↗
arXiv2026

Analytical Evaluation of DCA Convergence Properties for Minimizing Prediction Functions of Gaussian RBF Support Vector Regression

For nonconvex optimization problems whose objective is the prediction function of a trained Support Vector Regression (SVR) model with the Gaussian radial basis function (RBF) kernel (RBF-SVR), we present a framework that applies the difference of convex functions (DC) algorithm (DCA) by exploiting the analytical structure of the RBF kernel to construct an explicit DC decomposition. Specifically, we derive in closed form both the lower bound $\mu$ of the strong convexity parameter of the DC components and the upper bound $L$ of the gradient Lipschitz constant of the subproblem. Both $\mu$ and $L$ are determined solely by the post-training dual-coefficient sum $C_{\alpha}$ and the RBF kernel parameter $\gamma$, together with the DC decomposition parameter $\rho$, and they share a common leading term $C_{\alpha}\rho$. Through numerical experiments on six benchmark functions, we show that $C_{\alpha}\rho$ is the primary single quantity characterizing both the convergence properties and the initial-point dependence of DCA, and further demonstrate that it decomposes into two independent pathways, $C \to C_{\alpha}$ and $\gamma \to \rho$, with its primary variation governed by the SVR hyperparameters $(C, \gamma)$. Together, these results allow the convergence properties of DCA on RBF-SVR to be assessed in advance through the single scalar quantity $C_{\alpha}\rho$: approximately from $(C, \gamma)$ before training, and exactly in closed form after training.

cs.LG↗
arXiv2026

Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks

Neural networks (NNs) are central to modern machine learning and achieve state-of-the-art results in many applications. However, the relationship between loss geometry and generalization is still not well understood. The local geometry of the loss function near a critical point is well-approximated by its quadratic form, obtained through a second-order Taylor expansion. The coefficients of the quadratic term correspond to the Hessian matrix, whose eigenspectrum allows us to evaluate the sharpness of the loss at the critical point. Extensive research suggests flat critical points generalize better, while sharp ones lead to higher generalization error. However, sharpness requires the Hessian eigenspectrum, but general matrix characteristic equations have no closed-form solution. Therefore, most existing studies on evaluating loss sharpness rely on numerical approximation methods. Existing closed-form analyses of the eigenspectrum are primarily limited to simplified architectures, such as linear or ReLU-activated networks; consequently, theoretical analysis of smooth nonlinear multilayer neural networks remains limited. Against this background, this study focuses on nonlinear, smooth multilayer neural networks and derives a closed-form upper bound for the maximum eigenvalue of the Hessian with respect to the cross-entropy loss by leveraging the Wolkowicz-Styan bound. Specifically, the derived upper bound is expressed as a function of the affine transformation parameters, hidden layer dimensions, and the degree of orthogonality among the training samples. The primary contribution of this paper is an analytical characterization of loss sharpness in smooth nonlinear multilayer neural networks via a closed-form expression, avoiding explicit numerical eigenspectrum computation. We hope that this work provides a small yet meaningful step toward unraveling the mysteries of deep learning.

cs.LG↗
arXiv2025

PCA- and SVM-Grad-CAM for Convolutional Neural Networks: Closed-form Jacobian Expression

Convolutional Neural Networks (CNNs) are an effective approach for classification tasks, particularly when the training dataset is large. Although CNNs have long been considered a black-box classification method, they can be used as a white-box method through visualization techniques such as Grad-CAM. When training samples are limited, incorporating a Principal Component Analysis (PCA) layer and/or a Support Vector Machine (SVM) classifier into a CNN can effectively improve classification performance. However, traditional Grad-CAM cannot be directly applied to PCA and/or SVM layers. It is important to generate attention regions for PCA and/or SVM layers in CNNs to facilitate the development of white-box methods. Therefore, we propose ``PCA-Grad-CAM'', a method for visualizing attention regions in PCA feature vectors, and ``SVM-Grad-CAM'', a method for visualizing attention regions in an SVM classifier layer. To complete our methods analytically, it is necessary to solve the closed-form Jacobian consisting of partial derivatives from the last convolutional layer to the PCA and/or SVM layers. In this paper, we present the exact closed-form Jacobian and the visualization results of our methods applied to several major datasets.

cs.LG↗
arXiv2025

Cardiomyopathy Diagnosis Model from Endomyocardial Biopsy Specimens: Appropriate Feature Space and Class Boundary in Small Sample Size Data

As the number of patients with heart failure increases, machine learning (ML) has garnered attention in cardiomyopathy diagnosis, driven by the shortage of pathologists. However, endomyocardial biopsy specimens are often small sample size and require techniques such as feature extraction and dimensionality reduction. This study aims to determine whether texture features are effective for feature extraction in the pathological diagnosis of cardiomyopathy. Furthermore, model designs that contribute toward improving generalization performance are examined by applying feature selection (FS) and dimensional compression (DC) to several ML models. The obtained results were verified by visualizing the inter-class distribution differences and conducting statistical hypothesis testing based on texture features. Additionally, they were evaluated using predictive performance across different model designs with varying combinations of FS and DC (applied or not) and decision boundaries. The obtained results confirmed that texture features may be effective for the pathological diagnosis of cardiomyopathy. Moreover, when the ratio of features to the sample size is high, a multi-step process involving FS and DC improved the generalization performance, with the linear kernel support vector machine achieving the best results. This process was demonstrated to be potentially effective for models with reduced complexity, regardless of whether the decision boundaries were linear, curved, perpendicular, or parallel to the axes. These findings are expected to facilitate the development of an effective cardiomyopathy diagnostic model for its rapid adoption in medical practice.

cs.LG↗
arXiv2023

Comparative study of 1D and 2D convolutional neural network models with attribution analysis for gravitational wave detection from compact binary coalescences

Recent advancements in gravitational wave astronomy have seen the application of convolutional neural networks (CNNs) in signal detection from compact binary coalescences. This study presents a comparative analysis of two CNN architectures: one-dimensional (1D) and two-dimensional (2D) along with an ensemble model combining both. We trained these models to detect gravitational wave signals from binary black hole (BBH) mergers, neutron star-black hole (NSBH) mergers, and binary neutron star (BNS) mergers within real detector noise. Our investigation entailed a comprehensive evaluation of the detection performance of each model type across different signal classes. To understand the models' decision-making processes, we employed feature map visualization and attribution analysis. The findings revealed that while the 1D model showed superior performance in detecting BBH signals, the 2D model excelled in identifying NSBH and BNS signals. Notably, the ensemble model outperformed both individual models across all signal types, demonstrating enhanced detection capabilities. Additionally, input feature visualization indicated distinct areas of focus in the data for the 1D and 2D models, emphasizing the effectiveness of their combination.

gr-qc↗
arXiv2023

Visualizing convolutional neural network for classifying gravitational waves from core-collapse supernovae

In this study, we employ a convolutional neural network to classify gravitational waves originating from core-collapse supernovae. Training is conducted using spectrograms derived from three-dimensional numerical simulations of waveforms, which are injected onto real noise data from the third observing run of both Advanced LIGO and Advanced Virgo. To gain insights into the decision-making process of the model, we apply class activation mapping techniques to visualize the regions in the input image that are significant for the model's prediction. The class activation maps reveal that the model's predictions predominantly rely on specific features within the input spectrograms, namely, the $g$-mode and low-frequency modes. The visualization of convolutional neural network models provides interpretability to enhance their reliability and offers guidance for improving detection efficiency.

astro-ph.IM↗
arXiv2023

Multi-rules mining algorithm for combinatorially exploded decision trees with modified Aitchison-Aitken function-based Bayesian optimization

Decision trees offer the benefit of easy interpretation because they allow the classification of input data based on if--then rules. However, as decision trees are constructed by an algorithm that achieves clear classification with minimum necessary rules, the trees possess the drawback of extracting only minimum rules, even when various latent rules exist in data. Approaches that construct multiple trees using randomly selected feature subsets do exist. However, the number of trees that can be constructed remains at the same scale because the number of feature subsets is a combinatorial explosion. Additionally, when multiple trees are constructed, numerous rules are generated, of which several are untrustworthy and/or highly similar. Therefore, we propose "MAABO-MT" and "GS-MRM" algorithms that strategically construct trees with high estimation performance among all possible trees with small computational complexity and extract only reliable and non-similar rules, respectively. Experiments are conducted using several open datasets to analyze the effectiveness of the proposed method. The results confirm that MAABO-MT can discover reliable rules at a lower computational cost than other methods that rely on randomness. Furthermore, the proposed method is confirmed to provide deeper insights than single decision trees commonly used in previous studies. Therefore, MAABO-MT and GS-MRM can efficiently extract rules from combinatorially exploded decision trees.

cs.LG↗
arXiv2023

FG-SSA: Features Gradient-based Signals Selection Algorithm of Linear Complexity for Convolutional Neural Networks

Recently, many convolutional neural networks (CNNs) for classification by time domain data of multisignals have been developed. Although some signals are important for correct classification, others are not. When data that do not include important signals for classification are taken as the CNN input layer, the calculation, memory, and data collection costs increase. Therefore, identifying and eliminating nonimportant signals from the input layer are important. In this study, we proposed features gradient-based signals selection algorithm (FG-SSA), which can be used for finding and removing nonimportant signals for classification by utilizing features gradient obtained by the calculation process of grad-CAM. When we define N as the number of signals, the computational complexity of the proposed algorithm is linear time O(N), that is, it has a low calculation cost. We verified the effectiveness of the algorithm using the OPPORTUNITY Activity Recognition dataset, which is an open dataset comprising acceleration signals of human activities. In addition, we checked the average 6.55 signals from a total of 15 acceleration signals (five triaxial sensors) that were removed by FG-SSA while maintaining high generalization scores of classification. Therefore, the proposed algorithm FG-SSA has an effect on finding and removing signals that are not important for CNN-based classification.

eess.SP↗
arXiv2022

Theoretical Analysis of SIRVVD Model to Provide Insight on the Target Rate of COVID-19/SARS-CoV-2 Vaccination in Japan

The effectiveness of the first and second dose vaccinations are different for COVID-19; therefore, a susceptible-infected-recovered-vaccination1-vaccination2-death (SIRVVD) model that can represent the states of the first and second vaccination doses has been proposed. By the previous study, we can carry out simulating the spread of infectious disease considering the effects of the first and second doses of the vaccination based on the SIRVVD model. However, theoretical analysis of the SIRVVD Model is insufficient. Therefore, we obtained an analytical expression of the infectious number, by treating the numbers of susceptible persons and vaccinated persons as parameters. We used the solution to determine the target rate of the vaccination for decreasing the infection numbers of the COVID-19 Delta variant (B.1.617) in Japan. Further, we investigated the target vaccination rates for cases with strong or weak variants by comparison with the COVID-19 Delta variant (B.1.617). This study contributes to the mathematical development of the SIRVVD model and provides insight into the target rate of the vaccination to decrease the number of infections.

q-bio.PE↗
arXiv2020

Effectiveness of the COVID-19 Contact-Confirming Application (COCOA) based on a Multi Agent Simulation

As of Aug. 2020, coronavirus disease 2019 (COVID-19) is still spreading in the world. In Japan, the Ministry of Health, Labor, and Welfare developed "COVID-19 Contact-Confirming Application (COCOA)," which was released on Jun. 19, 2020. By utilizing COCOA, users can know whether or not they had contact with infected persons. If those who had contact with infectors keep staying at home, they may not infect those outside. However, effectiveness decreasing the number of infectors depending on the app's various usage parameters is not clear. If it is clear, we could set the objective value of the app's usage parameters (e.g., the usage rate of the total populations) and call for installation of the app. Therefore, we develop a multi-agent simulator that can express COVID-19 spreading and usage of the apps, such as COCOA. In this study, we describe the simulator and the effectiveness of the app in various scenarios. The result obtained in this study supports those of previously conducted studies.

cs.CY↗