arXiv subjects

Takashi Takahashi

Publications and source records attributed to Takashi Takahashi.

At least 19 recordsLinked to original sources

arXiv2026

Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data

Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric $α$-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.

cond-mat.dis-nn↗

arXiv2026

Dynamical Regimes of Discrete Diffusion Models

Diffusion models generate high-dimensional data such as images by learning a process that gradually removes noise from corrupted data. Recent studies have shown that the backward dynamics of diffusion models exhibit two characteristic transitions: the speciation transition, at which generated samples begin to capture the global structure of the training data, and the collapse transition, at which the generation dynamics starts committing to individual training samples. While these transitions have been theoretically analyzed for continuous data, the same theoretical criteria have not been applied for discrete diffusion models, which are diffusion models for discrete data. In this work, we propose a simple effective model for discrete diffusion models trained on two-class Ising variable data with a general mixture ratio and analyze its backward dynamics using methods from statistical mechanics. We show that, as in the previous study on continuous data, the speciation transition can be determined through a second-order phase transition analysis using high-temperature expansion, while the collapse transition corresponds to a condensation transition described by the Random Energy Model. An analytical expression for the speciation time is obtained, and we show that its scaling becomes consistent with that of the continuous case when the noise increases with time as in practical diffusion models. These theoretical predictions are confirmed by numerical simulations and experiments with trained discrete diffusion models on real datasets. These results suggest that the original theoretical framework for continuous data remain valid for discrete data, and may provide a useful starting point for the statistical-mechanics analysis of discrete generative diffusion in more realistic settings.

cond-mat.stat-mech↗

arXiv2026

The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Data

Self-training (ST) is a simple yet effective semi-supervised learning method. However, why and how ST improves generalization performance by using potentially erroneous pseudo-labels is still not well understood. To deepen the understanding of ST, we derive and analyze a sharp characterization of the behavior of iterative ST when training a linear classifier by minimizing the ridge-regularized convex loss on binary Gaussian mixtures, in the asymptotic limit where input dimension and data size diverge proportionally. The results show that ST improves generalization in different ways depending on the number of iterations. When the number of iterations is small, ST improves generalization performance by fitting the model to relatively reliable pseudo-labels and updating the model parameters by a large amount at each iteration. This suggests that ST works intuitively. On the other hand, with many iterations, ST can gradually improve the direction of the classification plane by updating the model parameters incrementally, using soft labels and small regularization. It is argued that this is because the small update of ST can extract information from the data in an almost noiseless way. However, in the presence of label imbalance, the generalization performance of ST underperforms supervised learning with true labels. To overcome this, two heuristics are proposed to enable ST to achieve nearly compatible performance with supervised learning even with significant label imbalance.

stat.ML↗

arXiv2026

Analysis of the Hopfield Model Incorporating the Effects of Unlearning

We analyze a variant of the Hopfield model that incorporates an unlearning mechanism based on spin correlations in the high-temperature regime. In the large system limit where extensively many patterns are stored, we employ the replica method under the replica symmetric ansatz to characterize the model analytically. Our analysis provides a systematic and self-consistent framework that yields order-parameter equations and stability conditions at finite temperatures over a wide range of parameter settings. The resulting theory accurately captures the behavior of the signal-to-noise ratio, the memory capacity, and the criteria for selecting optimal hyperparameters, in agreement with the qualitative findings of Nokura (1996 \textit{J. Phys. A: Math. Gen.} \textbf{29} 3871). Moreover, the theoretical predictions show good agreement with numerical simulations, supporting the conclusion that unlearning enhances memory capacity by suppressing spurious memories.

cond-mat.dis-nn↗

arXiv2026

Learning Linear Regression with Low-Rank Tasks in-Context

In-context learning (ICL) is a key building block of modern large language models, yet its theoretical mechanisms remain poorly understood. It is particularly mysterious how ICL operates in real-world applications where tasks have a common structure. In this work, we address this problem by analyzing a linear attention model trained on low-rank regression tasks. Within this setting, we precisely characterize the distribution of predictions and the generalization error in the high-dimensional limit. Moreover, we find that statistical fluctuations in finite pre-training data induce an implicit regularization. Finally, we identify a sharp phase transition of the generalization error governed by task structure. These results provide a framework for understanding how transformers learn to learn the task structure.

cond-mat.dis-nn↗

arXiv2026

Type-I and Type-II Saddle Points and a Topological Flat Band in a Bi-Pyrochlore Superconductor CsBi2

The divergence of the electron density of states (DOS) plays an important role in enhancing many-body interactions and inducing various quantum phases in low-dimensional systems. However, such unique electronic structures remain experimentally elusive in three-dimensional (3D) systems, particularly those with strong spin-orbit coupling (SOC). Using angle-resolved photoemission spectroscopy and first-principles calculations for a Laves-phase superconductor CsBi$_2$, which features a Bi-pyrochlore 3D network with strong SOC, we identify two characteristic electronic structures with a large DOS. One is a dispersionless topological flat band with p-orbital character, locally formed around the U-K line, which enhances DOS near the Fermi level. The other involves type-I and type-II saddle points connected by a flat band, which cooperatively produce an enhancement in the DOS. Our findings suggest a novel mechanism for achieving a DOS enhancement and lay a foundation for exploring exotic phenomena driven by the interplay of multiple singularities with a large DOS, nontrivial topology, and strong SOC in 3D pyrochlores.

cond-mat.mes-hall↗

arXiv2025

The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model

Self-distillation (SD), a technique where a model improves itself using its own predictions, has attracted attention as a simple yet powerful approach in machine learning. Despite its widespread use, the mechanisms underlying its effectiveness remain unclear. In this study, we investigate the efficacy of hyperparameter-tuned multi-stage SD with a linear classifier for binary classification on noisy Gaussian mixture data. For the analysis, we employ the replica method from statistical physics. Our findings reveal that the primary driver of SD's performance improvement is denoising through hard pseudo-labels, with the most notable gains observed in moderately sized datasets. We also identify two practical heuristics to enhance SD: early stopping that limits the number of stages, which is broadly effective, and bias parameter fixing, which helps under label imbalance. To empirically validate our theoretical findings derived from our toy model, we conduct additional experiments on CIFAR-10 classification using pretrained ResNet backbone. These results provide both theoretical and practical insights, advancing our understanding and application of SD in noisy settings.

stat.ML↗

arXiv2025

Variational Gaussian Approximation in Replica Analysis of Parametric Models

We revisit the replica method for analyzing inference and learning in parametric models, considering situations where the data-generating distribution is unknown or analytically intractable. Instead of assuming idealized distributions to carry out quenched averages analytically, we use a variational Gaussian approximation for the replicated system in grand canonical formalism in which the data average can be deferred and replaced by empirical averages, leading to stationarity conditions that adaptively determine the parameters of the trial Hamiltonian for each dataset. This approach clarifies how fluctuations affect information extraction and connects directly with the results of mathematical statistics or learning theory such as information criteria. As a concrete application, we analyze linear regression and derive learning curves. This includes cases with real-world datasets, where exact replica calculations are not feasible.

cond-mat.dis-nn↗

arXiv2025

Asymptotic Dynamics of Alternating Minimization for Bilinear Regression

This study investigates the dynamics of alternating minimization applied to a bilinear regression task with normally distributed covariates, under the asymptotic system size limit where the number of parameters and observations diverge at the same rate. This is achieved by employing the replica method to a multi-temperature glassy system which unfolds the algorithm's time evolution. Our results show that the dynamics can be described effectively by a two-dimensional discrete stochastic process, where each step depends on all previous time steps, revealing the structure of the memory dependence in the evolution of alternating minimization. The theoretical framework developed in this work can be applied to the analysis of various iterative algorithms, extending beyond the scope of alternating minimization.

math.OC↗

arXiv2025

Unusual temperature dependence of the band structure associated with local atomic distortion in monolayer 1T'-WTe2

The ground state of monolayer 1T'-WTe2 has been a target of intensive debate on whether or not it is a two-dimensional topological insulator (2D TI) associated with exciton formation. We investigated the band structure of an epitaxial monolayer 1T'-WTe2 film grown on graphene/SiC(0001) in a wide temperature range of T = 40 - 400 K by angle-resolved photoemission spectroscopy (ARPES). We observed an electron band above the Fermi level (EF) slightly away from the Γ point, together with four hole bands below EF just at the Γ point. This signifies an indirect band gap exceeding 0.1 eV in support of the 2D-TI phase with the inverted band structure. We uncovered an unexpectedly large downward shift of valence bands upon cooling, accompanied with an upward shift of the conduction band. Comparison of the ARPES-derived band structure with first-principles band calculations suggests that the observed band shift is ascribed to the systematic local atomic distortion of tungsten atoms, which should be incorporated into the interpretation of unusual transport properties of 1T'-WTe2.

cond-mat.mtrl-sci↗

arXiv2025

Selective Fabrication of Monolayer 1H- and 1T'-WTe2

We selectively fabricated monolayers of octahedral (1H) and distorted trigonal (1T') WTe2 on graphene/SiC(0001) by controlling the substrate temperature during epitaxy. Angle-resolved photoemission spectroscopy, combined with first-principles band-structure calculations, has revealed several drastic differences between these two polymorphs. The 1T' phase exhibits a semiconducting character with a nearly-zero energy gap, while the 1H phase shows a large band gap and the band splitting at the K/K' point. The present results pave a pathway toward developing nanoelectronic devices based with WTe2.

cond-mat.mes-hall↗

arXiv2024

Universal Role of Combined Symmetry for the Protection of the Dirac Cone in Antiferromagnetic Topological Insulators

Antiferromagnetic topological insulators (AF TIs) are predicted to exhibit exotic physical properties such as gigantic optical and topological magnetoelectric responses. While a key to achieving such phenomena relies on how to break the symmetry protecting the Dirac-cone surface state (SS) and acquire the mass of Dirac fermions, the mechanism has yet to be clarified. To address this issue, we carried out micro-focused angle-resolved photoemission spectroscopy for GdBi hosting the type-II AF order, and uncovered the stripe-type 2$\times$1 reconstruction of the Fermi surface associated with the AF band folding. Intriguingly, in contrast to NdBi with the type-I AF order displaying the surface-selective Dirac-fermion mass, GdBi shows massless behavior irrespective of AF domains due to the robust topological protection. These results strongly suggest a crucial role of the ThetaTD (time-reversal and translational) symmetry to create the Dirac-fermion mass in AF TIs.

cond-mat.mes-hall↗

arXiv2024

Replica Analysis for Ensemble Techniques in Variable Selection

Variable selection is a problem of statistics that aims to find the subset of the $N$-dimensional possible explanatory variables that are truly related to the generation process of the response variable. In high-dimensional setups, where the input dimension $N$ is comparable to the data size $M$, it is difficult to use classic methods based on $p$-values. Therefore, methods based on the ensemble learning are often used. In this review article, we introduce how the performance of these ensemble-based methods can be systematically analyzed using the replica method from statistical mechanics when $N$ and $M$ diverge at the same rate as $N,M\to\infty, M/N\toα\in(0,\infty)$. As a concrete application, we analyze the power of stability selection (SS) and the derandomized knockoff (dKO) with the $\ell_1$-regularized statistics in the high-dimensional linear model. The result indicates that dKO provably outperforms the vanilla knockoff and the standard SS, while increasing the bootstrap resampling rate in SS might further improve the detection power.

math.ST↗

arXiv2024

A replica analysis of under-bagging

Under-bagging (UB), which combines under-sampling and bagging, is a popular ensemble learning method for training classifiers on an imbalanced data. Using bagging to reduce the increased variance caused by the reduction in sample size due to under-sampling is a natural approach. However, it has recently been pointed out that in generalized linear models, naive bagging, which does not consider the class imbalance structure, and ridge regularization can produce the same results. Therefore, it is not obvious whether it is better to use UB, which requires an increased computational cost proportional to the number of under-sampled data sets, when training linear models. Given such a situation, in this study, we heuristically derive a sharp asymptotics of UB and use it to compare with several other popular methods for learning from imbalanced data, in the scenario where a linear classifier is trained from a two-component mixture data. The methods compared include the under-sampling (US) method, which trains a model using a single realization of the under-sampled data, and the simple weighting (SW) method, which trains a model with a weighted loss on the entire data. It is shown that the performance of UB is improved by increasing the size of the majority class while keeping the size of the minority fixed, even though the class imbalance can be large, especially when the size of the minority class is small. This is in contrast to US, whose performance is almost independent of the majority class size. In this sense, bagging and simple regularization differ as methods to reduce the variance increased by under-sampling. On the other hand, the performance of SW with the optimal weighting coefficients is almost equal to UB, indicating that the combination of reweighting and regularization may be similar to UB.

stat.ML↗

arXiv2024

Evolution of Band Structure in a Kagome Superconductor Cs(V1-xCrx)3Sb5: Toward Universal Understanding of CDW and Superconducting Phase Diagrams

Kagome superconductors AV3Sb5 (A = K, Rb, Cs) exhibit a characteristic superconducting and charge-density wave (CDW) phase diagram upon carrier doping and chemical substitution. However, the key electronic states responsible for such a phase diagram have yet to be clarified. Here we report a systematic micro-focused angle-resolved photoemission spectroscopy (ARPES) study of Cs(V1-xCrx)3Sb5 as a function of Cr content x, where Cr substitution causes monotonic reduction of superconducting and CDW transition temperatures. We found that the V-derived bands forming saddle points at the M point and Dirac nodes along high-symmetry cuts show an energy shift due to electron doping by Cr substitution, whereas the Sb-derived electron band at the Gamma point remains almost unchanged, signifying an orbital-selective band shift. We also found that band doubling associated with the emergence of three-dimensional CDW identified at x = 0 vanishes at x = 0.25, in line with the disappearance of CDW. A comparison of band diagrams among Ti-, Nb-, and Cr-substituted Cs(V1-xCrx)3Sb5 suggests the importance to simultaneously take into account the two saddle points at the M point and their proximity to the Fermi energy, to understand the complex phase diagram against carrier doping and chemical pressure.

cond-mat.supr-con↗

arXiv2023

Unusual surface states associated with the PT-symmetry breaking and antiferromagnetic band folding in NdSb

We have performed micro-focused angle-resolved photoemission spectroscopy on NdSb which exhibits the type-I antiferromagnetism below TN = 16 K. We succeeded in selectively observing the band structure for all three types of single-q antiferromagnetic (AF) domains at the surface. We found that two of the three surfaces whose AF-ordering vector lies within the surface plane commonly show twofold symmetric surface states (SSs) around the bulk-band edges, whereas the other surface with an out-of-plane AF-ordering vector displays fourfold symmetric shallow electronlike SS at the Brillouin-zone center. We suggest that these SSs commonly originate from the combination of the PT (space-inversion and time-reversal) symmetry breaking at the surface and the band folding due to the AF order. The present results pave a pathway toward understanding the relationship between the symmetry and the surface electronic states in antiferromagnets.

cond-mat.mes-hall↗

arXiv2023

Compressed Sensing Radar Detectors based on Weighted LASSO

The compressed sensing (CS) model can represent the signal recovery process of a large number of radar systems. The detection problem of such radar systems has been studied in many pieces of literature through the technology of debiased least absolute shrinkage and selection operator (LASSO). While naive LASSO treats all the entries equally, there are many applications in which prior information varies depending on each entry. Weighted LASSO, in which the weights of the regularization terms are tuned depending on the entry-dependent prior, is proven to be more effective with the prior information by many researchers. In the present paper, existing results obtained by methods of statistical mechanics are utilized to derive the debiased weighted LASSO estimator for randomly constructed row-orthogonal measurement matrices. Based on this estimator, we construct a detector, termed the debiased weighted LASSO detector (DWLD), for CS radar systems and prove its advantages. The threshold of this detector can be calculated by false alarm rate, which yields better detection performance than the naive weighted LASSO detector (NWLD) under the Neyman-Pearson principle. The improvement of the detection performance brought by tuning weights is demonstrated by numerical experiments. With the same false alarm rate, the detection probability of DWLD is obviously higher than those of NWLD and the debiased (non-weighted) LASSO detector (DLD).

eess.SP↗

arXiv2023

Role of Bootstrap Averaging in Generalized Approximate Message Passing

Generalized approximate message passing (GAMP) is a computationally efficient algorithm for estimating an unknown signal $w_0\in\mathbb{R}^N$ from a random linear measurement $y= Xw_0 + ε\in\mathbb{R}^M$, where $X\in\mathbb{R}^{M\times N}$ is a known measurement matrix and $ε$ is the noise vector. The salient feature of GAMP is that it can provide an unbiased estimator $\hat{r}^{\rm G}\sim\mathcal{N}(w_0, \hat{s}^2I_N)$, which can be used for various hypothesis-testing methods. In this study, we consider the bootstrap average of an unbiased estimator of GAMP for the elastic net. By numerically analyzing the state evolution of \emph{approximate message passing with resampling}, which has been proposed for computing bootstrap statistics of the elastic net estimator, we investigate when the bootstrap averaging reduces the variance of the unbiased estimator and the effect of optimizing the size of each bootstrap sample and hyperparameter of the elastic net regularization in the asymptotic setting $M, N\to\infty, M/N\toα\in(0,\infty)$. The results indicate that bootstrap averaging effectively reduces the variance of the unbiased estimator when the actual data generation process is inconsistent with the sparsity assumption of the regularization and the sample size is small. Furthermore, we find that when $w_0$ is less sparse, and the data size is small, the system undergoes a phase transition. The phase transition indicates the existence of the region where the ensemble average of unbiased estimators of GAMP for the elastic net norm minimization problem yields the unbiased estimator with the minimum variance.

cs.IT↗