subject

stat.TH

stat.TH: explore 76 source-linked works published from 2026 to 2026, with original documents and citations.

This collection is a preview while coverage and quality are evaluated.

Coverage and selection

Includes records with this source-supplied label or an explicit phrase match in their metadata. Matches indicate a mention, not proof that a paper uses a method or tests a material. Source versions are consolidated by DOI.

Sources: arxiv. Collection updated 2026-09-16. Counts describe this index, not the complete source archives.

arXiv2026

Coherent Information Deletion: Bayesian, Generalized Bayesian and Possibilistic Unlearning

Bayes' theorem admits an information processing interpretation due to Zellner (1988): under the Shannon information criterion, the posterior is the unique rule that processes prior and data information without information loss. We revisit these ideas from the perspective of information deletion. Given a posterior based on a complete dataset, what distribution should replace it when a subset of the data is removed? For ordinary Bayesian inference, we show that the corresponding information deletion objective is uniquely minimized by the leave-data-out posterior. We then extend the framework beyond likelihood-based inference from Bayes to the generalized Bayesian updating of Bissiri et al.(2016} based on loss functions. Requiring deletion to be coherent under sequential removal of loss contributions yields a deletion rule that exactly recovers the generalized posterior based only on the retained information. We further develop an axiomatic theory, beyond probabilistic inference, for possibilistic deletion and show that the same information-deletion perspective leads to a principled rule for exact possibilistic unlearning. Finally, restricting these exact deletion principles to tractable families leads naturally to variational Bayesian, generalized Bayesian, and maxitive possibilistic unlearning.

stat.ME↗

arXiv2026

The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Data

Self-training (ST) is a simple yet effective semi-supervised learning method. However, why and how ST improves generalization performance by using potentially erroneous pseudo-labels is still not well understood. To deepen the understanding of ST, we derive and analyze a sharp characterization of the behavior of iterative ST when training a linear classifier by minimizing the ridge-regularized convex loss on binary Gaussian mixtures, in the asymptotic limit where input dimension and data size diverge proportionally. The results show that ST improves generalization in different ways depending on the number of iterations. When the number of iterations is small, ST improves generalization performance by fitting the model to relatively reliable pseudo-labels and updating the model parameters by a large amount at each iteration. This suggests that ST works intuitively. On the other hand, with many iterations, ST can gradually improve the direction of the classification plane by updating the model parameters incrementally, using soft labels and small regularization. It is argued that this is because the small update of ST can extract information from the data in an almost noiseless way. However, in the presence of label imbalance, the generalization performance of ST underperforms supervised learning with true labels. To overcome this, two heuristics are proposed to enable ST to achieve nearly compatible performance with supervised learning even with significant label imbalance.

stat.ML↗

arXiv2026

From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesis

We inspect the deductive connection between the neural scaling law and Zipf's law -- two statements discussed in machine learning and quantitative linguistics. The neural scaling law describes how the cross entropy rate of a foundation model -- such as a large language model -- changes with respect to the amount of training tokens, parameters, and compute. By contrast, Zipf's law posits that the distribution of tokens exhibits a power law tail. Whereas similar claims have been made in more specific settings, we show that the neural scaling law is a consequence of Zipf's law under certain broad assumptions that we reveal systematically. The derivation steps are as follows: We derive Heaps' law on the vocabulary growth from Zipf's law, Hilberg's hypothesis on the entropy scaling from Heaps' law, and the neural scaling from Hilberg's hypothesis. We illustrate these inference steps by a toy example of the Santa Fe process that satisfies all four statistical laws.

cs.IT↗

arXiv2026

Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decay

The generalization error curve of certain kernel regression method aims at determining the exact order of generalization error with various source condition, noise level and choice of the regularization parameter rather than the minimax rate. In this work, under mild assumptions, we rigorously provide a full characterization of the generalization error curves of the kernel gradient descent method (and a large class of analytic spectral algorithms) in kernel regression. Consequently, we could sharpen the near inconsistency of kernel interpolation and clarify the saturation effects of kernel regression algorithms with higher qualification, etc. Thanks to the neural tangent kernel theory, these results greatly improve our understanding of the generalization behavior of training the wide neural networks. A novel technical contribution, the analytic functional argument, might be of independent interest.

cs.LG↗

Compare source metadata on this page

Work	Published	Source identifier	Source
Coherent Information Deletion: Bayesian, Generalized Bayesian and Possibilistic Unlearning	2026-10-12	2602.09061	arxiv
The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Data	2026-08-09	2205.07739	arxiv
From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesis	2026-06-22	2512.13491	arxiv
Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decay	2026-06-08	2401.01599	arxiv

These are bibliographic comparisons, not experimental rankings. Follow the original document for methods and conditions.