Searcharxiv⌕ Search

arXiv · 2611.12647

Near-Purity Screening and Dominant-Atom Estimation in Softmax Mixtures and Their Ensembles

Abstract

We study near-purity screening and dominant-atom estimation in softmax mixtures, whose components share a common set of category features. A mixture is called $\varepsilon$-pure if one of its mixing weights is at least $1-\varepsilon$. We introduce the Trace of Latent Covariance Criterion ($\mathrm{TLC}^{2}$), a direct estimator of the trace of the covariance matrix of the latent mixing measure, constructed without estimating its atoms or weights, or knowing the number of components. Unlike existing moment-based approaches, our estimator is constructed for the softmax family and does not require knowledge or estimation of the feature distribution. We establish finite-sample guarantees for $\mathrm{TLC}^{2}$, showing that it achieves a parametric convergence rate up to an additive approximation error of order $p^{-1/2}$, without requiring minimum separation between mixture atoms. We also study estimation of the dominant atom in near-pure mixtures by fitting a single softmax component. We prove that the resulting maximum likelihood estimator has an error bound consisting of the usual parametric term and a bias term governed by the weighted distances between the dominant and remaining atoms, revealing adaptation to the geometry of the mixture. Finally, we extend near-purity screening and dominant-atom estimation to ensembles of softmax mixtures with shared atoms and mixture-specific weights. We apply $\mathrm{TLC}^{2}$ separately to each ensemble member to screen for near-purity without knowing the number of components. Under additional assumptions, clustering the resulting dominant-atom estimators yields finite-sample guarantees for estimating the shared atoms. An application to the Million Song Dataset illustrates how the estimated softmax directions reveal interpretable structure in the data.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Xin Bing, Florentina Bunea, Marten Wegkamp, Weicheng Wu. 2026-11-09. Near-Purity Screening and Dominant-Atom Estimation in Softmax Mixtures and Their Ensembles. https://searcharxiv.com/abs/2611.12647

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning

Across a wide array of disciplines, many researchers use machine learning (ML) algorithms to identify a subgroup of individuals who are likely to benefit from a treatment the most (``exceptional responders'') or those who are harmed by it. A common approach to this subgroup identification problem consists of two steps. First, researchers estimate the conditional average treatment effect (CATE) using an ML algorithm. Next, they use the estimated CATE to select those individuals who are predicted to be most affected by the treatment, either positively or negatively. Unfortunately, CATE estimates are often biased and noisy. In addition, utilizing the same data to both identify a subgroup and estimate its group average treatment effect results in a multiple testing problem. To address these challenges, we develop uniform confidence bands for estimation of the group average treatment effect sorted by generic ML algorithm (GATES). Using these uniform confidence bands, researchers can identify, with a statistical guarantee, a subgroup whose GATES exceeds a certain effect size, regardless of how this effect size is chosen. The proposed methodology uses random treatment assignment, random sampling of units, and continuity and moment conditions, without requiring consistency or monotonicity of the ML score. We derive a Wiener representation of the estimation error, with corrections for the empirical subgroup threshold and the fixed treatment sample size. Estimating this representation gives asymptotically exact uniform coverage using Gaussian draws, without refitting the ML model. We also develop uniform bands for mean-adjusted treatment effects and other evaluation metrics. A simulation design and an application to a clinical trial of late-stage prostate cancer illustrate the subgroup identification problem.

stat.ME↗

Frequentist forecasting in regime-switching models with extended Hamilton filter

Psychological change processes, such as university student dropout in math, often exhibit discrete latent state transitions and can be studied using regime-switching models with intensive longitudinal data (ILD). Recently, regime-switching state-space (RSSS) models have been extended to allow for latent variables and their autoregressive effects. Despite this progress, there is a need for frequentist estimation methods in dynamic latent variable frameworks that handle both intra-individual changes and inter-individual differences as predictors of regime-switches and allow real-time inferences and forecasts during ongoing data collection. Building on Chow and Zhang's (2013) extended Kim filter, we introduce a first frequentist filter for RSSS models which allows hidden Markov(-switching) models to depend on both latent within- and between-individual characteristics. As a counterpart of Kelava et al.'s (2022) Bayesian forecasting filter for nonlinear dynamic latent class structural equation models (NDLC-SEM), our method is the first frequentist approach within this general class of models. In an empirical study, the filter is applied to forecast emotions and behavior related to student dropout in math. A simulation study shows that marginalizing person-specific random intercepts within the filter cleanly separates autoregressive persistence from stable between-person differences, with accurate out-of-sample regime forecasts, while within-person transition effects stay weakly identified.

stat.ME↗

PRESCCO: Efficient Prediction Intervals under a Right-Censored Covariate

In neurodegenerative diseases, atypical patients can reveal disease-modifying biology. A patient is atypical when their outcome falls outside a prediction interval for patients at the same point in the disease, measured by the time to a disease event. Most patients leave the study before the event, so their time to it is right-censored, and no method computes a prediction interval with a right-censored covariate. We develop the first methods. We adapt three conformal prediction methods and show that their estimated half-length, half the interval's width, varies across studies, changing which patients are identified as atypical. We then develop the PRESCCO method. We prove its estimator of the half-length is doubly robust, consistent when one of its time-to-event and censoring models is misspecified, and semiparametrically efficient when both are correct. At a 90% prediction level, its prediction interval identifies close to 10% of patients as atypical, across studies and, with high probability, in a single study. In simulations, its estimated half-length is four to thirty times less variable than under conformal prediction. In Huntington disease data with 77.2% right-censoring, the PRESCCO method identifies close to the targeted 10% of patients as atypical, whereas conformal prediction identifies too many or too few.

stat.ME↗