Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.04376

Optimal Community Recovery by Spectrally Initialized Variational EM in General Stochastic Block Models

Jiangzhou Wang
View original source ↗Read source document ↗Export metadata ↓

Abstract

We prove a Chernoff-exponent guarantee for the output of spectrally initialized batch variational EM after a prescribed iteration budget, without assuming global maximization of its variational objective. The iteration repeatedly estimates the entire block probability matrix and community proportions from the same sparse graph. We consider a fixed number of communities with proportions bounded away from zero and fixed, positive, distinct connectivity profiles; neither assortativity nor full rank is required. The algorithm uses simultaneous softmax updates without sample splitting or posterior thresholding. Its analysis must control the feedback between estimated parameters, soft labels and reused edges at an exponentially small risk scale. We establish a uniform one-step bound over data-dependent soft assignments whose random remainder has exponentially small expectation. Combined with an exponentially reliable regularized spectral initializer, this yields an unconditional expected misclassification rate bounded by $\exp\{-(1-o(1))J_n\}$ throughout the sparse, diverging-degree regime, where $J_n$ is the minimum nodewise Chernoff information. Matching lower bounds establish first-order logarithmic minimax optimality on local parameter spaces allowing unknown connectivity and varying community counts. The algorithm also attains the sharp first-order exact-recovery threshold. Numerical experiments illustrate refinement gains and sensitivity to initialization and imbalance.

Explore related subjects

math.STstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jiangzhou Wang. 2026-10-03. Optimal Community Recovery by Spectrally Initialized Variational EM in General Stochastic Block Models. https://arxiv.org/abs/2610.04376

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A resolution of the Borel-Kolmogorov Paradox via the Maximum Entropy Principle

Bayesian updating routinely conditions on events of prior probability zero, such as an exact observation of a continuous parameter or a parameter confined to a submanifold. Two equally natural parametrizations of the same event can then return different posteriors, which is the Borel--Kolmogorov paradox. We add a metric to the probabilistic model and define the posterior given a closed null set as the limit of Maximum Entropy solutions under a vanishing constraint on the distance to that set. We give a sufficient condition for existence and prove that, given this metric, the posterior is unique and invariant under measure-preserving isometries. It recovers the textbook Bayes formulas where these apply unambiguously, on positive-measure sets and in Euclidean models. Each stage of the limit has an explicit density that depends only on the distance to the conditioning set, so the posterior can be computed with standard tools for Kullback--Leibler minimization. The construction does not remove the choice of a metric but makes it an explicit modelling input. In a controlled experiment where the geometry of the measurement is known, we show that a Bayesian analysis ignoring this geometry can yield uncalibrated posteriors. On real palaeomagnetic data, where the geometry is disputed, we show how candidate metrics can be compared within the same Bayesian framework.

math.ST↗
arXiv2026

A Two-Sample Test on Weighted Persistence Intensity Functions in Topological Data Analysis

Persistence intensity functions provide interpretable and informative first-order summaries of random persistence diagram distributions. We study two-sample testing for equality of persistence intensity functions, allowing the underlying diagram distributions to differ under the null. We construct a weighted-kernel statistic as an unbiased estimator of the squared reproducing-kernel Hilbert space distance between the corresponding weighted intensity embeddings, and calibrate it by studentization. For shrinking bandwidths, we establish uniform asymptotic normality under the null and thereby obtain asymptotic Type I error control. Its power is characterized in terms of the $L^2$ discrepancy between the weighted intensity functions. To accommodate persistence diagrams with possibly unbounded cardinality, we introduce regularity conditions that control the effect of cardinality variation and yield the desired moment bounds for the test statistic. We further show that every probability density on $\{(x,y)\in\mathbb{R}^2:y>x > 0\}$ can be realized as the persistence intensity function of a random diagram. Using these results, we establish that the proposed test attains minimax-optimal separation rates over anisotropic Sobolev balls. Lastly, since the optimal bandwidth is not directly accessible in practice, we adapt a bandwidth aggregation framework.

math.ST↗
arXiv2026

Human-Anchored Inference for Ranking New Models with Large Language Model Judges

Human pairwise comparisons provide a reference for evaluating large language models (LLMs), but collecting sufficient judgments for each new release is costly and time-consuming. LLM judges offer a scalable alternative, although their comparisons may differ systematically from human preferences and across judges. We study the ranking of a new model that has received LLM-judge comparisons but no human comparisons. We propose ANCHOR (ANchored Comparisons for Human-reference inference with Orthogonal Riesz correction), which uses historical human and LLM comparisons to learn judge-specific sensitivities to human score differences and feature-dependent judge biases. These estimates are then used to infer the new model's human-reference score from its judge comparisons. The framework allows the feature distribution to change between historical and new-model comparisons. For inference, we construct a Neyman-orthogonal estimator through a joint Riesz correction that removes the first-order effects of estimating the historical human scores, judge sensitivities, and bias functions. We establish identification, convergence rates, and asymptotic normality with consistently estimable variance, and show that ANCHOR attains the semiparametric efficiency bound. Simulations demonstrate gains in score estimation and ranking accuracy. On Chatbot Arena, ANCHOR achieves the lowest score RMSE and insertion MAE among competing methods, with narrower score intervals on average.

math.ST↗