arXiv subjects

Tamir Bendory

Publications and source records attributed to Tamir Bendory.

At least 19 recordsLinked to original sources

arXiv2026

A Geometric Analysis of Initialization Bias in Spherical $K$-means in the Weak Signal Regime

We study initialization bias in spherical $K$-means for weakly informative directional mixtures. We model the observations by a $K$-component von Mises-Fisher mixture with a small concentration parameter $\kappa$, corresponding to a high-dispersion regime in which the data provide limited information about the underlying directions. Our analysis begins with the limiting case $\kappa=0$ (corresponding to a uniform distribution over the sphere), where one population spherical $K$-means update is governed entirely by the Voronoi tessellation induced by the initialized templates. For uniformly random initializations in fixed dimension $d$, the updated templates become asymptotically aligned with their initial values as $K\to\infty$: the average squared geodesic error scales as $O(K^{-2/(d-1)})$, while the worst-case error is $O((\log K/K)^{2/(d-1)})$. We then show that, in the weak-signal regime of small positive $\kappa$, the population update remains an $O(\kappa)$ perturbation of this limiting map. Thus, in the weak-signal regime, spherical $K$-means can preserve initialization-induced structure despite the presence of a genuine but highly dispersed directional signal.

eess.SP↗

arXiv2026

The generalized method of moments is (almost) statistically efficient in low-SNR Gaussian latent-variable models

We study estimation in the low signal-to-noise ratio (SNR) regime for a broad class of Gaussian latent-variable models, including Gaussian mixtures and orbit recovery problems. We show that, in this regime, the generalized method-of-moments (GMoM) matches the first-order asymptotic efficiency of maximum likelihood. In particular, if the moment features are chosen up to the minimal local order required for identification and are weighted optimally, then the resulting GMoM estimator has the same leading asymptotic covariance as the maximum-likelihood estimator. Our analysis shows that, in low SNR, this equivalence is governed by a layered local geometry: different directions become informative at different moment orders, partitioning the space into layers with distinct SNR scalings. We prove that the observed Fisher information and the GMoM information operator admit matching layerwise expansions across these layers. As a consequence, in the low-SNR regime, GMoM provides a statistically efficient alternative to maximum likelihood, while preserving the computational advantages of moment-based estimation.

math.ST↗

arXiv2026

Projected multi-reference alignment

Motivated by structural biology applications, we study the projected multi-reference alignment (MRA) model, in which an unknown signal is observed through noisy samples, each generated by applying a random cyclic shift followed by a fixed projection. The projection merges reflection-symmetric index pairs, thereby discarding orientation information. The goal is to recover the dihedral orbit of the signal. We prove that in the high-noise regime, the first three moments of the projected observations determine a generic dihedral orbit. The main mechanism is a reduction, at the moment level, from projected MRA to the reflection-invariant phase-coupling structure of dihedral MRA. In Fourier-cosine coordinates adapted to the projection, the first moment determines the mean component, the second moment determines the Fourier magnitudes, and selected third moments yield the cosine phase-coupling relations appearing in the dihedral bispectrum. These relations lead to a constructive recovery scheme from moments up to order three. We complement the population theory with finite-sample experiments comparing expectation--maximization (EM), direct moment optimization, and direct Fourier-cosine moment optimization. The results show that, in the high-noise regime, both EM and direct moment optimization are consistent with the predicted third-moment sample-complexity scaling $n \gtrsim \sigma^6$, where $n$ is the number of observations and $\sigma^2$ is the noise variance.

eess.SP↗

arXiv2026

The interplay of signal-to-noise ratio and variance misspecification in Gaussian mixtures

We study estimation and clustering in Gaussian mixture models under variance misspecification. Observations are generated with true variance $\sigma^2$, while the component means are estimated using a likelihood with variance $\tau^2$, yielding a family of mismatched likelihood functions parameterized by the ratio $\rho=\tau/\sigma$. We show that the interplay between $\rho$ and the signal-to-noise ratio (SNR) induces a sharp phase diagram. Under correct specification ($\rho=1$), maximum likelihood recovers the true means, independently of the SNR. However, once the model is misspecified, two different regimes emerge. Under under-smoothing ($\rho<1$), the estimated Gaussian means are displaced from the truth, and in low SNR this discrepancy grows as the SNR decreases: for every fixed $\rho<1$, the squared error scales as $\mathrm{SNR}^{-1}$. Under over-smoothing ($\rho>1$), the fitted likelihood blurs the cluster separation, causing distinct component means to collapse towards the overall mixture center once $\rho^2$ exceeds a threshold of the form $1 + \lambda\,\mathrm{SNR}$, where $\lambda$ depends on the geometry of the true means. We further show that the hard assignment objective arises as the limit $\tau\to 0$ of the same mismatched likelihood family, and derive corresponding low- and high-SNR results for hard-assignment mean estimation and latent-label recovery. Furthermore, in low SNR, Bayes-optimal clustering is close to random guessing, and the hard-assignment target remains far from the true means. These results show that in low-SNR applications, even mild variance misspecification or hard-assignment procedures can induce substantial bias, whereas in high SNR these effects are largely absent.

eess.SP↗

arXiv2026

Group-invariant moments under tomographic projections

Let $f:\mathbb{R}^n\to\mathbb{R}$ be an unknown object, and suppose the observations are tomographic projections of randomly rotated copies of $f$ of the form $Y = P(R\cdot f)$, where $R$ is Haar-uniform in $\mathrm{SO}(n)$ and $P$ is the projection onto an $m$-dimensional subspace, so that $Y:\mathbb{R}^m\to\mathbb{R}$. We prove that, whenever $d\le m$, the $d$-th order moment of the projected data determines the full $d$-th order Haar-orbit moment of $f$, independently of the ambient dimension $n$. We further provide an explicit algorithmic procedure for recovering the latter from the former. As a consequence, any identifiability result for the unprojected model based on the $d$-th order group-invariant moment extends directly to the tomographic setting at the same moment order. In particular, for $n=3$, $m=2$, and $d=2$, our result recovers a classical result in the cryo-EM literature: the covariance of the 2D projection images determines the second order rotationally invariant moment of the underlying 3D object.

eess.SP↗

arXiv2026

Inhomogeneous Submatrix Detection

In this paper, we study the problem of detecting multiple hidden submatrices in a large Gaussian random matrix when the planted signal is inhomogeneous across entries. Under the null hypothesis, the observed matrix has independent and identically distributed standard normal entries. Under the alternative, there exist several planted submatrices whose entries deviate from the background in one of two ways: in the mean-shift model, planted entries (templates) have nonzero and possibly varying means; in the variance-shift model, planted entries have inflated and possibly varying variances. We consider two placement regimes for the planted submatrices. In the first, the row and column index sets are arbitrary. Motivated by scientific applications, in the second regime the row and column indices are restricted to be consecutive. For both alternatives and both placement regimes, we analyze the statistical limits of detection by proving information-theoretic lower bounds and by designing algorithms that match these bounds up to logarithmic factors, for a wide family of templates.

math.ST↗

arXiv2026

Provable orbit recovery over SO(3) from the non-uniform second moment

We study the recovery of an unknown three-dimensional band-limited signal from multiple noisy observations that are randomly rotated by latent elements of SO(3), where the rotations are drawn from an unknown, non-uniform distribution. Because the rotations are unobserved, only the signal orbit under the rotation group can be recovered. We show that the signal orbit and the rotation distribution are jointly identifiable from the first and second moments. This yields an improved high-noise sample complexity that scales quadratically with the noise variance, rather than cubically as in the uniform-rotation case. We further develop a provable, computationally efficient reconstruction algorithm that recovers the 3-D signal by successively solving a sequence of well-conditioned linear systems. The algorithm is validated through extensive numerical experiments. Our results provide a principled and tractable framework for high-noise 3-D orbit recovery, with potential relevance to cryo-electron microscopy and cryo-electron tomography modeling, where molecules are observed in unknown orientations.

eess.SP↗

arXiv2026

The Catastrophic Failure of The k-Means Algorithm in High Dimensions, and How Hartigan's Algorithm Avoids It

Lloyd's k-means algorithm is one of the most widely used clustering methods. We prove that in high-dimensional, high-noise settings, the algorithm exhibits catastrophic failure: with high probability, essentially every partition of the data is a fixed point. Consequently, Lloyd's algorithm simply returns its initial partition - even when the underlying clusters are trivially recoverable by other methods. In contrast, we prove that Hartigan's k-means algorithm does not exhibit this pathology. Our results show the stark difference between these algorithms and offer a theoretical explanation for the empirical difficulties often observed with k-means in high dimensions.

stat.ML↗

arXiv2026

A Fast Approximate Maximum Likelihood Estimator for Low SNR Multi-Reference Alignment

Motivated by single-particle cryo-electron microscopy, multi-reference alignment (MRA) models the task of recovering an unknown signal from multiple noisy observations corrupted by random rotations. The standard approach for computing the maximum likelihood estimator (MLE) is the expectation-maximization (EM) algorithm; however, it often becomes computationally prohibitive, particularly in low signal-to-noise ratio (SNR) settings. We introduce a fast approximate MLE for MRA over the special orthogonal groups $\mathrm{SO}(2)$ and $\mathrm{SO}(3)$ in the low-SNR regime. A low-SNR Taylor expansion of the likelihood reveals a closed-form, non-iterative approximate MLE. We show that this approach yields a consistent estimator in the low-SNR limit and requires substantially lower computational complexity than both EM and invariant-based alternatives. Numerical experiments generated from molecular volumes show that the proposed method provides a favorable accuracy-runtime trade-off, especially in challenging low-SNR regimes, and can serve as an effective initialization for EM.

eess.SP↗

arXiv2025

Orbit recovery under the rigid motions group

We study the orbit recovery problem under the rigid-motion group SE(n), where the objective is to reconstruct an unknown signal from multiple noisy observations subjected to unknown rotations and translations. This problem is fundamental in signal processing, computer vision, and structural biology. Our main theoretical contribution is bounding the sample complexity of this problem. We show that if the d-th order moment under the rotation group SO(n) uniquely determines the signal orbit, then orbit recovery under SE(n) is achievable with $N\gtrsim \sigma^{2d+4}$ samples as the noise variance $\sigma^2 \to \infty$. The key technical insight is that the d-th order SO(n) moments can be explicitly recovered from (d+2)-order SE(n) autocorrelations, enabling us to transfer known results from the rotation-only setting to the rigid-motion case. We further harness this result to derive a matching bound to the sample complexity of the multi-target detection model that serves as an abstract framework for electron-microscopy-based technologies in structural biology, such as single-particle cryo-electron microscopy (cryo-EM) and cryo-electron tomography (cryo-ET). Beyond theory, we present a provable computational pipeline for rigid-motion orbit recovery in three dimensions. Starting from rigid-motion autocorrelations, we extract the SO(3) moments and demonstrate successful reconstruction of a 3-D macromolecular structure. Importantly, this algorithmic approach is valid at any noise level, suggesting that even very small macromolecules, long believed to be inaccessible using structural biology electron-microscopy-based technologies, may, in principle, be reconstructed given sufficient data.

cs.IT↗

arXiv2025

SO(3)-invariant PCA with application to molecular data

Principal component analysis (PCA) is a fundamental technique for dimensionality reduction and denoising; however, its application to three-dimensional data with arbitrary orientations -- common in structural biology -- presents significant challenges. A naive approach requires augmenting the dataset with many rotated copies of each sample, incurring prohibitive computational costs. In this paper, we extend PCA to 3D volumetric datasets with unknown orientations by developing an efficient and principled framework for SO(3)-invariant PCA that implicitly accounts for all rotations without explicit data augmentation. By exploiting underlying algebraic structure, we demonstrate that the computation involves only the square root of the total number of covariance entries, resulting in a substantial reduction in complexity. We validate the method on real-world molecular datasets, demonstrating its effectiveness and opening up new possibilities for large-scale, high-dimensional reconstruction problems.

eess.SP↗

arXiv2025

Sample Complexity Analysis of Multi-Target Detection via Markovian and Hard-Core Multi-Reference Alignment

Motivated by single-particle cryo-electron microscopy, we study the sample complexity of the multi-target detection (MTD) problem, in which an unknown signal appears multiple times at unknown locations within a long, noisy observation. We propose a patching scheme that reduces MTD to a non-i.i.d. multi-reference alignment (MRA) model. In the one-dimensional setting, the latent group elements form a Markov chain, and we show that the convergence rate of any estimator matches that of the corresponding i.i.d. MRA model, up to a logarithmic factor in the number of patches. Moreover, for estimators based on empirical averaging, such as the method of moments, the convergence rates are identical in both settings. We further establish an analogous result in two dimensions, where the latent structure arises from an exponentially mixing random field generated by a hard-core placement model. As a consequence, if the signal in the corresponding i.i.d. MRA model is determined by moments up to order $n_{\min}$, then in the low-SNR regime the number of patches required to estimate the signal in the MTD model scales as $\sigma^{2n_{\min}}$, where $\sigma^2$ denotes the noise variance.

eess.SP↗

arXiv2025

Solving ill-conditioned polynomial equations using score-based priors with application to multi-target detection

Recovering signals from low-order moments is a fundamental yet notoriously difficult task in inverse problems. This recovery process often reduces to solving ill-conditioned systems of polynomial equations. In this work, we propose a new framework that integrates score-based diffusion priors with moment-based estimators to regularize and solve these nonlinear inverse problems. This introduces a new role for generative models: stabilizing polynomial recovery from noisy statistical features. As a concrete application, we study the multi-target detection (MTD) model in the high-noise regime. We demonstrate two main results: (i) diffusion priors substantially improve recovery from third-order moments, and (ii) they make the super-resolution MTD problem, otherwise ill-posed, feasible. Numerical experiments on MNIST data confirm consistent gains in reconstruction accuracy across SNR levels. Our results suggest a promising new direction for combining generative priors with nonlinear polynomial inverse problems.

eess.SP↗

arXiv2025

Structure from Noise: Confirmation Bias in Particle Picking in Structural Biology

The computational pipelines of single-particle cryo-electron microscopy (cryo-EM) and cryo-electron tomography (cryo-ET) include an early particle-picking stage, in which a micrograph or tomogram is scanned to extract candidate particles, typically via template matching or deep-learning-based techniques. The extracted particles are then passed to downstream tasks such as classification and 3D reconstruction. Although it is well understood empirically that particle picking can be sensitive to the choice of templates or learned priors, a quantitative theory of the bias introduced by this stage has been lacking. Here, we develop a mathematical framework for analyzing bias in template matching-based detection with concrete applications to cryo-EM and cryo-ET. We study this bias through two downstream tasks: (i) maximum-likelihood estimation of class means in a Gaussian mixture model (GMM) and (ii) 3D volume reconstruction from the extracted particle stack. We show that when template matching is applied to pure noise, then under broad noise models, the resulting maximum-likelihood estimates converge asymptotically to deterministic, noise-dependent transforms of the user-specified templates, yielding a structure from noise effect. We further characterize how the resulting bias depends on the noise statistics, sample size, dimension, and detection threshold. Finally, controlled experiments using standard cryo-EM software corroborate the theory, demonstrating reproducible structure from noise artifacts in low-SNR data.

eess.SP↗

arXiv2025

Orbit recovery for spherical functions

Orbit recovery is a central problem in both mathematics and applied sciences, with important applications to structural biology. This paper focuses on recovering generic orbits of functions on ${\mathbb R}^{n}$ and the sphere $S^{n-1}$ under the rotation action of $SO(n)$. Specifically, we demonstrate that invariants of degree three (called the bispectrum) suffice to recover generic orbits of functions in finite-dimensional approximations of $L^2({\mathbb R}^n)$ obtained by band-limiting the spherical component and discretizing the radial direction. In particular, our main result explicitly bounds the number of samples in the radial direction required for recovery from the degree three invariants. From an application perspective, the most important case is $SO(3)$, which arises in many scientific fields, and in particular, plays a central role in leading structural biology applications such as cryo-electron tomography and cryo-electron microscopy. Our result for $SO(3)$ states that considering three spherical shells (i.e., samples in the radial direction) is sufficient to recover generic orbits, which verifies an implicit conjecture made in a paper of Bandeira et al. Our proof technique provides an explicit, computationally efficient algorithm to recover the signal by successively solving systems of linear equations. We implemented this algorithm and demonstrated its effectiveness on three protein structures.

math.NA↗

arXiv2025

Expectation-maximization for low-SNR multi-reference alignment

We study the multi-reference alignment (MRA) problem of recovering a signal from noisy observations acted on by unknown random circular shifts. While the information-theoretic limits of MRA are well characterized in many settings, the algorithmic behavior at low signal-to-noise ratio (SNR), the regime of practical interest, remains poorly understood. In this paper, we analyze the expectation-maximization (EM) algorithm, a widely used method for MRA, and characterize its convergence dynamics and initialization dependence in the low-SNR limit. On the convergence side, we prove a two-phase phenomenon near the ground truth as $\mathrm{SNR}\to 0$: an initial contraction with error decaying as $\exp(-\, \mathrm{SNR} \cdot t)$ followed by a much slower phase scaling as $\exp(- \,\mathrm{SNR}^2 \cdot t)$, where $t$ is the iteration number. This yields an iteration-complexity lower bound $T \gtrsim \mathrm{SNR}^{-2}$ to reach a small fixed target accuracy, revealing a severe computational bottleneck at low SNR. We also identify a finite-sample instability, which we term \emph{Ghost of Newton}, in which EM initially approaches the ground truth but later diverges, degrading reconstruction quality. On the bias side, we analyze EM in the noise-only setting ($\mathrm{SNR}=0$), a regime referred to as Einstein from Noise, to highlight its pronounced sensitivity to initialization. We prove that the EM map preserves the Fourier phases of the initialization across all iterations, while the corresponding Fourier magnitudes contract toward zero at a slow rate of $(1+T)^{-1/2}$. Consequently, although the amplitudes vanish in the limit of $T \to \infty$ iterations, the reconstructed structure continues to reflect the geometry encoded by the template's Fourier phases. Together, these results expose fundamental computational and initialization-driven limitations of EM for MRA in the low-SNR regime.

eess.SP↗

arXiv2025

The stability of generalized phase retrieval problem over compact groups

The generalized phase retrieval problem over compact groups aims to recover a set of matrices -- representing an unknown signal -- from their associated Gram matrices. This framework generalizes the classical phase retrieval problem, which reconstructs a signal from the magnitudes of its Fourier transform, to a richer setting involving non-abelian compact groups. In this broader context, the unknown phases in Fourier space are replaced by unknown orthogonal matrices that arise from the action of a compact group on a finite-dimensional vector space. This problem is primarily motivated by advances in electron microscopy to determining the 3D structure of biological macromolecules from highly noisy observations. To capture realistic assumptions from machine learning and signal processing, we model the signal as belonging to one of several broad structural families: a generic linear subspace, a sparse representation in a generic basis, the output of a generic ReLU neural network, or a generic low-dimensional manifold. Our main result shows that, for a prior of sufficiently low dimension, the generalized phase retrieval problem not only admits a unique solution (up to inherent group symmetries), but also satisfies a bi-Lipschitz property. This implies robustness to both noise and model mismatch -- an essential requirement for practical use, especially when measurements are severely corrupted by noise. These findings provide theoretical support for a wide class of scientific problems under modern structural assumptions, and they offer strong foundations for developing robust algorithms in high-noise regimes.

eess.SP↗

arXiv2025

Provable algorithms for multi-reference alignment over $\SO(2)$

The multi-reference alignment (MRA) problem involves reconstructing a signal from multiple noisy observations, each transformed by a random group element. In this paper, we focus on the group $\mathrm{SO}(2)$ of in-plane rotations and propose two computationally efficient algorithms with theoretical guarantees for accurate signal recovery under a non-uniform distribution over the group. The first algorithm exploits the spectral properties of the second moment of the data, while the second utilizes the frequency marching principle. Both algorithms achieve the optimal estimation rate in high-noise regimes, marking a significant advancement in the development of computationally efficient and statistically optimal methods for estimation problems over groups.

math.NA↗