Searcharxiv⌕ Search

arXiv · 2610.23504

Weighted spatial averages with missing observations: Convergence and accuracy of delta-method variance approximations

Abstract

Weighted spatial averages from incomplete networks are ratios whose variance is approximated by truncating a Taylor series (the delta method). Three questions remain open: whether the series converges, how large a truncation's error is, and how it behaves as the network grows. We address these for observations missing completely at random, independently with a common probability $α$. First, the truncations, which count corrections for the denominator (the total reported weight) and for its coupling with the numerator, converge for a given network only if $α>\tfrac12$ and both corrections increase together. Second, whether or not the series converges, the error of truncations correcting only for the denominator is controlled at any network size by its coefficient of variation $ρ$, with $ρ^2=\tfrac{1-α}α\sum_iβ_i^2$ for weights $β_i$, and by the probability of rare reporting patterns. Third, for uniform weights and bounded fields, these errors vanish as the network grows, at any fixed availability. If correlations also decay with distance, the first-order approximation of earlier work has error of order $ρ^4$, against $ρ^2$ for the simplest approximation. Only truncations of equal order in numerator and denominator, beginning with the first-order truncation, avoid spurious dependence on the mean level; higher ones need not be more accurate. The exact variance for uniform weights, a single sum, shows that for correlated fields the first-order approximation overstates the contribution of missing data but is conservative overall. Concentrated weights degrade it more than $ρ$ indicates. Rainfall simulations over India illustrates results, several of which extend to self-normalized estimators.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ashwin K Seshadri, Abhishek Pal Majumder. 2026-10-16. Weighted spatial averages with missing observations: Convergence and accuracy of delta-method variance approximations. https://arxiv.org/abs/2610.23504

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Rao-Blackwellized e-variables

We show that for any concave utility, the expected utility of an e-variable can only increase after conditioning on a sufficient statistic, echoing the classic Rao-Blackwell theorem. The simplest form of the result has a straightforward proof using Jensen's inequality, but we prove more sophisticated versions. We provide several applications of this insight, including a derivation of the log-optimal e-variable for testing the variance in a linear regression model.

math.ST↗

A Data-Consistent Approach to Ensemble Filtering

Ensemble data assimilation is challenging when the ensemble size is substantially smaller than the state and observation dimensions, leaving empirical covariance information rank deficient and sensitive to finite sampling. We introduce the Quantity-of-Interest Principal Component Analysis Ensemble Data-Consistent Filter (QPCA-EnDCF), a deterministic filtering method motivated by data-consistent inversion that regularizes the update in the whitened forecast--observation mismatch space. The method restricts the correction to a rank-$κ$ empirical spectral subspace before mapping it to state space through an empirical EnDCF gain. We establish concentration and spectral-projector perturbation results and derive a bias--variance decomposition showing that truncation bias is governed by the discarded component of the population mean residual, while finite-ensemble variability depends on estimation of the retained correction. QPCA-EnDCF avoids the analysis-mean variance contribution from perturbed observations in stochastic EnKF updates, and Lorenz--96 experiments show improved calibration and lower mean-squared error relative to the tested stochastic and deterministic controls, primarily through lower across-realization variance despite increased bias. An idealized inlet shallow-water experiment further shows that QPCA-EnDCF can be implemented in a high-dimensional, severely rank-deficient nonlinear model and remains numerically viable through the prescribed shelf-crossing endpoints.

math.ST↗

Almost Sharp Equivalence between Approximate Message Passing and Low-Degree Polynomials

We prove a sharp lower bound for growing-degree polynomial estimation in the symmetric rank-one matrix estimation problem. The observation is \[ \boldsymbol{Y}=\frac{1}{\sqrt{n}} \boldsymbolθ \boldsymbolθ^{\top}+\boldsymbol{W}, \] where the coordinates of $\boldsymbolθ$ are independent with a common, fixed sub-Gaussian distribution $π_Θ$ satisfying $\mathbb E[Θ]\neq0$, and $\boldsymbol{W}$ is symmetric with independent standard Gaussian upper-triangular entries. For every such prior, we give an explicit finite-dimensional bound implying that every sequence of polynomial estimators of degree $D(n)=o(n^{1/60})$ has normalized mean-square error with limit inferior at least $\mathbb E[Θ^2]-q_{\mathsf{amp}}$, the limiting error of Bayes approximate message passing (AMP). This extends the constant-degree equivalence of Montanari and Wein~\cite{montanari2025equivalence} to growing degree under the same prior assumptions. Combined with their polynomial approximation of fixed-iteration AMP, the bound identifies the exact limiting low-degree MMSE whenever $D(n)\to\infty$ within this range. It thereby answers the growing-degree AMP-equivalence question discussed in~\cite{wein2025computational, maleki2026high} for nonzero-mean sub-Gaussian priors in the stated degree range. We also obtain a moment-dependent finite-dimensional lower bound for any nonzero-mean prior with all moments finite. The proof bounds low-degree MMSE using \emph{conditional} joint cumulants and a refined application of Jensen's inequality. The conditional cumulants are defined using an auxiliary Gaussian channel $\boldsymbol{R}$ calibrated to the AMP fixed point. This retains signal dependence that is lost in unconditional cumulant bounds and produces the cancellations needed for quantitative control as the degree grows.

math.ST↗