Robust statistics for deterministic and stochastic gravitational waves in non-Gaussian noise I: Frequentist analyses

Abstract

Gravitational wave detectors will need optimal signal-processing algorithms to extract weak signals from the detector noise. Most algorithms designed to date are based on the unrealistic assumption that the detector noise may be modeled as a stationary Gaussian process. However most experiments exhibit a non-Gaussian ``tail'' in the probability distribution. This ``excess'' of large signals can be a troublesome source of false alarms. This article derives an optimal (in the Neyman-Pearson sense, for weak signals) signal processing strategy when the detector noise is non-Gaussian and exhibits tail terms. This strategy is robust, meaning that it is close to optimal for Gaussian noise but far less sensitive than conventional methods to the excess large events that form the tail of the distribution. The method is analyzed for two different signal analysis problems: (i) a known waveform (e.g., a binary inspiral chirp) and (ii) a stochastic background, which requires a multi-detector signal processing algorithm. The methods should be easy to implement: they amount to truncation or clipping of sample values which lie in the outlier part of the probability distribution.