arXiv subjects

Idris Eckley

Publications and source records attributed to Idris Eckley.

9 recordsLinked to original sources

arXiv2026

Detecting interpolation errors in infant mortality counts in 20th Century England and Wales

Understanding historical datasets, such as the England and Wales infant mortality data, for local government districts can provide valuable insights into our changing society. Such analyses can prove challenging in practice, due to frequent changes in the boundaries of local government districts for which records are collected. One solution adopted in the literature to overcome such practical challenges is to pre-process data using areal interpolation to render the units consistent over the time period of focus. However, such methods are prone to errors. In this paper we introduce a novel changepoint method to detect instances where interpolation performs poorly. We demonstrate the utility of our method on original data, and also demonstrate how correcting interpolation errors can affect the clustering of the infant mortality curves.

stat.AP↗

arXiv2023

A Constant-per-Iteration Likelihood Ratio Test for Online Changepoint Detection for Exponential Family Models

Online changepoint detection algorithms that are based on likelihood-ratio tests have been shown to have excellent statistical properties. However, a simple online implementation is computationally infeasible as, at time $T$, it involves considering $O(T)$ possible locations for the change. Recently, the FOCuS algorithm has been introduced for detecting changes in mean in Gaussian data that decreases the per-iteration cost to $O(\log T)$. This is possible by using pruning ideas, which reduce the set of changepoint locations that need to be considered at time $T$ to approximately $\log T$. We show that if one wishes to perform the likelihood ratio test for a different one-parameter exponential family model, then exactly the same pruning rule can be used, and again one need only consider approximately $\log T$ locations at iteration $T$. Furthermore, we show how we can adaptively perform the maximisation step of the algorithm so that we need only maximise the test statistic over a small subset of these possible locations. Empirical results show that the resulting online algorithm, which can detect changes under a wide range of models, has a constant-per-iteration cost on average.

stat.CO↗

arXiv2022

Sparse Temporal Disaggregation

Temporal disaggregation is a method commonly used in official statistics to enable high-frequency estimates of key economic indicators, such as GDP. Traditionally, such methods have relied on only a couple of high-frequency indicator series to produce estimates. However, the prevalence of large, and increasing, volumes of administrative and alternative data-sources motivates the need for such methods to be adapted for high-dimensional settings. In this article, we propose a novel sparse temporal-disaggregation procedure and contrast this with the classical Chow-Lin method. We demonstrate the performance of our proposed method through simulation study, highlighting various advantages realised. We also explore its application to disaggregation of UK gross domestic product data, demonstrating the method's ability to operate when the number of potential indicators is greater than the number of low-frequency observations.

econ.EM↗

arXiv2022

Poisson-FOCuS: An efficient online method for detecting count bursts with application to gamma ray burst detection

Gamma-ray bursts are flashes of light from distant exploding stars. Cube satellites that monitor photons across different energy bands are used to detect these bursts. There is a need for computationally efficient algorithms, able to run using the limited computational resource onboard a cube satellite, that can detect when gamma-ray bursts occur. Current algorithms are based on monitoring photon counts across a grid of different sizes of time window. We propose a new algorithm, which extends the recently developed FOCuS algorithm for online change detection to Poisson data. Our algorithm is mathematically equivalent to searching over all possible window sizes, but at half the computational cost of the current grid-based methods. We demonstrate the additional power of our approach using simulations and data drawn from the Fermi gamma-ray burst catalogue.

astro-ph.HE↗

arXiv2022

Fast Online Changepoint Detection via Functional Pruning CUSUM statistics

Many modern applications of online changepoint detection require the ability to process high-frequency observations, sometimes with limited available computational resources. Online algorithms for detecting a change in mean often involve using a moving window, or specifying the expected size of change. Such choices affect which changes the algorithms have most power to detect. We introduce an algorithm, Functional Online CuSUM (FOCuS), which is equivalent to running these earlier methods simultaneously for all sizes of window, or all possible values for the size of change. Our theoretical results give tight bounds on the expected computational cost per iteration of FOCuS, with this being logarithmic in the number of observations. We show how FOCuS can be applied to a number of different change in mean scenarios, and demonstrate its practical utility through its state-of-the art performance at detecting anomalous behaviour in computer server data.

stat.ME↗

arXiv2021

Collective anomaly detection in High-dimensional VAR Models

There is increasing interest in detecting collective anomalies: potentially short periods of time where the features of data change before reverting back to normal behaviour. We propose a new method for detecting a collective anomaly in VAR models. Our focus is on situations where the change in the VAR coefficient matrix at an anomaly is sparse, i.e. a small number of entries of the VAR coefficient matrix change. To tackle this problem, we propose a test statistic for a local segment that is built on the lasso estimator of the change in model parameters. This enables us to detect a sparse change more efficiently and our lasso-based approach becomes especially advantageous when the anomalous interval is short. We show that the new procedure controls Type 1 error and has asymptotic power tending to one. The practicality of our approach is demonstrated through simulations and two data examples, involving New York taxi trip data and EEG data.

stat.ME↗

arXiv2020

A novel change point approach for the detection of gas emission sources using remotely contained concentration data

Motivated by an example from remote sensing of gas emission sources, we derive two novel change point procedures for multivariate time series where, in contrast to classical change point literature, the changes are not required to be aligned in the different components of the time series. Instead the change points are described by a functional relationship where the precise shape depends on unknown parameters of interest such as the source of the gas emission in the above example. Two different types of tests and the corresponding estimators for the unknown parameters describing the change locations are proposed. We derive the null asymptotics for both tests under weak assumptions on the error time series and show asymptotic consistency under alternatives. Furthermore, we prove consistency for the corresponding estimators of the parameters of interest. The small sample behavior of the methodology is assessed by means of a simulation study and the above remote sensing example analyzed in detail.

stat.ME↗

arXiv2019

Subspace Clustering of Very Sparse High-Dimensional Data

In this paper we consider the problem of clustering collections of very short texts using subspace clustering. This problem arises in many applications such as product categorisation, fraud detection, and sentiment analysis. The main challenge lies in the fact that the vectorial representation of short texts is both high-dimensional, due to the large number of unique terms in the corpus, and extremely sparse, as each text contains a very small number of words with no repetition. We propose a new, simple subspace clustering algorithm that relies on linear algebra to cluster such datasets. Experimental results on identifying product categories from product names obtained from the US Amazon website indicate that the algorithm can be competitive against state-of-the-art clustering algorithms.

stat.ML↗

arXiv2017

Most recent changepoint detection in Panel data

Detecting recent changepoints in time-series can be important for short-term prediction, as we can then base predictions just on the data since the changepoint. In many applications we have panel data, consisting of many related univariate time-series. We present a novel approach to detect sets of most recent changepoints in such panel data which aims to pool information across time-series, so that we preferentially infer a most recent change at the same time-point in multiple series. Our approach is computationally efficient as it involves analysing each time-series independently to obtain a profile likelihood like quantity that summarises the evidence for the series having either no change or a specific value for its most recent changepoint. We then post-process this output from each time-series to obtain a potentially small set of times for the most recent changepoints, and, for each time, the set of series which has their most recent changepoint at that time. We demonstrate the usefulness of this method on two data sets: forecasting events in a telecommunications network and inference about changes in the net asset ratio for a panel of US firms.

stat.AP↗