arXiv subjects

Eric L. Miller

Publications and source records attributed to Eric L. Miller.

At least 19 recordsLinked to original sources

arXiv2025

On Multi-entity, Multivariate Quickest Change Point Detection

We propose a framework for online Change Point Detection (CPD) from multi-entity, multivariate time series data, motivated by applications in crowd monitoring where traditional sensing methods (e.g., video surveillance) may be infeasible. Our approach addresses the challenge of detecting system-wide behavioral shifts in complex, dynamic environments where the number and behavior of individual entities may be uncertain or evolve. We introduce the concept of Individual Deviation from Normality (IDfN), computed via a reconstruction-error-based autoencoder trained on normal behavior. We aggregate these individual deviations using mean, variance, and Kernel Density Estimates (KDE) to yield a System-Wide Anomaly Score (SWAS). To detect persistent or abrupt changes, we apply statistical deviation metrics and the Cumulative Sum (CUSUM) technique to these scores. Our unsupervised approach eliminates the need for labeled data or feature extraction, enabling real-time operation on streaming input. Evaluations on both synthetic datasets and crowd simulations, explicitly designed for anomaly detection in group behaviors, demonstrate that our method accurately detects significant system-level changes, offering a scalable and privacy-preserving solution for monitoring complex multi-agent systems. In addition to this methodological contribution, we introduce new, challenging multi-entity multivariate time series datasets generated from crowd simulations in Unity and coupled nonlinear oscillators. To the best of our knowledge, there is currently no publicly available dataset of this type designed explicitly to evaluate CPD in complex collective and interactive systems, highlighting an essential gap that our work addresses.

eess.SP↗

arXiv2023

Nonparametric and Regularized Dynamical Wasserstein Barycenters for Sequential Observations

We consider probabilistic models for sequential observations which exhibit gradual transitions among a finite number of states. We are particularly motivated by applications such as human activity analysis where observed accelerometer time series contains segments representing distinct activities, which we call pure states, as well as periods characterized by continuous transition among these pure states. To capture this transitory behavior, the dynamical Wasserstein barycenter (DWB) model of Cheng et al. in 2021 [1] associates with each pure state a data-generating distribution and models the continuous transitions among these states as a Wasserstein barycenter of these distributions with dynamically evolving weights. Focusing on the univariate case where Wasserstein distances and barycenters can be computed in closed form, we extend [1] specifically relaxing the parameterization of the pure states as Gaussian distributions. We highlight issues related to the uniqueness in identifying the model parameters as well as uncertainties induced when estimating a dynamically evolving distribution from a limited number of samples. To ameliorate non-uniqueness, we introduce regularization that imposes temporal smoothness on the dynamics of the barycentric weights. A quantile-based approximation of the pure state distributions yields a finite dimensional estimation problem which we numerically solve using cyclic descent alternating between updates to the pure-state quantile functions and the barycentric weights. We demonstrate the utility of the proposed algorithm in segmenting both simulated and real world human activity time series.

cs.LG↗

arXiv2022

Dynamical Wasserstein Barycenters for Time-series Modeling

Many time series can be modeled as a sequence of segments representing high-level discrete states, such as running and walking in a human activity application. Flexible models should describe the system state and observations in stationary "pure-state" periods as well as transition periods between adjacent segments, such as a gradual slowdown between running and walking. However, most prior work assumes instantaneous transitions between pure discrete states. We propose a dynamical Wasserstein barycentric (DWB) model that estimates the system state over time as well as the data-generating distributions of pure states in an unsupervised manner. Our model assumes each pure state generates data from a multivariate normal distribution, and characterizes transitions between states via displacement-interpolation specified by the Wasserstein barycenter. The system state is represented by a barycentric weight vector which evolves over time via a random walk on the simplex. Parameter learning leverages the natural Riemannian geometry of Gaussian distributions under the Wasserstein distance, which leads to improved convergence speeds. Experiments on several human activity datasets show that our proposed DWB model accurately learns the generating distribution of pure states while improving state estimation for transition periods compared to the commonly used linear interpolation mixture models.

cs.LG↗

arXiv2022

Easy Variational Inference for Categorical Models via an Independent Binary Approximation

We pursue tractable Bayesian analysis of generalized linear models (GLMs) for categorical data. Thus far, GLMs are difficult to scale to more than a few dozen categories due to non-conjugacy or strong posterior dependencies when using conjugate auxiliary variable methods. We define a new class of GLMs for categorical data called categorical-from-binary (CB) models. Each CB model has a likelihood that is bounded by the product of binary likelihoods, suggesting a natural posterior approximation. This approximation makes inference straightforward and fast; using well-known auxiliary variables for probit or logistic regression, the product of binary models admits conjugate closed-form variational inference that is embarrassingly parallel across categories and invariant to category ordering. Moreover, an independent binary model simultaneously approximates multiple CB models. Bayesian model averaging over these can improve the quality of the approximation for any given dataset. We show that our approach scales to thousands of categories, outperforming posterior estimation competitors like Automatic Differentiation Variational Inference (ADVI) and No U-Turn Sampling (NUTS) in the time required to achieve fixed prediction quality.

stat.ML↗

arXiv2021

A novel reconstruction technique for two-dimensional Bragg scatter imaging

Here we introduce a new reconstruction technique for two-dimensional Bragg Scattering Tomography (BST), based on the Radon transform models of [arXiv preprint, arXiv:2004.10961 (2020)]. Our method uses a combination of ideas from multibang control and microlocal analysis to construct an objective function which can regularize the BST artifacts; specifically the boundary artifacts due to sharp cutoff in sinogram space (as observed in [arXiv preprint, arXiv:2007.00208 (2020)]), and artifacts arising from approximations made in constructing the model used for inversion. We then test our algorithm in a variety of Monte Carlo (MC) simulated examples of practical interest in airport baggage screening and threat detection. The data used in our studies is generated with a novel Monte-Carlo code presented here. The model, which is available from the authors upon request, captures both the Bragg scatter effects described by BST as well as beam attenuation and Compton scatter.

math.NA↗

arXiv2020

Bragg scattering tomography

Here we introduce a new forward model and imaging modality for Bragg Scattering Tomography (BST). The model we propose is based on an X-ray portal scanner with linear detector collimation, currently being developed for use in airport baggage screening. The geometry under consideration leads us to a novel two-dimensional inverse problem, where we aim to reconstruct the Bragg scattering differential cross section function from its integrals over a set of symmetric $C^2$ curves in the plane. The integral transform which describes the forward problem in BST is a new type of Radon transform, which we introduce and denote as the Bragg transform. We provide new injectivity results for the Bragg transform here, and describe how the conditions of our theorems can be applied to assist in the machine design of the portal scanner. Further we provide an extension of our results to $n$-dimensions, where a generalization of the Bragg transform is introduced. Here we aim to reconstruct a real valued function on $\mathbb{R}^{n+1}$ from its integrals over $n$-dimensional surfaces of revolution of $C^2$ curves embedded in $\mathbb{R}^{n+1}$. Injectivity proofs are provided also for the generalized Bragg transform.

math.FA↗

arXiv2020

On Matched Filtering for Statistical Change Point Detection

Non-parametric and distribution-free two-sample tests have been the foundation of many change point detection algorithms. However, randomness in the test statistic as a function of time makes them susceptible to false positives and localization ambiguity. We address these issues by deriving and applying filters matched to the expected temporal signatures of a change for various sliding window, two-sample tests under IID assumptions on the data. These filters are derived asymptotically with respect to the window size for the Wasserstein quantile test, the Wasserstein-1 distance test, Maximum Mean Discrepancy squared (MMD^2), and the Kolmogorov-Smirnov (KS) test. The matched filters are shown to have two important properties. First, they are distribution-free, and thus can be applied without prior knowledge of the underlying data distributions. Second, they are peak-preserving, which allows the filtered signal produced by our methods to maintain expected statistical significance. Through experiments on synthetic data as well as activity recognition benchmarks, we demonstrate the utility of this approach for mitigating false positives and improving the test precision. Our method allows for the localization of change points without the use of ad-hoc post-processing to remove redundant detections common to current methods. We further highlight the performance of statistical tests based on the Quantile-Quantile (Q-Q) function and show how the invariance property of the Q-Q function to order-preserving transformations allows these tests to detect change points of different scales with a single threshold within the same dataset.

eess.SP↗

arXiv2020

A joint reconstruction and lambda tomography regularization technique for energy-resolved X-ray imaging

Here we present new joint reconstruction and regularization techniques inspired by ideas in microlocal analysis and lambda tomography, for the simultaneous reconstruction of the attenuation coefficient and electron density from X-ray transmission (i.e., X-ray CT) and backscattered data (assumed to be primarily Compton scattered). To demonstrate our theory and reconstruction methods, we consider the "parallel line segment" acquisition geometry of Webber and Miller ("Compton scattering tomography in translational geometries." Inverse Problems 36, no. 2 (2020): 025007), which is motivated by system architectures currently under development for airport security screening. We first present a novel microlocal analysis of the parallel line geometry which explains the nature of image artefacts when the attenuation coefficient and electron density are reconstructed separately. We next introduce a new joint reconstruction scheme for low effective $Z$ (atomic number) imaging ($Z<20$) characterized by a regularization strategy whose structure is derived from lambda tomography principles and motivated directly by the microlocal analytic results. Finally we show the effectiveness of our method in combating noise and image artefacts on simulated phantoms.

eess.IV↗

arXiv2020

Optimal Transport Based Change Point Detection and Time Series Segment Clustering

Two common problems in time series analysis are the decomposition of the data stream into disjoint segments that are each in some sense "homogeneous" - a problem known as Change Point Detection (CPD) - and the grouping of similar nonadjacent segments, a problem that we call Time Series Segment Clustering (TSSC). Building upon recent theoretical advances characterizing the limiting distribution-free behavior of the Wasserstein two-sample test (Ramdas et al. 2015), we propose a novel algorithm for unsupervised, distribution-free CPD which is amenable to both offline and online settings. We also introduce a method to mitigate false positives in CPD and address TSSC by using the Wasserstein distance between the detected segments to build an affinity matrix to which we apply spectral clustering. Results on both synthetic and real data sets show the benefits of the approach.

eess.SP↗

arXiv2019

An Inner-Outer Iterative Method for Edge Preservation in Image Restoration and Reconstruction

We present a new inner-outer iterative algorithm for edge enhancement in imaging problems. At each outer iteration, we formulate a Tikhonov-regularized problem where the penalization is expressed in the 2-norm and involves a regularization operator designed to improve edge resolution as the outer iterations progress, through an adaptive process. An efficient hybrid regularization method is used to project the Tikhonov-regularized problem onto approximation subspaces of increasing dimensions (inner iterations), while conveniently choosing the regularization parameter (by applying well-known techniques, such as the discrepancy principle or the ${\mathcal L}$-curve criterion, to the projected problem). This procedure results in an automated algorithm for edge recovery that does not involve regularization parameter tuning by the user, nor repeated calls to sophisticated optimization algorithms, and is therefore particularly attractive from a computational point of view. A key to the success of the new algorithm is the design of the regularization operator through the use of an adaptive diagonal weighting matrix that effectively enforces smoothness only where needed. We demonstrate the value of our approach on applications in X-ray CT image reconstruction and in image deblurring, and show that it can be computationally much more attractive than other well-known strategies for edge preservation, while providing solutions of greater or equal quality.

math.NA↗

arXiv2018

Ensemble Multi-task Gaussian Process Regression with Multiple Latent Processes

Multi-task/Multi-output learning seeks to exploit correlation among tasks to enhance performance over learning or solving each task independently. In this paper, we investigate this problem in the context of Gaussian Processes (GPs) and propose a new model which learns a mixture of latent processes by decomposing the covariance matrix into a sum of structured hidden components each of which is controlled by a latent GP over input features and a "weight" over tasks. From this sum structure, we propose a parallelizable parameter learning algorithm with a predetermined initialization for the "weights". We also notice that an ensemble parameter learning approach using mini-batches of training data not only reduces the computation complexity of learning but also improves the regression performance. We evaluate our model on two datasets, the smaller Swiss Jura dataset and another relatively larger ATMS dataset from NOAA. Substantial improvements are observed compared with established alternatives.

stat.ML↗

arXiv2017

On the Fusion of Compton Scatter and Attenuation Data for Limited-view X-ray Tomographic Applications

In this paper we demonstrate the utility of fusing energy-resolved observations of Compton scattered photons with traditional attenuation data for the joint recovery of mass density and photoelectric absorption in the context of limited view tomographic imaging applications. We begin with the development of a physical and associated numerical model for the Compton scatter process. Using this model, we propose a variational approach recovering these two material properties. In addition to the typical data-fidelity terms, the optimization functional includes regularization for both the mass density and photoelectric coefficients. We consider a novel edge-preserving method in the case of mass density. To aid in the recovery of the photoelectric information, we draw on our recent method in \cite{r15} and employ a non-local regularization scheme that builds on the fact that mass density is more stably imaged. Simulation results demonstrate clear advantages associated with the use of both scattered photon data and energy resolved information in mapping the two material properties of interest. Specifically, comparing images obtained using only conventional attenuation data with those where we employ only Compton scatter photons and images formed from the combination of the two, shows that taking advantage of both types of data for reconstruction provides far more accurate results.

cs.CV↗

arXiv2014

Stabilizing dual-energy X-ray computed tomography reconstructions using patch-based regularization

Recent years have seen growing interest in exploiting dual- and multi-energy measurements in computed tomography (CT) in order to characterize material properties as well as object shape. Material characterization is performed by decomposing the scene into constitutive basis functions, such as Compton scatter and photoelectric absorption functions. While well motivated physically, the joint recovery of the spatial distribution of photoelectric and Compton properties is severely complicated by the fact that the data are several orders of magnitude more sensitive to Compton scatter coefficients than to photoelectric absorption, so small errors in Compton estimates can create large artifacts in the photoelectric estimate. To address these issues, we propose a model-based iterative approach which uses patch-based regularization terms to stabilize inversion of photoelectric coefficients, and solve the resulting problem though use of computationally attractive Alternating Direction Method of Multipliers (ADMM) solution techniques. Using simulations and experimental data acquired on a commercial scanner, we demonstrate that the proposed processing can lead to more stable material property estimates which should aid materials characterization in future dual- and multi-energy CT systems.

cs.CV↗

arXiv2013

A Geometric Approach to Joint Inversion with Applications to Contaminant Source Zone Characterization

This paper presents a new joint inversion approach to shape-based inverse problems. Given two sets of data from distinct physical models, the main objective is to obtain a unified characterization of inclusions within the spatial domain of the physical properties to be reconstructed. Although our proposed method generally applies to many types of inversion problems, the main motivation here is to characterize subsurface contaminant source-zones by processing down gradient hydrological data and cross-gradient electrical resistance tomography (ERT) observations. Inspired by Newton's method for multi-objective optimization, we present an iterative inversion scheme that suggests taking descent steps that can simultaneously reduce both data-model misfit terms. Such an approach, however, requires solving a non-smooth convex problem at every iteration, which is computationally expensive for a pixel-based inversion over the whole domain. Instead, we employ a parametric level set (PaLS) technique that substantially reduces the number of underlying parameters, making the inversion computationally tractable. The performance of the technique is examined and discussed through the reconstruction of source zone architectures that are representative of dense non-aqueous phase liquid (DNAPL) contaminant release in a statistically homogenous sandy aquifer. In these examples, the geometric configuration of the DNAPL mass is considered along with additional information about its spatial variability within the contaminated zone, such as the identification of low and high saturation regions. Comparison of the reconstructions with the true DNAPL architectures highlights the superior performance of the model-based technique and joint inversion scheme.

physics.geo-ph↗

arXiv2013

Tensor-based formulation and nuclear norm regularization for multi-energy computed tomography

The development of energy selective, photon counting X-ray detectors allows for a wide range of new possibilities in the area of computed tomographic image formation. Under the assumption of perfect energy resolution, here we propose a tensor-based iterative algorithm that simultaneously reconstructs the X-ray attenuation distribution for each energy. We use a multi-linear image model rather than a more standard "stacked vector" representation in order to develop novel tensor-based regularizers. Specifically, we model the multi-spectral unknown as a 3-way tensor where the first two dimensions are space and the third dimension is energy. This approach allows for the design of tensor nuclear norm regularizers, which like its two dimensional counterpart, is a convex function of the multi-spectral unknown. The solution to the resulting convex optimization problem is obtained using an alternating direction method of multipliers (ADMM) approach. Simulation results shows that the generalized tensor nuclear norm can be used as a stand alone regularization technique for the energy selective (spectral) computed tomography (CT) problem and when combined with total variation regularization it enhances the regularization capabilities especially at low energy images where the effects of noise are most prominent.

cs.CV↗

arXiv2013

Exploiting Structural Complexity for Robust and Rapid Hyperspectral Imaging

This paper presents several strategies for spectral de-noising of hyperspectral images and hypercube reconstruction from a limited number of tomographic measurements. In particular we show that the non-noisy spectral data, when stacked across the spectral dimension, exhibits low-rank. On the other hand, under the same representation, the spectral noise exhibits a banded structure. Motivated by this we show that the de-noised spectral data and the unknown spectral noise and the respective bands can be simultaneously estimated through the use of a low-rank and simultaneous sparse minimization operation without prior knowledge of the noisy bands. This result is novel for for hyperspectral imaging applications. In addition, we show that imaging for the Computed Tomography Imaging Systems (CTIS) can be improved under limited angle tomography by using low-rank penalization. For both of these cases we exploit the recent results in the theory of low-rank matrix completion using nuclear norm minimization.

physics.geo-ph↗

arXiv2012

High-order regularized regression in Electrical Impedance Tomography

We present a novel approach for the inverse problem in electrical impedance tomography based on regularized quadratic regression. Our contribution introduces a new formulation for the forward model in the form of a nonlinear integral transform, that maps changes in the electrical properties of a domain to their respective variations in boundary data. Using perturbation theory the transform is approximated to yield a high-order misfit unction which is then used to derive a regularized inverse problem. In particular, we consider the nonlinear problem to second-order accuracy, hence our approximation method improves upon the local linearization of the forward mapping. The inverse problem is approached using Newton's iterative algorithm and results from simulated experiments are presented. With a moderate increase in computational complexity, the method yields superior results compared to those of regularized linear regression and can be implemented to address the nonlinear inverse problem.

physics.geo-ph↗

arXiv2011

Parametric Level Set Methods for Inverse Problems

In this paper, a parametric level set method for reconstruction of obstacles in general inverse problems is considered. General evolution equations for the reconstruction of unknown obstacles are derived in terms of the underlying level set parameters. We show that using the appropriate form of parameterizing the level set function results a significantly lower dimensional problem, which bypasses many difficulties with traditional level set methods, such as regularization, re-initialization and use of signed distance function. Moreover, we show that from a computational point of view, low order representation of the problem paves the path for easier use of Newton and quasi-Newton methods. Specifically for the purposes of this paper, we parameterize the level set function in terms of adaptive compactly supported radial basis functions, which used in the proposed manner provides flexibility in presenting a larger class of shapes with fewer terms. Also they provide a "narrow-banding" advantage which can further reduce the number of active unknowns at each step of the evolution. The performance of the proposed approach is examined in three examples of inverse problems, i.e., electrical resistance tomography, X-ray computed tomography and diffuse optical tomography.

math.AP↗