arXiv subjects

Dave Zachariah

Publications and source records attributed to Dave Zachariah.

At least 55 records · Page 3Linked to original sources

arXiv2018

Learning Localized Spatio-Temporal Models From Streaming Data

We address the problem of predicting spatio-temporal processes with temporal patterns that vary across spatial regions, when data is obtained as a stream. That is, when the training dataset is augmented sequentially. Specifically, we develop a localized spatio-temporal covariance model of the process that can capture spatially varying temporal periodicities in the data. We then apply a covariance-fitting methodology to learn the model parameters which yields a predictor that can be updated sequentially with each new data point. The proposed method is evaluated using both synthetic and real climate data which demonstrate its ability to accurately predict data missing in spatial regions over time.

stat.ML↗

arXiv2018

Model-Robust Counterfactual Prediction Method

We develop a novel method for counterfactual analysis based on observational data using prediction intervals for units under different exposures. Unlike methods that target heterogeneous or conditional average treatment effects of an exposure, the proposed approach aims to take into account the irreducible dispersions of counterfactual outcomes so as to quantify the relative impact of different exposures. The prediction intervals are constructed in a distribution-free and model-robust manner based on the conformal prediction approach. The computational obstacles to this approach are circumvented by leveraging properties of a tuning-free method that learns sparse additive predictor models for counterfactual outcomes. The method is illustrated using both real and synthetic data.

math.ST↗

arXiv2018

Recursive nonlinear-system identification using latent variables

In this paper we develop a method for learning nonlinear systems with multiple outputs and inputs. We begin by modelling the errors of a nominal predictor of the system using a latent variable framework. Then using the maximum likelihood principle we derive a criterion for learning the model. The resulting optimization problem is tackled using a majorization-minimization approach. Finally, we develop a convex majorization technique and show that it enables a recursive identification method. The method learns parsimonious predictive models and is tested on both synthetic and real nonlinear systems.

stat.ML↗

arXiv2018

Composite Gaussian Processes: Scalable Computation and Performance Analysis

Gaussian process (GP) models provide a powerful tool for prediction but are computationally prohibitive using large data sets. In such scenarios, one has to resort to approximate methods. We derive an approximation based on a composite likelihood approach using a general belief updating framework, which leads to a recursive computation of the predictor as well as of learning the hyper-parameters. We then provide an analysis of the derived composite GP model in predictive and information-theoretic terms. Finally, we evaluate the approximation with both synthetic data and a real-world application.

stat.ML↗

arXiv2017

How consistent is my model with the data? Information-Theoretic Model Check

The choice of model class is fundamental in statistical learning and system identification, no matter whether the class is derived from physical principles or is a generic black-box. We develop a method to evaluate the specified model class by assessing its capability of reproducing data that is similar to the observed data record. This model check is based on the information-theoretic properties of models viewed as data generators and is applicable to e.g. sequential data and nonlinear dynamical models. The method can be understood as a specific two-sided posterior predictive test. We apply the information-theoretic model check to both synthetic and real data and compare it with a classical whiteness test.

stat.ML↗

arXiv2017

Regularized parametric system identification: a decision-theoretic formulation

Parametric prediction error methods constitute a classical approach to the identification of linear dynamic systems with excellent large-sample properties. A more recent regularized approach, inspired by machine learning and Bayesian methods, has also gained attention. Methods based on this approach estimate the system impulse response with excellent small-sample properties. In several applications, however, it is desirable to obtain a compact representation of the system in the form of a parametric model. By viewing the identification of such models as a decision, we develop a decision-theoretic formulation of the parametric system identification problem that bridges the gap between the classical and regularized approaches above. Using the output-error model class as an illustration, we show that this decision-theoretic approach leads to a regularized method that is robust to small sample-sizes as well as overparameterization.

eess.SY↗

arXiv2017

Leontief Meets Shannon - Measuring the Complexity of the Economic System

We develop a complexity measure for large-scale economic systems based on Shannon's concept of entropy. By adopting Leontief's perspective of the production process as a circular flow, we formulate the process as a Markov chain. Then we derive a measure of economic complexity as the average number of bits required to encode the flow of goods and services in the production process. We illustrate this measure using data from seven national economies, spanning several decades.

q-fin.GN↗

arXiv2017

Prediction performance after learning in Gaussian process regression

This paper considers the quantification of the prediction performance in Gaussian process regression. The standard approach is to base the prediction error bars on the theoretical predictive variance, which is a lower bound on the mean square-error (MSE). This approach, however, does not take into account that the statistical model is learned from the data. We show that this omission leads to a systematic underestimation of the prediction errors. Starting from a generalization of the Cramér-Rao bound, we derive a more accurate MSE bound which provides a measure of uncertainty for prediction of Gaussian processes. The improved bound is easily computed and we illustrate it using synthetic and real data examples. of uncertainty for prediction of Gaussian processes and illustrate it using synthetic and real data examples.

stat.ML↗

arXiv2017

Online Learning for Distribution-Free Prediction

We develop an online learning method for prediction, which is important in problems with large and/or streaming data sets. We formulate the learning approach using a covariance-fitting methodology, and show that the resulting predictor has desirable computational and distribution-free properties: It is implemented online with a runtime that scales linearly in the number of samples; has a constant memory requirement; avoids local minima problems; and prunes away redundant feature dimensions without relying on restrictive assumptions on the data distribution. In conjunction with the split conformal approach, it also produces distribution-free prediction confidence intervals in a computationally efficient manner. The method is demonstrated on both real and synthetic datasets.

cs.LG↗

arXiv2017

PUMA criterion = MODE criterion

We show that the recently proposed (enhanced) PUMA estimator for array processing minimizes the same criterion function as the well-established MODE estimator. (PUMA = principal-singular-vector utilization for modal analysis, MODE = method of direction estimation.)

stat.OT↗

arXiv2016

Pearson information-based lower bound on Fisher information

The Fisher information matrix (FIM) plays an important role in the analysis of parameter inference and system design problems. In a number of cases, however, the statistical data distribution and its associated information matrix are either unknown or intractable. For this reason, it is of interest to develop useful lower bounds on the FIM. In this lecture note, we derive such a bound based on moment constraints. We call this bound the Pearson information matrix (PIM) and relate it to properties of a misspecified data distribution. Finally, we show that the inverse PIM coincides with the asymptotic covariance matrix of the optimally weighted generalized method of moments.

math.ST↗

arXiv2016

Scalable and Passive Wireless Network Clock Synchronization

Clock synchronization is ubiquitous in wireless systems for communication, sensing and control. In this paper we design a scalable system in which an indefinite number of passively receiving wireless units can synchronize to a single master clock at the level of discrete clock ticks. Accurate synchronization requires an estimate of the node positions. If such information is available the framework developed here takes position uncertainties into account. In the absence of such information we propose a mechanism which enables simultaneous synchronization and positioning. Furthermore we derive the Cramer-Rao bounds for the system which show that it enables synchronization accuracy at sub-nanosecond levels. Finally, we develop and evaluate an online estimation method which is statistically efficient.

cs.NI↗

arXiv2015

Robust Optimal Power Distribution for Hyperthermia Cancer Treatment

We consider an optimization problem for spatial power distribution generated by an array of transmitting elements. Using ultrasound hyperthermia cancer treatment as a motivating example, the signal design problem consists of optimizing the power distribution across the tumor and healthy tissue regions, respectively. The models used in the optimization problem are, however, invariably subject to errors. deposition as well as inefficient treatment. To combat such unknown model errors, we formulate a robust signal design framework that can take the uncertainty into account using a worst-case approach. This leads to a semi-infinite programming (SIP) robust design problem which we reformulate as a tractable convex problem, potentially has a wider range of applications.

cs.IT↗

arXiv2015

Cramer-Rao bound analog of Bayes rule

In this lecture note, we show a general property of the Cramer-Rao bound (CRB) that quantifies the interdependencies between the parameters in a vector. The presented result is valid for more general models than the additive noise model and also generalizes previous results to vector parameters. The CRB analog to Bayes' rule will be illustrated via two examples.

math.ST↗

arXiv2015

Online Hyperparameter-Free Sparse Estimation Method

In this paper we derive an online estimator for sparse parameter vectors which, unlike the LASSO approach, does not require the tuning of any hyperparameters. The algorithm is based on a covariance matching approach and is equivalent to a weighted version of the square-root LASSO. The computational complexity of the estimator is of the same order as that of the online versions of regularized least-squares (RLS) and LASSO. We provide a numerical comparison with feasible and infeasible implementations of the LASSO and RLS to illustrate the advantage of the proposed online hyperparameter-free estimator.

math.ST↗

arXiv2015

Joint Ranging and Clock Parameter Estimation by Wireless Round Trip Time Measurements

In this paper we develop a new technique for estimating fine clock errors and range between two nodes simultaneously by two-way time-of-arrival measurements us- ing impulse-radio ultra-wideband signals. Estimators for clock parameters and the range are proposed that are robust with respect to outliers. They are analyzed numerically and by means of experimental measurement campaigns. The technique and derived estimators achieve accuracies below 1Hz for frequency estimation, below 1 ns for phase estimation and 20 cm for range estimation, at 4m distance using 100MHz clocks at both nodes. Therefore, we show that the proposed joint approach is practical and can simultaneously provide clock synchronization and positioning in an experimental system.

cs.NI↗

arXiv2014

Weighted SPICE: A Unifying Approach for Hyperparameter-Free Sparse Estimation

In this paper we present the SPICE approach for sparse parameter estimation in a framework that unifies it with other hyperparameter-free methods, namely LIKES, SLIM and IAA. Specifically, we show how the latter methods can be interpreted as variants of an adaptively reweighted SPICE method. Furthermore, we establish a connection between SPICE and the l1-penalized LAD estimator as well as the square-root LASSO method. We evaluate the four methods mentioned above in a generic sparse regression problem and in an array processing application.

math.ST↗

arXiv2014

Estimation for the Linear Model with Uncertain Covariance Matrices

We derive a maximum a posteriori estimator for the linear observation model, where the signal and noise covariance matrices are both uncertain. The uncertainties are treated probabilistically by modeling the covariance matrices with prior inverse-Wishart distributions. The nonconvex problem of jointly estimating the signal of interest and the covariance matrices is tackled by a computationally efficient fixed-point iteration as well as an approximate variational Bayes solution. The statistical performance of estimators is compared numerically to state-of-the-art estimators from the literature and shown to perform favorably.

math.ST↗