SearcharxivSearch

arXiv subjects

Wicher Bergsma

Publications and source records attributed to Wicher Bergsma.

14 recordsLinked to original sources

A note on the distribution of the partial correlation coefficient with nonparametrically estimated marginal regressions

There has been much interest in the nonparametric testing of conditional independence in the econometric and statistical literature, but the simplest and potentially most useful method, based on the sample partial correlation, seems to have been overlooked, its distribution only having been investigated in some simple parametric instances. The present note shows that an easy to apply permutation test based on the sample partial correlation with nonparametrically estimated marginal regressions has good large and small sample properties.

math.ST

Bergsma--Dassios sign covariance with ties: a nonnegative decomposition and sharp bounds

We construct tie-symmetrised extensions $\widetilde D$ and $\widetilde R$ of the unnormalised Hoeffding and Blum--Kiefer--Rosenblatt functionals $D$ and $R$ and prove, for every real-valued bivariate law, the exact nonnegative decomposition $ τ^{*}(X,Y)=12\widetilde D(X,Y)+24\widetilde R(X,Y), \qquad \widetilde D,\widetilde R\geq0, \qquad \widetilde R(X,Y)=0\quad\Longleftrightarrow\quad X\perp\!\!\!\perp Y. $ For atomless margins the components reduce to $D$ and $R$, whereas in the presence of ties the corresponding classical identity $τ^{*}=12D+24R$ can fail. The components arise from the Hoeffding projections of a symmetrised pair kernel. For the four strict/non-strict boundary versions $D_{ε,η}$ and $R_{ε,η}$, we establish the sharp comparisons $ 9\widetilde D\geq\sum_{ε,η}D_{ε,η}, \qquad 9\widetilde R\geq\sum_{ε,η}R_{ε,η}. $ Consequently, $τ^{*}\geq0$ for every bivariate law, and the Bergsma--Dassios conjecture is settled: $τ^{*}=0$ if and only if $X\perp\!\!\!\perp Y$. The boundary comparisons also yield the universal bound $τ^{*}\geq(8/3)R$. Finite-table arguments, including an exact sum-of-squares certificate, establish the boundary inequalities, and weak-order quantisation transfers them to arbitrary laws. The factor $9$ in each comparison and the coefficient $24$ in $τ^{*}\geq24\widetilde R$ are sharp. The usual permutation test is consistent against every fixed dependent alternative, without assumptions on the margins.

math.ST

Hierarchical additive interaction modelling with Gaussian process prior and its efficient implementation for multidimensional grid data

Additive Gaussian process (GP) models offer flexible tools for modelling complex non-linear relationships and interaction effects among covariates. While most studies have focused on predictive performance, relatively little attention has been given to identifying the underlying interaction structure, which may be of scientific interest in many applications. In practice, the use of additive GP models in this context has been limited by the cubic computational cost and quadratic storage requirements of GP inference. This paper presents a fast hierarchical additive interaction GP model for multi-dimensional grid data. A hierarchical ANOVA decomposition kernel forms the foundation of our model, which incorporate main and interaction effects under the principle of marginality. Kernel centring ensures identifiability and provides a unique, interpretable decomposition of lower- and higher-order effects. For datasets forming a multi-dimensional grid, efficient implementation is achieved by exploiting the Kronecker product structure of the covariance matrix. Our contribution is to extend Kronecker-based computation to handle any interaction structure within the proposed class of hierarchical additive GP models, whereas previous methods were limited to separable or fully saturated cases. The benefits of the proposed approach are demonstrated through simulation studies and an application to high-frequency nitrogen dioxide concentration data in London.

stat.ME

Additive interaction modelling using I-priors

Additive regression models with interactions are widely studied in the literature, using methods such as splines or Gaussian process regression. However, these methods can pose challenges for estimation and model selection, due to the presence of many smoothing parameters and the lack of suitable criteria. We propose to address these challenges by extending the I-prior methodology (Bergsma, 2020) to multiple covariates, which may be multidimensional. The I-prior methodology has some advantages over other methods, such as Gaussian process regression and Tikhonov regularization, both theoretically and practically. In particular, the I-prior is a proper prior, is based on minimal assumptions, yields an admissible posterior mean, and estimation of the scale (or smoothing) parameters can be done using an EM algorithm with simple E and M steps. Moreover, we introduce a parsimonious specification of models with interactions, which has two benefits: (i) it reduces the number of scale parameters and thus facilitates the estimation of models with interactions, and (ii) it enables straightforward model selection (among models with different interactions) based on the marginal likelihood.

math.ST

Marginal Models: an Overview

Marginal models involve restrictions on the conditional and marginal association structure of a set of categorical variables. They generalize log-linear models for contingency tables, which are the fundamental tools for modelling the conditional association structure. This chapter gives an overview of the development of marginal models during the past 20 years. After providing some motivating examples, the first few sections focus on the definition and characteristics of marginal models. Specifically, we show how their fundamental properties can be understood from the properties of marginal log-linear parameterizations. Algorithms for estimating marginal models are discussed, focussing on the maximum likelihood and the generalized estimating equations approaches. It is shown how marginal models can help to understand directed graphical and path models, and a description is given of marginal models with latent variables.

stat.ME

Double Generative Adversarial Networks for Conditional Independence Testing

In this article, we study the problem of high-dimensional conditional independence testing, a key building block in statistics and machine learning. We propose an inferential procedure based on double generative adversarial networks (GANs). Specifically, we first introduce a double GANs framework to learn two generators of the conditional distributions. We then integrate the two generators to construct a test statistic, which takes the form of the maximum of generalized covariance measures of multiple transformation functions. We also employ data-splitting and cross-fitting to minimize the conditions on the generators to achieve the desired asymptotic properties, and employ multiplier bootstrap to obtain the corresponding $p$-value. We show that the constructed test statistic is doubly robust, and the resulting test both controls type-I error and has the power approaching one asymptotically. Also notably, we establish those theoretical guarantees under much weaker and practically more feasible conditions compared to the existing tests, and our proposal gives a concrete example of how to utilize some state-of-the-art deep learning tools, such as GANs, to help address a classical but challenging statistical problem. We demonstrate the efficacy of our test through both simulations and an application to an anti-cancer drug dataset. A Python implementation of the proposed procedure is available at https://github.com/tianlinxu312/dgcit.

stat.ML

Causal Modeling with Stochastic Confounders

This work extends causal inference with stochastic confounders. We propose a new approach to variational estimation for causal inference based on a representer theorem with a random input space. We estimate causal effects involving latent confounders that may be interdependent and time-varying from sequential, repeated measurements in an observational study. Our approach extends current work that assumes independent, non-temporal latent confounders, with potentially biased estimators. We introduce a simple yet elegant algorithm without parametric specification on model components. Our method avoids the need for expensive and careful parameterization in deploying complex models, such as deep neural networks, for causal inference in existing approaches. We demonstrate the effectiveness of our approach on various benchmark temporal datasets.

stat.ML

Regression with I-priors

The problem of estimating a parametric or nonparametric regression function in a model with normal errors is considered. For this purpose, a novel objective prior for the regression function is proposed, defined as the distribution maximizing entropy subject to a suitable constraint based on the Fisher information on the regression function. The prior is named I-prior. For the present model, it is Gaussian with covariance kernel proportional to the Fisher information, and mean chosen a priori (e.g., 0). The I-prior has the intuitively appealing property that the more information is available about a linear functional of the regression function, the larger its prior variance, and, broadly speaking, the less influential the prior is on the posterior. Unlike the Jeffreys prior, it can be used in high dimensional settings. The I-prior methodology can be used as a principled alternative to Tikhonov regularization, which suffers from well-known theoretical problems which are briefly reviewed. The regression function is assumed to lie in a reproducing kernel Hilbert space (RKHS) over a low or high dimensional covariate space, giving a high degree of generality. Analysis of some real data sets and a small-scale simulation study show competitive performance of the I-prior methodology, which is implemented in the R-package iprior (Jamil, 2019).

math.ST

iprior: An R Package for Regression Modelling using I-priors

This is an overview of the R package iprior, which implements a unified methodology for fitting parametric and nonparametric regression models, including additive models, multilevel models, and models with one or more functional covariates. Based on the principle of maximum entropy, an I-prior is an objective Gaussian process prior for the regression function with covariance kernel equal to its Fisher information. The regression function is estimated by its posterior mean under the I-prior, and hyperparameters are estimated via maximum marginal likelihood. Estimation of I-prior models is simple and inference straightforward, while small and large sample predictive performances are comparative, and often better, to similar leading state-of-the-art models. We illustrate the use of the iprior package by analysing a simulated toy data set as well as three real-data examples, in particular, a multilevel data set, a longitudinal data set, and a dataset involving a functional covariate.

stat.ME

Estimation for Dynamic and Static Panel Probit Models with Large Individual Effects

For discrete panel data, the dynamic relationship between successive observations is often of interest. We consider a dynamic probit model for short panel data. A problem with estimating the dynamic parameter of interest is that the model contains a large number of nuisance parameters, one for each individual. Heckman proposed to use maximum likelihood estimation of the dynamic parameter, which, however, does not perform well if the individual effects are large. We suggest new estimators for the dynamic parameter, based on the assumption that the individual parameters are random and possibly large. Theoretical properties of our estimators are derived and a simulation study shows they have some advantages compared to Heckman's estimator.

stat.ME

A consistent test of independence based on a sign covariance related to Kendall's tau

The most popular ways to test for independence of two ordinal random variables are by means of Kendall's tau and Spearman's rho. However, such tests are not consistent, only having power for alternatives with ``monotonic'' association. In this paper, we introduce a natural extension of Kendall's tau, called $τ^*$, which is non-negative and zero if and only if independence holds, thus leading to a consistent independence test. Furthermore, normalization gives a rank correlation which can be used as a measure of dependence, taking values between zero and one. A comparison with alternative measures of dependence for ordinal random variables is given, and it is shown that, in a well-defined sense, $τ^*$ is the simplest, similarly to Kendall's tau being the simplest of ordinal measures of monotone association. Simulation studies show our test compares well with the alternatives in terms of average $p$-values.

math.ST

A Kernel Test for Three-Variable Interactions

We introduce kernel nonparametric tests for Lancaster three-variable interaction and for total independence, using embeddings of signed measures into a reproducing kernel Hilbert space. The resulting test statistics are straightforward to compute, and are used in powerful interaction tests, which are consistent against all alternatives for a large family of reproducing kernels. We show the Lancaster test to be sensitive to cases where two independent causes individually have weak influence on a third dependent variable, but their combined effect has a strong influence. This makes the Lancaster test especially suited to finding structure in directed graphical models, where it outperforms competing nonparametric tests in detecting such V-structures.

stat.ME

Nonparametric testing of conditional independence by means of the partial copula

We propose a new method to test conditional independence of two real random variables $Y$ and $Z$ conditionally on an arbitrary third random variable $X$. %with $F_{.|.}$ representing conditional distribution functions, The partial copula is introduced, defined as the joint distribution of $U=F_{Y|X}(Y|X)$ and $V=F_{Z|X}(Z|X)$. We call this transformation of $(Y,Z)$ into $(U,V)$ the partial copula transform. It is easy to show that if $Y$ and $Z$ are continuous for any given value of $X$, then $Y\ind Z|X$ implies $U\ind V$. Conditional independence can then be tested by (i) applying the partial copula transform to the data points and (ii) applying a test of ordinary independence to the transformed data. In practice, $F_{Y|X}$ and $F_{Z|X}$ will need to be estimated, which can be done by, e.g., standard kernel methods. We show that under easily satisfied conditions, and for a very large class of test statistics for independence which includes the covariance, Kendall's tau, and Hoeffding's test statistic, the effect of this estimation vanishes asymptotically. Thus, for large samples, the estimation can be ignored and we have a simple method which can be used to apply a wide range of tests of independence, including ones with consistency for arbitrary alternatives, to test for conditional independence. A simulation study indicates good small sample performance. Advantages of the partial copula approach compared to competitors seem to be simplicity and generality.

math.ST