arXiv subjects

Jonas Wallin

Publications and source records attributed to Jonas Wallin.

At least 37 records · Page 2Linked to original sources

arXiv2025

Nonparametric Shrinkage Estimation in High Dimensional Generalized Linear Models via Polya Trees

Regularization in fitting regression models has been a highly active topic of research in the past few decades, but most of the existing methods are designed for particular situations, e.g. for the case of a sparse coefficient vector. We consider the problem of designing $\textit{universally}$ optimal regularized estimators in a given generalized linear model with fixed effects. First, we propose as a contender the Bayes estimator against an $\textit{ideal}$ prior that assigns equal mass to every permutation of the fixed coefficient vector, thus depending on the true coefficients only through their empirical CDF. We prove some optimality properties of this oracle estimator in both the frequentist and Bayesian frameworks. To compete with the oracle estimator, we posit a hierarchical Bayes model where the individual coefficients are modeled as i.i.d. draws from a common distribution $π$, which is in turn assigned a Polya tree prior to reflect indefiniteness. We demonstrate in examples that the posterior mean of $π$ under the postulated model adapts nonparametrically to the empirical CDF of the true coefficients. Correspondingly, the posterior means of the coefficients themselves are used to mimic the ideal estimator. Numerical experiments show that our method has better estimation and prediction accuracy compared to various parametric and nonparametric alternatives, from relatively standard $L_p$-regularized estimators to modern penalized-likelihood and Bayesian estimators for high dimensional regression.

stat.ME↗

arXiv2025

An explicit link between graphical models and Gaussian Markov random fields on metric graphs

We derive an explicit link between Gaussian Markov random fields on metric graphs and graphical models, and in particular show that a Markov random field restricted to the vertices of the graph is, under mild regularity conditions, a Gaussian graphical model with a distribution which is faithful to its pairwise independence graph, which coincides with the neighbor structure of the metric graph. This is used to show that there are no Gaussian random fields on general metric graphs which are both Markov and isotropic in some suitably regular metric on the graph, such as the geodesic or resistance metrics.

math.PR↗

arXiv2025

Unveiling low-dimensional patterns induced by convex non-differentiable regularizers

Popular regularizers with non-differentiable penalties, such as Lasso, Elastic Net, Generalized Lasso, or SLOPE, reduce the dimension of the parameter space by inducing sparsity or clustering in the estimators' coordinates. In this paper, we focus on linear regression and explore the asymptotic distributions of the resulting low-dimensional patterns when the number of regressors $p$ is fixed, the number of observations $n$ goes to infinity, and the penalty function increases at the rate of $\sqrt{n}$. While the asymptotic distribution of the rescaled estimation error can be derived by relatively standard arguments, convergence of patterns requires a separate proof, which is yet missing from the literature, even for the simplest case of Lasso. To fill this gap, we use the Hausdorff distance as a suitable mode of convergence for subdifferentials, resulting in the desired pattern convergence. Furthermore, we derive the exact limiting probability of recovering the true model pattern. This probability goes to 1 if and only if the penalty scaling constant diverges to infinity and the regularizer-specific asymptotic irrepresentability condition is satisfied. We then propose simple two-step procedures that asymptotically recover the model patterns, irrespective of whether the irrepresentability condition holds or not. Interestingly, our theory shows that Fused Lasso cannot reliably recover its own clustering pattern, even for independent regressors. It also demonstrates how this problem can be resolved by "concavifying" the Fused Lasso penalty coefficients. Additionally, sampling from the asymptotic error distribution facilitates comparisons between different regularizers. We provide short simulation studies showcasing an illustrative comparison between the asymptotic properties of Lasso, Fused Lasso, and SLOPE.

math.ST↗

arXiv2024

Markov properties of Gaussian random fields on compact metric graphs

There has recently been much interest in Gaussian fields on linear networks and, more generally, on compact metric graphs. One proposed strategy for defining such fields on a metric graph $Γ$ is through a covariance function that is isotropic in a metric on the graph. Another is through a fractional-order differential equation $L^{α/2} (τu) = \mathcal{W}$ on $Γ$, where $L = κ^2 - \nabla(a\nabla)$ for (sufficiently nice) functions $κ, a$, and $\mathcal{W}$ is Gaussian white noise. We study Markov properties of these two types of fields. First, we show that no Gaussian random fields exist on general metric graphs that are both isotropic and Markov. Then, we show that the second type of fields, the generalized Whittle--Matérn fields, are Markov if $α\in\mathbb{N}$, and conversely, if $a$ and $κ$ are constant and $u$ is Markov, then $α\in\mathbb{N}$. Further, if $α\in\mathbb{N}$, a generalized Whittle--Matérn field $u$ is Markov of order $α$, which means that the field $u$ in one region $S\subsetΓ$ is conditionally independent of $u$ in $Γ\setminus S$ given the values of $u$ and its $α-1$ derivatives on $\partial S$. Finally, we provide two results as consequences of the theory developed: first we prove that the Markov property implies an explicit characterization of $u$ on a fixed edge $e$, revealing that the conditional distribution of $u$ on $e$ given the values at the two vertices connected to $e$ is independent of the geometry of $Γ$; second, we show that the solution to $L^{1/2}(τu) = \mathcal{W}$ on $Γ$ can obtained by conditioning independent generalized Whittle--Matérn processes on the edges, with $α=1$ and Neumann boundary conditions, on being continuous at the vertices.

math.PR↗

arXiv2024

Spatial confounding under infill asymptotics

The estimation of regression parameters in spatially referenced data plays a crucial role across various scientific domains. A common approach involves employing an additive regression model to capture the relationship between observations and covariates, accounting for spatial variability not explained by the covariates through a Gaussian random field. While theoretical analyses of such models have predominantly focused on prediction and covariance parameter inference, recent attention has shifted towards understanding the theoretical properties of regression coefficient estimates, particularly in the context of spatial confounding. This article studies the effect of misspecified covariates, in particular when the misspecification changes the smoothness. We analyze the theoretical properties of the generalize least-square estimator under infill asymptotics, and show that the estimator can have counter-intuitive properties. In particular, the estimated regression coefficients can converge to zero as the number of observations increases, despite high correlations between observations and covariates. Perhaps even more surprising, the estimates can diverge to infinity under certain conditions. Through an application to temperature and precipitation data, we show that both behaviors can be observed for real data. Finally, we propose a simple fix to the problem by adding a smoothing step in the regression.

math.ST↗

arXiv2023

Statistical inference for Gaussian Whittle-Matérn fields on metric graphs

Whittle-Matérn fields are a recently introduced class of Gaussian processes on metric graphs, which are specified as solutions to a fractional-order stochastic differential equation. Unlike earlier covariance-based approaches for specifying Gaussian fields on metric graphs, the Whittle-Matérn fields are well-defined for any compact metric graph and can provide Gaussian processes with differentiable sample paths. We derive the main statistical properties of the model class, particularly the consistency and asymptotic normality of maximum likelihood estimators of model parameters and the necessary and sufficient conditions for asymptotic optimality properties of linear prediction based on the model with misspecified parameters. The covariance function of the Whittle-Matérn fields is generally unavailable in closed form, and they have therefore been challenging to use for statistical inference. However, we show that for specific values of the fractional exponent, when the fields have Markov properties, likelihood-based inference and spatial prediction can be performed exactly and computationally efficiently. This facilitates using the Whittle-Matérn fields in statistical applications involving big datasets without the need for any approximations. The methods are illustrated via an application to modeling of traffic data, where allowing for differentiable processes dramatically improves the results.

stat.ME↗

arXiv2023

Weak pattern convergence for SLOPE and its robust versions

The Sorted L-One Estimator (SLOPE) is a popular regularization method in regression, which induces clustering of the estimated coefficients. That is, the estimator can have coefficients of identical magnitude. In this paper, we derive an asymptotic distribution of SLOPE for the ordinary least squares, Huber, and Quantile loss functions, and use it to study the clustering behavior in the limit. This requires a stronger type of convergence since clustering properties do not follow merely from the classical weak convergence. For this aim, we utilize the Hausdorff distance, which provides a suitable notion of convergence for the penalty subdifferentials and a bridge toward weak convergence of the clustering pattern. We establish asymptotic control of the false discovery rate for the asymptotic orthogonal design of the regressor. We also show how to extend the framework to a broader class of regularizers other than SLOPE.

math.ST↗

arXiv2023

Gaussian Whittle-Matérn fields on metric graphs

We define a new class of Gaussian processes on compact metric graphs such as street or river networks. The proposed models, the Whittle--Matérn fields, are defined via a fractional stochastic differential equation on the compact metric graph and are a natural extension of Gaussian fields with Matérn covariance functions on Euclidean domains to the non-Euclidean metric graph setting. Existence of the processes, as well as some of their main properties, such as sample path regularity are derived. The model class in particular contains differentiable processes. To the best of our knowledge, this is the first construction of a differentiable Gaussian process on general compact metric graphs. Further, we prove an intrinsic property of these processes: that they do not change upon addition or removal of vertices with degree two. Finally, we obtain Karhunen--Loève expansions of the processes, provide numerical experiments, and compare them to Gaussian processes with isotropic covariance functions.

math.ST↗

arXiv2022

Coordinate Descent for SLOPE

The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we show that our method outperforms a long list of competing algorithms.

math.OC↗

arXiv2022

The Hessian Screening Rule

Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for $\ell_1$-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.

stat.ML↗

arXiv2022

The Strong Screening Rule for SLOPE

Extracting relevant features from data sets where the number of observations ($n$) is much smaller then the number of predictors ($p$) is a major challenge in modern statistics. Sorted L-One Penalized Estimation (SLOPE), a generalization of the lasso, is a promising method within this setting. Current numerical procedures for SLOPE, however, lack the efficiency that respective tools for the lasso enjoy, particularly in the context of estimating a complete regularization path. A key component in the efficiency of the lasso is predictor screening rules: rules that allow predictors to be discarded before estimating the model. This is the first paper to establish such a rule for SLOPE. We develop a screening rule for SLOPE by examining its subdifferential and show that this rule is a generalization of the strong rule for the lasso. Our rule is heuristic, which means that it may discard predictors erroneously. We present conditions under which this may happen and show that such situations are rare and easily safeguarded against by a simple check of the optimality conditions. Our numerical experiments show that the rule performs well in practice, leading to improvements by orders of magnitude for data in the $p \gg n$ domain, as well as incurring no additional computational overhead when $n \gg p$. We also examine the effect of correlation structures in the design matrix on the rule and discuss algorithmic strategies for employing the rule. Finally, we provide an efficient implementation of the rule in our R package SLOPE.

stat.ML↗

arXiv2022

Local scale invariance and robustness of proper scoring rules

Averages of proper scoring rules are often used to rank probabilistic forecasts. In many cases, the individual terms in these averages are based on observations and forecasts from different distributions. We show that some of the most popular proper scoring rules, such as the continuous ranked probability score (CRPS), give more importance to observations with large uncertainty which can lead to unintuitive rankings. To describe this issue, we define the concept of local scale invariance for scoring rules. A new class of generalized proper kernel scoring rules is derived and as a member of this class we propose the scaled CRPS (SCRPS). This new proper scoring rule is locally scale invariant and therefore works in the case of varying uncertainty. Like CRPS it is computationally available for output from ensemble forecasts, and does not require the ability to evaluate densities of forecasts. We further define robustness of scoring rules, show why this also is an important concept for average scores, and derive new proper scoring rules that are robust against outliers. The theoretical findings are illustrated in three different applications from spatial statistics, stochastic volatility models, and regression for count data.

math.ST↗

arXiv2021

Efficient methods for Gaussian Markov random fields under sparse linear constraints

Methods for inference and simulation of linearly constrained Gaussian Markov Random Fields (GMRF) are computationally prohibitive when the number of constraints is large. In some cases, such as for intrinsic GMRFs, they may even be unfeasible. We propose a new class of methods to overcome these challenges in the common case of sparse constraints, where one has a large number of constraints and each only involves a few elements. Our methods rely on a basis transformation into blocks of constrained versus non-constrained subspaces, and we show that the methods greatly outperform existing alternatives in terms of computational cost. By combining the proposed methods with the stochastic partial differential equation approach for Gaussian random fields, we also show how to formulate Gaussian process regression with linear constraints in a GMRF setting to reduce computational cost. This is illustrated in two applications with simulated data.

stat.ME↗

arXiv2020

Nowcasting Covid-19 statistics reported withdelay: a case-study of Sweden

The new corona virus disease -- COVID-2019 -- is rapidly spreading through the world. The availability of unbiased timely statistics of trends in disease events are a key to effective responses. But due to reporting delays, the most recently reported numbers are frequently underestimating of the total number of infections, hospitalizations and deaths creating an illusion of a downward trend. Here we describe a statistical methodology for predicting true daily quantities and their uncertainty, estimated using historical reporting delays. The methodology takes into account the observed distribution pattern of the lag. It is derived from the removal method, a well-established estimation framework in the field of ecology.

q-bio.PE↗

arXiv2020

Generalized bounds for active subspaces

In this article, we consider scenarios in which traditional estimates for the active subspace method based on probabilistic Poincaré inequalities are not valid due to unbounded Poincaré constants. Consequently, we propose a framework that allows to derive generalized estimates in the sense that it enables to control the trade-off between the size of the Poincaré constant and a weaker order of the final error bound. In particular, we investigate independently exponentially distributed random variables in dimension two or larger and give explicit expressions for corresponding Poincaré constants showing their dependence on the dimension of the problem. Finally, we suggest possibilities for future work that aim for extending the class of distributions applicable to the active subspace method as we regard this as an opportunity to enlarge its usability.

math.PR↗

arXiv2019

Multivariate type G Matérn stochastic partial differential equation random fields

For many applications with multivariate data, random field models capturing departures from Gaussianity within realisations are appropriate. For this reason, we formulate a new class of multivariate non-Gaussian models based on systems of stochastic partial differential equations with additive type G noise whose marginal covariance functions are of Matérn type. We consider four increasingly flexible constructions of the noise, where the first two are similar to existing copula-based models. In contrast to these, the latter two constructions can model non-Gaussian spatial data without replicates. Computationally efficient methods for likelihood-based parameter estimation and probabilistic prediction are proposed, and the flexibility of the suggested models is illustrated by numerical examples and two statistical applications.

stat.ME↗

arXiv2019

Robust joint modelling of longitudinal and survival data with a time-varying degrees-of-freedom parameter

Repeated measures of biomarkers have the potential of explaining hazards of survival outcomes. In practice, these measurements are intermittently measured and are known to be subject to substantial measurement error. Joint modelling of longitudinal and survival data enables us to associate intermittently measured error-prone biomarkers with risks of survival outcomes. Most of the joint models available in the literature have been built on the Gaussian assumption. This makes them sensitive to outliers. In this work, we study a range of robust models to address this issue. For medical data, it has been observed that outliers might occur with different frequencies over time. To address this, a new model with a time varying robustness is introduced. Through both a simulation study and analysis of two real-life data examples, this research not only stresses the need to account for longitudinal outliers in joint modelling research but also highlights the bias and inefficiency from not properly estimating the degrees-of-freedom parameter. Each technique presented in this work can be fitted using the R package robjm.

stat.AP↗

arXiv2018

Linear Mixed-Effects Models for Non-Gaussian Repeated Measurement Data

We consider the analysis of continuous repeated measurement outcomes that are collected through time, also known as longitudinal data. A standard framework for analysing data of this kind is a linear Gaussian mixed-effects model within which the outcome variable can be decomposed into fixed-effects, time-invariant and time-varying random-effects, and measurement noise. We develop methodology that, for the first time, allows any combination of these stochastic components to be non-Gaussian, using multivariate Normal variance-mean mixtures. We estimate parameters by max- imum likelihood, implemented with a novel, computationally efficient stochastic gradient algorithm. We obtain standard error estimates by inverting the observed Fisher-information matrix, and obtain the predictive distributions for the random-effects in both filtering (conditioning on past and current data) and smoothing (conditioning on all data) contexts. To implement these procedures, we intro- duce an R package, ngme. We re-analyse two data-sets, from cystic fibrosis and nephrology research, that were previously analysed using Gaussian linear mixed effects models.

stat.ME↗