arXiv subjects

Ian Barnett

Publications and source records attributed to Ian Barnett.

At least 19 recordsLinked to original sources

arXiv2025

Bayesian Neural Networks vs. Mixture Density Networks: Theoretical and Empirical Insights for Uncertainty-Aware Nonlinear Modeling

This paper investigates two prominent probabilistic neural modeling paradigms: Bayesian Neural Networks (BNNs) and Mixture Density Networks (MDNs) for uncertainty-aware nonlinear regression. While BNNs incorporate epistemic uncertainty by placing prior distributions over network parameters, MDNs directly model the conditional output distribution, thereby capturing multimodal and heteroscedastic data-generating mechanisms. We present a unified theoretical and empirical framework comparing these approaches. On the theoretical side, we derive convergence rates and error bounds under H\"older smoothness conditions, showing that MDNs achieve faster Kullback-Leibler (KL) divergence convergence due to their likelihood-based nature, whereas BNNs exhibit additional approximation bias induced by variational inference. Empirically, we evaluate both architectures on synthetic nonlinear datasets and a radiographic benchmark (RSNA Pediatric Bone Age Challenge). Quantitative and qualitative results demonstrate that MDNs more effectively capture multimodal responses and adaptive uncertainty, whereas BNNs provide more interpretable epistemic uncertainty under limited data. Our findings clarify the complementary strengths of posterior-based and likelihood-based probabilistic learning, offering guidance for uncertainty-aware modeling in nonlinear systems.

stat.CO↗

arXiv2025

Multiple Hypothesis Testing To Estimate The Number Of Communities in Stochastic Block Models

Clustering of single-cell RNA sequencing (scRNA-seq) datasets can give key insights into the biological functions of cells. Therefore, it is not surprising that network-based community detection methods (one of the better clustering methods) are increasingly being used for the clustering of scRNA-seq datasets. The main challenge in implementing network-based community detection methods for scRNA-seq datasets is that these methods \emph{apriori} require the true number of communities or blocks for estimating the community memberships. Although there are existing methods for estimating the number of communities, they are not suitable for noisy scRNA-seq datasets. Moreover, we require an appropriate method for extracting suitable networks from scRNA-seq datasets. For addressing these issues, we present a two-fold solution: i) a simple likelihood-based approach for extracting stochastic block models (SBMs) out of scRNA-seq datasets, ii) a new sequential multiple testing (SMT) method for estimating the number of communities in SBMs. We study the theoretical properties of SMT and establish its consistency under moderate sparsity conditions. In addition, we compare the numerical performance of the SMT with several existing methods. We also show that our approach performs competitively well against existing methods for estimating the number of communities on benchmark scRNA-seq datasets. Finally, we use our approach for estimating subgroups of a human retina bipolar single cell dataset.

stat.ME↗

arXiv2025

A Hessian-informed hyperparameter optimization for differential learning rate

Differential learning rate (DLR), a technique that applies different learning rates to different model parameters, has been widely used in deep learning and achieved empirical success via its various forms. For example, parameter-efficient fine-tuning (PEFT) applies zero learning rates to most parameters so as to significantly save the computational cost. At the core, DLR leverages the observation that different parameters can have different loss curvature, which is hard to characterize in general. We propose the Hessian-informed differential learning rate (Hi-DLR), an efficient approach that solves the hyperparameter optimization (HPO) of learning rates and captures the loss curvature for any model and optimizer adaptively. Given a proper grouping of parameters, we empirically demonstrate that Hi-DLR can improve the convergence by dynamically determining the learning rates during the training.

cs.LG↗

arXiv2024

Semiparametric quantile functional regression analysis of adolescent physical activity distributions in the presence of missing data

In the age of digital healthcare, passively collected physical activity profiles from wearable sensors are a preeminent tool for evaluating health outcomes. In order to fully leverage the vast amounts of data collected through wearable accelerometers, we propose to use quantile functional regression to model activity profiles as distributional outcomes through quantile responses, which can be used to evaluate activity level differences across covariates based on any desired distributional summary. Our proposed framework addresses two key problems not handled in existing distributional regression literature. First, we use spline mixed model formulations in the basis space to model nonparametric effects of continuous predictors on the distributional response. Second, we address the underlying missingness problem that is common in these types of wearable data but typically not addressed. We show that the missingness can induce bias in the subject-specific distributional summaries that leads to biased distributional regression estimates and even bias the frequently used scalar summary measures, and introduce a nonparametric function-on-function modeling approach that adjusts for each subject's missingness profile to address this problem. We evaluate our nonparametric modeling and missing data adjustment using simulation studies based on realistically simulated activity profiles and use it to gain insights into adolescent activity profiles from the Teen Environment and Neighborhood study.

stat.ME↗

arXiv2024

The Cox-Polya-Gamma Algorithm for Flexible Bayesian Inference of Multilevel Survival Models

Bayesian Cox semiparametric regression is an important problem in many clinical settings. Bayesian procedures provide finite-sample inference and naturally incorporate prior information if MCMC algorithms and posteriors are well behaved. Survival analysis should also be able to incorporate multilevel modeling such as case weights, frailties and smoothing splines, in a straightforward manner. To tackle these modeling challenges, we propose the Cox-Polya-Gamma (Cox-PG) algorithm for Bayesian multilevel Cox semiparametric regression and survival functions. Our novel computational procedure succinctly addresses the difficult problem of monotonicity constrained modeling of the nonparametric baseline cumulative hazard along with multilevel regression. We develop two key strategies. First, we exploit an approximation between Cox models and negative binomial processes through the Poisson process to reduce Bayesian computation to iterative Gaussian sampling. Next, we appeal to sufficient dimension reduction to address the difficult computation of nonparametric baseline cumulative hazard, allowing for the collapse of the Markov transition within the Gibbs sampler based on beta sufficient statistics. In addition, we explore conditions for uniform ergodicity of the Cox-PG algorithm. We demonstrate our multilevel modeling approach using open source data and simulations. We provide software for our Bayesian procedure in the supplement.

stat.ME↗

arXiv2024

Two-Sample Hypothesis Testing for Large Random Graphs of Unequal Size

Two-sample hypothesis testing for large graphs is popular in cognitive science, probabilistic machine learning and artificial intelligence. While numerous methods have been proposed in the literature to address this problem, less attention has been devoted to scenarios involving graphs of unequal size or situations where there are only one or a few samples of graphs. In this article, we propose a Frobenius test statistic tailored for small sample sizes and unequal-sized random graphs to test whether they are generated from the same model or not. Our approach involves an algorithm for generating bootstrapped adjacency matrices from estimated community-wise edge probability matrices, forming the basis of the Frobenius test statistic. We derive the asymptotic distribution of the proposed test statistic and validate its stability and efficiency in detecting minor differences in underlying models through simulations. Furthermore, we explore its application to fMRI data where we are able to distinguish brain activity patterns when subjects are exposed to sentences and pictures for two different stimuli and the control group.

stat.ME↗

arXiv2024

A Generalized Estimating Equation Approach to Network Regression

Regression models applied to network data where node attributes are the dependent variables poses a methodological challenge. As has been well studied, naive regression neither properly accounts for community structure, nor does it account for the dependent variable acting as both model outcome and covariate. To address this methodological gap, we propose a network regression model motivated by the important observation that controlling for community structure can, when a network is modular, significantly account for meaningful correlation between observations induced by network connections. We propose a generalized estimating equation (GEE) approach to learn model parameters based on clusters defined through any single-membership community detection algorithm applied to the observed network. We provide a necessary condition on the network size and edge formation probabilities to establish the asymptotic normality of the model parameters under the assumption that the graph structure is a stochastic block model. We evaluate the performance of our approach through simulations and apply it to estimate the joint impact of baseline covariates and network effects on COVID-19 incidence rate among countries connected by a network of commercial airline traffic. We find that during the beginning of the pandemic the network effect has some influence, the percentage of urban population has more influence on the incidence rate compared to the network effect after the travel ban was in effect.

stat.ME↗

arXiv2023

Non-asymptotic inference for multivariate change point detection

Traditional methods for inference in change point detection often rely on a large number of observed data points and can be inaccurate in non-asymptotic settings. With the rise of mobile health and digital phenotyping studies, where patients are monitored through the use of smartphones or other digital devices, change point detection is needed in non-asymptotic settings where it may be important to identify behavioral changes that occur just days before an adverse event such as relapse or suicide. Furthermore, analytical and computationally efficient means of inference are necessary for the monitoring and online analysis of large-scale digital phenotyping cohorts. We extend the result for asymptotic tail probabilities of the likelihood ratio test to the multivariate change point detection setting, and demonstrate through simulation its inaccuracy when the number of observed data points is not large. We propose a non-asymptotic approach for inference on the likelihood ratio test, and compare the efficiency of this estimated p-value to the popular empirical p-value obtained through simulation of the null distribution. The accuracy and power of this approach relative to competing methods is demonstrated through simulation and through the detection of a change point in the behavior of a patient with schizophrenia in the week prior to relapse.

stat.ME↗

arXiv2023

The impact of the COVID-19 pandemic on daily rhythms

The COVID-19 pandemic has significantly impacted daily activity rhythms and life routines. Understanding the dynamics of these impacts on different groups of people is essential for creating environments where people's lives and well-being are least disturbed during such circumstances. Starting in June 2021, we conducted a year-long study to collect high-resolution data from fitness trackers as well as answers to monthly questionnaires from 128 working adults. Using questionnaires, we investigate how routines of exercising and working have changed throughout the pandemic for different people. In addition to that, for each person in the study, we build temporal distributions of daily step counts to quantify their daily movement rhythms and use the inverse of the Earth mover's distance between different movement rhythms to quantify the movement consistency over time. Throughout the pandemic, our cohort shows a shift in exercise routines, manifested in a decrease in time spent on non-walking physical exercises as opposed to the unchanged amount of time spent on walking. In terms of daily rhythms of movement, we show that migrants and those who live alone demonstrate a lower level of consistency of daily rhythms of movement compared to their counterparts. We also observe a relationship between movement and on-site work attendance, as participants who go to work (as opposed to working remotely) also tend to maintain more consistent daily rhythms of movement. Men and migrants show a faster pace in going back to work after the decrease in restriction measures that were set in place due to the pandemic. Our results quantitatively demonstrate the unequal effect of the pandemic among different sub-populations and inform organizations and policymakers to provide more adequate support and adapt to the different needs of different groups in the post-pandemic era.

cs.HC↗

arXiv2023

Permutation-based Hypothesis Testing for Neural Networks

Neural networks are powerful predictive models, but they provide little insight into the nature of relationships between predictors and outcomes. Although numerous methods have been proposed to quantify the relative contributions of input features, statistical inference and hypothesis testing of feature associations remain largely unexplored. We propose a permutation-based approach to testing that uses the partial derivatives of the network output with respect to specific inputs to assess both the significance of input features and whether significant features are linearly associated with the network output. These tests, which can be flexibly applied to a variety of network architectures, enhance the explanatory power of neural networks, and combined with powerful predictive capability, extend the applicability of these models.

stat.ME↗

arXiv2022

Analysis of block slice samplers for Bayesian GLMMs and GAMs with linear inequality and shape constraints

Exponential family models, generalized linear models (GLMs), generalized linear mixed models (GLMMs) and generalized additive models (GAMs) are widely used methods in statistics. However, many scientific applications necessitate constraints be placed on model parameters such as shape and linear inequality constraints. Constrained estimation and inference of parameters remains a pervasive problem in statistics where many methods rely on modifying rigid large sample theory assumptions for inference. We propose a flexible slice sampler Gibbs algorithm for Bayesian GLMMs and GAMs with linear inequality and shape constraints. We prove our posterior samples follow a Markov chain central limit theorem (CLT) by proving uniform ergodicity of our Markov chain and existence of the a moment generating function for our posterior distributions. We use our CLT results to derive joint bands and multiplicity adjusted Bayesian inference for nonparametric functional effects. Our rigorous CLT results address a shortcoming in the literature by obtaining valid estimation and inference on constrained parameters in finite sample settings. Our algorithmic and proof techniques are adaptable to a myriad of important statistical modeling problems. We apply our Bayesian GAM to a real data analysis example involving proportional odds regression for concussion recovery in children with shape constraints and smoothed nonparametric effects. We obtain multiplicity adjusted inference on monotonic nonparametric time effect to elucidate recovery trends in children as a function of time.

stat.ME↗

arXiv2022

Combining Mixed Effects Hidden Markov Models with Latent Alternating Recurrent Event Processes to Model Diurnal Active-Rest Cycles

Data collected from wearable devices and smartphones can shed light on an individual's pattern of behavioral and circadian routine. Phone use can be modeled as alternating event process, between the state of active use and the state of being idle. Markov chains and alternating recurrent event models are commonly used to model state transitions in cases such as these, and the incorporation of random effects can be used to introduce diurnal effects. While state labels can be derived prior to modeling dynamics, this approach omits informative regression covariates that can influence state memberships. We instead propose an alternating recurrent event proportional hazards (PH) regression to model the transitions between latent states. We propose an Expectation-Maximization (EM) algorithm for imputing latent state labels and estimating regression parameters. We show that our E-step simplifies to the hidden Markov model (HMM) forward-backward algorithm, allowing us to recover a HMM with logistic regression transition probabilities. In addition, we show that PH modeling of discrete-time transitions implicitly penalizes the logistic regression likelihood and results in shrinkage estimators for the relative risk. We derive asymptotic distributions for our model parameter estimates and compare our approach against competing methods through simulation as well as in a digital phenotyping study that followed smartphone use in a cohort of adolescents with mood disorders.

stat.ME↗

arXiv2022

Selecting a significance level in sequential testing procedures for community detection

While there have been numerous sequential algorithms developed to estimate community structure in networks, there is little available guidance and study of what significance level or stopping parameter to use in these sequential testing procedures. Most algorithms rely on prespecifiying the number of communities or use an arbitrary stopping rule. We provide a principled approach to selecting a nominal significance level for sequential community detection procedures by controlling the tolerance ratio, defined as the ratio of underfitting and overfitting probability of estimating the number of clusters in fitting a network. We introduce an algorithm for specifying this significance level from a user-specified tolerance ratio, and demonstrate its utility with a sequential modularity maximization approach in a stochastic block model framework. We evaluate the performance of the proposed algorithm through extensive simulations and demonstrate its utility in controlling the tolerance ratio in single-cell RNA sequencing clustering by cell type and by clustering a congressional voting network.

stat.ME↗

arXiv2022

Confidence Intervals for the Number of Components in Factor Analysis and Principal Components Analysis via Subsampling

Factor analysis (FA) and principal component analysis (PCA) are popular statistical methods for summarizing and explaining the variability in multivariate datasets. By default, FA and PCA assume the number of components or factors to be known \emph{a priori}. However, in practice the users first estimate the number of factors or components and then perform FA and PCA analyses using the point estimate. Therefore, in practice the users ignore any uncertainty in the point estimate of the number of factors or components. For datasets where the uncertainty in the point estimate is not ignorable, it is prudent to perform FA and PCA analyses for the range of positive integer values in the confidence intervals for the number of factors or components. We address this problem by proposing a subsampling-based data-intensive approach for estimating confidence intervals for the number of components in FA and PCA. We study the coverage probability of the proposed confidence intervals and provide non-asymptotic theoretical guarantees concerning the accuracy of the confidence intervals. As a byproduct, we derive the first-order \emph{Edgeworth expansion} for spiked eigenvalues of the sample covariance matrix when the data matrix is generated under a factor model. We also demonstrate the usefulness of our approach through numerical simulations and by applying our approach for estimating confidence intervals for the number of factors of the genotyping dataset of the Human Genome Diversity Project.

stat.ME↗

arXiv2022

Autoregressive Mixture Models for Serial Correlation Clustering of Time Series Data

Clustering time series into similar groups can improve models by combining information across like time series. While there is a well developed body of literature for clustering of time series, these approaches tend to generate clusters independently of model training which can lead to poor model fit. We propose a novel distributed approach that simultaneously clusters and fits autoregression models for groups of similar individuals. We apply a Wishart mixture model so as to cluster individuals while modeling the corresponding autocovariance matrices at the same time. The fitted Wishart scale matrices map to cluster-level autoregressive coefficients through the Yule-Walker equations, fitting robust parsimonious autoregressive mixture models. This approach is able to discern differences in underlying autocorrelation variation of time series in settings with large heterogeneous datasets. We prove consistency of our cluster membership estimator, asymptotic distributions of coefficients and compare our approach against competing methods through simulation as well as by fitting a COVID-19 forecast model.

stat.ME↗

arXiv2022

Multiple Hypothesis Testing To Estimate The Number of Communities in Sparse Stochastic Block Models

Network-based clustering methods frequently require the number of communities to be specified \emph{a priori}. Moreover, most of the existing methods for estimating the number of communities assume the number of communities to be fixed and not scale with the network size $n$. The few methods that assume the number of communities to increase with the network size $n$ are only valid when the average degree $d$ of a network grows at least as fast as $O(n)$ (i.e., the dense case) or lies within a narrow range. This presents a challenge in clustering large-scale network data, particularly when the average degree $d$ of a network grows slower than the rate of $O(n)$ (i.e., the sparse case). To address this problem, we proposed a new sequential procedure utilizing multiple hypothesis tests and the spectral properties of Erdös Rényi graphs for estimating the number of communities in sparse stochastic block models (SBMs). We prove the consistency of our method for sparse SBMs for a broad range of the sparsity parameter. As a consequence, we discover that our method can estimate the number of communities $K^{(n)}_{\star}$ with $K^{(n)}_{\star}$ increasing at the rate as high as $O(n^{(1 - 3γ)/(4 - 3γ)})$, where $d = O(n^{1 - γ})$. Moreover, we show that our method can be adapted as a stopping rule in estimating the number of communities in binary tree stochastic block models. We benchmark the performance of our method against other competing methods on six reference single-cell RNA sequencing datasets. Finally, we demonstrate the usefulness of our method through numerical simulations and by using it for clustering real single-cell RNA-sequencing datasets.

stat.ME↗

arXiv2020

The Asymptotic Distribution of Modularity in Weighted Signed Networks

Modularity is a popular metric for quantifying the degree of community structure within a network. The distribution of the largest eigenvalue of a network's edge weight or adjacency matrix is well studied and is frequently used as a substitute for modularity when performing statistical inference. However, we show that the largest eigenvalue and modularity are asymptotically uncorrelated, which suggests the need for inference directly on modularity itself when the network size is large. To this end, we derive the asymptotic distributions of modularity in the case where the network's edge weight matrix belongs to the Gaussian Orthogonal Ensemble, and study the statistical power of the corresponding test for community structure under some alternative model. We empirically explore universality extensions of the limiting distribution and demonstrate the accuracy of these asymptotic distributions through type I error simulations. We also compare the empirical powers of the modularity based tests with some existing methods. Our method is then used to test for the presence of community structure in two real data applications.

stat.ME↗

arXiv2019

Novel Non-Negative Variance Estimator for (Modified) Within-Cluster Resampling

This article proposes a novel variance estimator for within-cluster resampling (WCR) and modified within-cluster resampling (MWCR) - two existing methods for analyzing longitudinal data. WCR is a simple but computationally intensive method, in which a single observation is randomly sampled from each cluster to form a new dataset. This process is repeated numerous times, and in each resampled dataset (or outputation), we calculate beta using a generalized linear model. The final resulting estimator is an average across estimates from all outputations. MWCR is an extension of WCR that can account for the within-cluster correlation of the dataset; consequently, there are two noteworthy differences: 1) in MWCR, each resampled dataset is formed by randomly sampling multiple observations without replacement from each cluster and 2) generalized estimating equations (GEEs) are used to estimate the parameter of interest. While WCR and MWCR are relatively simple to implement, a key challenge is that the proposed moment-based estimator is often times negative in practice. Our modified variance estimator is not only strictly positive, but simulations show that it preserves the type I error and allows statistical power gains associated with MWCR to be realized.

stat.ME↗