arXiv subjects

Srijan Sengupta

Publications and source records attributed to Srijan Sengupta.

At least 19 recordsLinked to original sources

arXiv2026

Understanding Geopolitical Alignments Through Covariate Augmented Spectral Clustering of Heterogeneous UNGA Voting Data

Community detection is a fundamental problem in network analysis. While many existing methods focus on homogeneous networks, real world networks are often heterogeneous, involving multiple node types and interaction mechanisms. In addition, node specific covariates frequently provide valuable information about the underlying community structure. Existing methodologies typically account for either network heterogeneity or covariate information, but seldom both simultaneously. In this paper, we propose a covariate assisted spectral clustering framework for heterogeneous networks that jointly utilizes network connectivity in a heterogeneous setting and node level covariates. The proposed method extends covariate assisted spectral clustering to heterogeneous settings and operates directly on the heterogeneous network without relying on projection based simplifications. Under a heterogeneous node contextualized stochastic blockmodel, we establish theoretical guarantees for the proposed procedure, including concentration results, eigenspace perturbation bounds, and an explicit upper bound on the misclustering rate. Simulation studies demonstrate that incorporating covariate information substantially improves community recovery and consistently outperforms several benchmark methods. We further apply the proposed framework to United Nations General Assembly voting data, where it reveals meaningful geopolitical structures by combining voting interactions with auxiliary covariate information.

stat.AP↗

arXiv2026

Two-Sample Testing for Multivariate Cross-Correlation Functions with Applications to Gut-Brain Reward Learning

Cross-correlation functions (CCFs) are classical tools for studying lead-lag relationships between paired time series, but they are most often used descriptively rather than inferentially. Motivated by mouse experiments on gut-brain interactions in reward learning, we carry out a two-sample hypothesis test for formal statistical inference on collections of subject-specific CCF curves. In our application, each experimental session yields two related CCFs describing the temporal association of dopamine activity with locomotor velocity and acceleration, which leads naturally to a multivariate functional data formulation. We treat each empirical CCF as a functional observation indexed by lag and test equality of mean multivariate CCF functions across groups using integrated and maximum-type global statistics, \(F_{\mathrm{int}}\) and \(F_{\max}\), constructed from pointwise Hotelling \(T^2\) statistics. The integrated test targets broad differences across the lag domain, whereas the maximum test is sensitive to local differences. Applied to free-feeding and intragastric infusion datasets, the proposed methods detect substantial differences in dopamine-locomotion coupling across brain region and biological sex in the free-feeding experiment, with more selective effects in the infusion setting. The proposed framework provides a flexible and rigorous FDA-based approach for comparing dynamic dependence structures across experimental conditions.

stat.AP↗

arXiv2026

Predictive Subsampling for Scalable Inference in Networks

Current methods for statistical inference in networks often encounter substantial computational bottlenecks when applied to the massive network datasets that are increasingly common across scientific domains. In this paper, we develop \textit{Predictive Subsampling} (\texttt{PredSub}), a scalable framework for estimation and two-sample testing in networks. The central idea is to replace a full-sample estimation procedure by estimation on a random subsample, followed by out-of-sample prediction of the remaining vertices. This construction exploits the fact that the subsample provides an inferential anchor, which enables each remaining vertex to be incorporated in a \textit{predictive} manner through a fast vector operation. Building on this estimator, we develop two scalable procedures for two-sample testing, namely \texttt{PredSubTest} and \texttt{PureSubTest}. We establish finite-sample error bounds as well as estimation and testing consistency of the proposed methods in both Frobenius and two-to-infinity norms. These results formally characterize the trade-offs between statistical accuracy and computational efficiency with respect to the subsample size, the choice of test statistic, and the choice of norm. We demonstrate the empirical performance of the proposed methods through detailed simulation studies and two real-world applications involving DBLP coauthorship networks and the Cannes 2013 social media networks.

stat.ME↗

arXiv2026

Uncertainty Quantification for Named Entity Recognition via Full-Sequence and Subsequence Conformal Prediction

Named Entity Recognition (NER) serves as a foundational component in many natural language processing (NLP) pipelines. However, current NER models typically output a single predicted label sequence without any accompanying measure of uncertainty, leaving downstream applications vulnerable to cascading errors. In this paper, we introduce a general framework for adapting sequence-labeling-based NER models to produce uncertainty-aware prediction sets. These prediction sets are collections of full-sentence labelings that are guaranteed to contain the correct labeling with a user-specified confidence level. This approach serves a role analogous to confidence intervals in classical statistics by providing formal guarantees about the reliability of model predictions. Our method builds on conformal prediction, which offers finite-sample coverage guarantees under minimal assumptions. We design efficient nonconformity scoring functions to construct efficient, well-calibrated prediction sets that support both unconditional and class-conditional coverage. This framework accounts for heterogeneity across sentence length, language, entity type, and number of entities within a sentence. Empirical experiments on four NER models across three benchmark datasets demonstrate the broad applicability, validity, and efficiency of the proposed methods.

cs.CL↗

arXiv2025

Statistical inference for core-periphery structures

Core-periphery (CP) structure is an important meso-scale network property where nodes group into a small, densely interconnected {core} and a sparse {periphery} whose members primarily connect to the core rather than to each other. While this structure has been observed in numerous real-world networks, there has been minimal statistical formalization of it. In this work, we develop a statistical framework for CP structures by introducing a model-agnostic and generalizable population parameter which quantifies the strength of a CP structure at the level of the data-generating mechanism. We study this parameter under four canonical random graph models and establish theoretical guarantees for label recovery, including exact label recovery. Next, we construct intersection tests for validating the presence and strength of a CP structure under multiple null models, and prove theoretical guarantees for type I error and power. These tests provide a formal distinction between exogenous (or induced) and endogenous (or intrinsic) CP structure in heterogeneous networks, enabling a level of structural resolution that goes beyond merely detecting the presence of CP structure. The proposed methods show excellent performance on synthetic data, and our applications demonstrate that statistically significant CP structure is somewhat rare in real-world networks.

stat.ME↗

arXiv2025

A label-switching algorithm for fast core-periphery identification

Core-periphery (CP) structure is frequently observed in networks where the nodes form two distinct groups: a small, densely interconnected core and a sparse periphery. Borgatti and Everett (2000) proposed one of the most popular methods to identify and quantify CP structure by comparing the observed network with an ``ideal'' CP structure. While this metric has been widely used, an improved algorithm is still needed. In this work, we detail a greedy, label-switching algorithm to identify CP structure that is both fast and accurate. By leveraging a mathematical reformulation of the CP metric, our proposed heuristic offers an order-of-magnitude improvement on the number of operations compared to a naive implementation. We prove that the algorithm monotonically ascends to a local maximum while consistently yielding solutions within 90% of the global optimum on small toy networks. On synthetic networks, our algorithm exhibits superior classification accuracies and run-times compared to a popular competing method, and on one-real world network, it is 340 times faster.

stat.CO↗

arXiv2025

A Unified Framework for Community Detection and Model Selection in Blockmodels

Blockmodels are a foundational tool for modeling community structure in networks, with the stochastic blockmodel (SBM), degree-corrected blockmodel (DCBM), and popularity-adjusted blockmodel (PABM) forming a natural hierarchy of increasing generality. While community detection under these models has been extensively studied, much less attention has been paid to the model selection problem, i.e., determining which model best fits a given network. Building on recent theoretical insights about the spectral geometry of these models, we propose a unified framework for simultaneous community detection and model selection across the full blockmodel hierarchy. A key innovation is the use of loss functions that serve a dual role: they act as objective functions for community detection and as test statistics for hypothesis testing. We develop a greedy algorithm to minimize these loss functions and establish theoretical guarantees for exact label recovery and model selection consistency under each model. Extensive simulation studies demonstrate that our method achieves high accuracy in both tasks, outperforming or matching state-of-the-art alternatives. Applications to five real-world networks further illustrate the interpretability and practical utility of our approach. R code for implementing the method is available at https://github.com/subhankarbhadra/model-selection.

stat.ME↗

arXiv2025

Network Cross-Validation and Model Selection via Subsampling

Complex and larger networks are becoming increasingly prevalent in scientific applications in various domains. Although a number of models and methods exist for such networks, cross-validation on networks remains challenging due to the unique structure of network data. In this paper, we propose a general cross-validation procedure called NETCROP (NETwork CRoss-Validation using Overlapping Partitions). The key idea is to divide the original network into multiple subnetworks with a shared overlap part, producing training sets consisting of the subnetworks and a test set with the node pairs between the subnetworks. This train-test split provides the basis for a network cross-validation procedure that can be applied on a wide range of model selection and parameter tuning problems for networks. The method is computationally efficient for large networks as it uses smaller subnetworks for the training step. We provide methodological details and theoretical guarantees for several model selection and parameter tuning tasks using NETCROP. Numerical results demonstrate that NETCROP performs accurate cross-validation on a diverse set of network model selection and parameter tuning problems. The results also indicate that NETCROP is computationally much faster while being often more accurate than the existing methods for network cross-validation.

stat.ME↗

arXiv2025

Scalable community detection in massive networks via predictive assignment

Massive network datasets are becoming increasingly common in scientific applications. Existing community detection methods encounter significant computational challenges for such massive networks due to two reasons. First, the full network needs to be stored and analyzed on a single server, leading to high memory costs. Second, existing methods typically use matrix factorization or iterative optimization using the full network, resulting in high runtimes. We propose a strategy called \textit{predictive assignment} to enable computationally efficient community detection while ensuring statistical accuracy. The core idea is to avoid large-scale matrix computations by breaking up the task into a smaller matrix computation plus a large number of vector computations that can be carried out in parallel. Under the proposed method, community detection is carried out on a small subgraph to estimate the relevant model parameters. Next, each remaining node is assigned to a community based on these estimates. We prove that predictive assignment achieves strong consistency under the stochastic blockmodel and its degree-corrected version. We also demonstrate the empirical performance of predictive assignment on simulated networks and two large real-world datasets: DBLP (Digital Bibliography \& Library Project), a computer science bibliographical database, and the Twitch Gamers Social Network.

stat.ME↗

arXiv2023

A generalized hypothesis test for community structure in networks

Researchers theorize that many real-world networks exhibit community structure where within-community edges are more likely than between-community edges. While numerous methods exist to cluster nodes into different communities, less work has addressed this question: given some network, does it exhibit statistically meaningful community structure? We answer this question in a principled manner by framing it as a statistical hypothesis test in terms of a general and model-agnostic community structure parameter. Leveraging this parameter, we propose a simple and interpretable test statistic used to formulate two separate hypothesis testing frameworks. The first is an asymptotic test against a baseline value of the parameter while the second tests against a baseline model using bootstrap-based thresholds. We prove theoretical properties of these tests and demonstrate how the proposed method yields rich insights into real-world data sets.

cs.SI↗

arXiv2023

Statistical Network Analysis: Past, Present, and Future

This article provides a brief overview of statistical network analysis, a rapidly evolving field of statistics, which encompasses statistical models, algorithms, and inferential methods for analyzing data in the form of networks. Particular emphasis is given to connecting the historical developments in network science to today's statistical network analysis, and outlining important new areas for future research. This invited article is intended as a book chapter for the volume "Frontiers of Statistics and Data Science" edited by Subhashis Ghoshal and Anindya Roy for the International Indian Statistical Association Series on Statistics and Data Science, published by Springer. This review article covers the material from the short course titled "Statistical Network Analysis: Past, Present, and Future" taught by the author at the Annual Conference of the International Indian Statistical Association, June 6-10, 2023, at Golden, Colorado.

stat.ME↗

arXiv2023

Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

The rise of internet-based services and products in the late 1990's brought about an unprecedented opportunity for online businesses to engage in large scale data-driven decision making. Over the past two decades, organizations such as Airbnb, Alibaba, Amazon, Baidu, Booking, Alphabet's Google, LinkedIn, Lyft, Meta's Facebook, Microsoft, Netflix, Twitter, Uber, and Yandex have invested tremendous resources in online controlled experiments (OCEs) to assess the impact of innovation on their customers and businesses. Running OCEs at scale has presented a host of challenges requiring solutions from many domains. In this paper we review challenges that require new statistical methodologies to address them. In particular, we discuss the practice and culture of online experimentation, as well as its statistics literature, placing the current methodologies within their relevant statistical lineages and providing illustrative examples of OCE applications. Our goal is to raise academic statisticians' awareness of these new research opportunities to increase collaboration between academia and the online industry.

stat.AP↗

arXiv2023

Scalable Resampling in Massive Generalized Linear Models via Subsampled Residual Bootstrap

Residual bootstrap is a classical method for statistical inference in regression settings. With massive data sets becoming increasingly common, there is a demand for computationally efficient alternatives to residual bootstrap. We propose a simple and versatile scalable algorithm called subsampled residual bootstrap (SRB) for generalized linear models (GLMs), a large class of regression models that includes the classical linear regression model as well as other widely used models such as logistic, Poisson and probit regression. We prove consistency and distributional results that establish that the SRB has the same theoretical guarantees under the GLM framework as the classical residual bootstrap, while being computationally much faster. We demonstrate the empirical performance of SRB via simulation studies and a real data analysis of the Forest Covertype data from the UCI Machine Learning Repository.

stat.ME↗

arXiv2023

Core-periphery structure in networks: a statistical exposition

Many real-world networks are theorized to have core-periphery structure consisting of a densely-connected core and a loosely-connected periphery. While this phenomenon has been extensively studied in a range of scientific disciplines, it has not received sufficient attention in the statistics community. In this expository article, our goal is to raise awareness about this topic and encourage statisticians to address the many open inference problems in this area. To this end, we first summarize the current research landscape by reviewing the metrics and models that have been used for quantitative studies on core-periphery structure. Next, we formulate and explore various inferential problems in this context, such as estimation, hypothesis testing, and Bayesian inference, and discuss related computational techniques. We also outline the multidisciplinary scientific impact of core-periphery structure in a number of real-world networks. Throughout the article, we provide our own interpretation of the literature from a statistical perspective, with the goal of prioritizing open problems where contribution from the statistics community will be most effective and important.

stat.ME↗

arXiv2023

Word Embeddings as Statistical Estimators

Word embeddings are a fundamental tool in natural language processing. Currently, word embedding methods are evaluated on the basis of empirical performance on benchmark data sets, and there is a lack of rigorous understanding of their theoretical properties. This paper studies word embeddings from a statistical theoretical perspective, which is essential for formal inference and uncertainty quantification. We propose a copula-based statistical model for text data and show that under this model, the now-classical Word2Vec method can be interpreted as a statistical estimation method for estimating the theoretical pointwise mutual information (PMI). Next, by building on the work of Levy and Goldberg (2014), we develop a missing value-based estimator as a statistically tractable and interpretable alternative to the Word2Vec approach. The estimation error of this estimator is comparable to Word2Vec and improves upon the truncation-based method proposed by Levy and Goldberg (2014). The proposed estimator also performs comparably to Word2Vec in a benchmark sentiment analysis task on the IMDb Movie Reviews data set.

stat.ME↗

arXiv2021

Social Sensors in Epidemiological Networks via Graph Eigenvectors

In this paper, we consider epidemiological networks which are used for modeling the transmission of contagious diseases through a population. Specifically, we study the so-called social sensors problem: given an epidemiological network, can we find a small set of nodes such that by monitoring disease transmission on these nodes, we can get ahead of the overall epidemic in the full population? In spite of its societal relevance, there has not been much statistical work on this problem, and we aim to provide an exposition that will hopefully stimulate interest in the research community. Furthermore, by leveraging classical results in spectral graph theory, we propose a novel method for finding social sensors, which achieves substantial improvement over existing methods in both synthetic and real-world epidemiological networks.

cs.SI↗

arXiv2021

Scalable Community Extraction of Text Networks for Automated Grouping in Medical Databases

Networks are ubiquitous in today's world. Community structure is a well-known feature of many empirical networks, and a lot of statistical methods have been developed for community detection. In this paper, we consider the problem of community structure in text networks,which is greatly relevant in medical errors and patient safety databases. We adapt a well-known community extraction method to develop a scalable algorithm for community extraction in large text databases. The application of our method on a real-world patient safety report database demonstrates that the groups generated from community extraction are much more accurate than manual tagging by frontline workers.

cs.SI↗

arXiv2021

Testing for the Network Small-World Property

Researchers have long observed that the ``small-world" property, which combines the concepts of high transitivity or clustering with a low average path length, is ubiquitous for networks obtained from a variety of disciplines, including social sciences, biology, neuroscience, and ecology. However, we find several shortcomings of the currently prevalent definition and detection methods rendering the concept less powerful. First, the widely used \textit{small world coefficient} metric combines high transitivity with a low average path length in a single measure that confounds the two separate aspects. We find that the value of the metric is dominated by transitivity, and in several cases, networks get flagged as ``small world" solely because of their high transitivity. Second, the detection methods lack a formal statistical inference. Third, the comparison is typically performed against simplistic random graph models as the baseline, ignoring well-known network characteristics and risks confounding the small world property with other network properties. We decouple the properties of high transitivity and low average path length as separate events to test for. Then we define the property as a statistical test between a suitable null hypothesis and a superimposed alternative hypothesis. We propose a parametric bootstrap test with several null hypothesis models to allow a wide range of background structures in the network. In addition to the bootstrap tests, we also propose an asymptotic test under the Erd\"{o}s-Ren\'{y}i null model for which we provide theoretical guarantees on the asymptotic level and power. Our theoretical results include asymptotic distributions of clustering coefficient for various asymptotic growth rates on the probability of an edge. Applying the proposed methods to a large number of network datasets, we uncover new insights about their small-world property.

stat.ME↗