arXiv subjects

Shankar Bhamidi

Publications and source records attributed to Shankar Bhamidi.

At least 37 records · Page 2Linked to original sources

arXiv2023

Correlation networks, dynamic factor models and community detection

A dynamic factor model with a mixture distribution of the loadings is introduced and studied for multivariate, possibly high-dimensional time series. The correlation matrix of the model exhibits a block structure, reminiscent of correlation patterns for many real multivariate time series. A standard $k$-means algorithm on the loadings estimated through principal components is used to cluster component time series into communities with accompanying bounds on the misclustering rate. This is one standard method of community detection applied to correlation matrices viewed as weighted networks. This work puts a mixture model, a dynamic factor model and network community detection in one interconnected framework. Performance of the proposed methodology is illustrated on simulated and real data.

stat.ME↗

arXiv2023

Dynamic factor and VARMA models: equivalent representations, dimension reduction and nonlinear matrix equations

A dynamic factor model with factor series following a VAR$(p)$ model is shown to have a VARMA$(p,p)$ model representation. Reduced-rank structures are identified for the VAR and VMA components of the resulting VARMA model. It is also shown how the VMA component parameters can be computed numerically from the original model parameters via the innovations algorithm, and connections of this approach to non-linear matrix equations are made. Some VAR models related to the resulting VARMA model are also discussed.

stat.ME↗

arXiv2022

Fluctuation Bounds for Continuous Time Branching Processes and Evolution of Growing Trees With a Change Point

We consider dynamic random trees constructed using an attachment function $f : \mathbb{N} \to \mathbb{R}_+$ where, at each step of the evolution, a new vertex attaches to an existing vertex $v$ in the current tree with probability proportional to $f$(degree(v)). We explore the effect of a change point in the system; the dynamics are initially driven by a function f until the tree reaches size $τ(n) \in (0,n)$, at which point the attachment function switches to another function, $g$, until the tree reaches size $n$. Two change point time scales are considered, namely the standard model where $τ(n) = γn$, and the quick big bang model where $τ(n) = n^γ$, for some $0 < γ< 1$. In the former case, we obtain deterministic approximations for the evolution of the empirical degree distribution (EDF) in sup-norm and use these to devise a provably consistent non-parametric estimator for the change point $γ$. In the latter case, we show that the effect of pre-change point dynamics asymptotically vanishes in the EDF, although this effect persists in functionals such as the maximal degree. Our proofs rely on embedding the discrete time tree dynamics in an associated (time) inhomogeneous continuous time branching process (CTBP). In the course of proving the above results, we develop novel mathematical techniques to analyze both homogeneous and inhomogeneous CTBPs and obtain rates of convergence for functionals of such processes, which are of independent interest.

math.PR↗

arXiv2022

Global lower mass-bound for critical configuration models in the heavy-tailed regime

We establish the global lower mass-bound property for the largest connected components in the critical window for the configuration model when the degree distribution has an infinite third moment. The scaling limit of the critical percolation clusters, viewed as measured metric spaces, was established in [7] with respect to the Gromov-weak topology. Our result extends those scaling limit results to the stronger Gromov-Hausdorff-Prokhorov topology under slightly stronger assumptions on the degree distribution. This implies the distributional convergence of global functionals such as the diameter of the largest critical components. Further, our result gives a sufficient condition for compactness of the random metric spaces that arise as scaling limits of critical clusters in the heavy-tailed regime.

math.PR↗

arXiv2021

Community detection using low-dimensional network embedding algorithms

With the increasing relevance of large networks in important areas such as the study of contact networks for spread of disease, or social networks for their impact on geopolitics, it has become necessary to study machine learning tools that are scalable to very large networks, often containing millions of nodes. One major class of such scalable algorithms is known as network representation learning or network embedding. These algorithms try to learn representations of network functionals (e.g.~nodes) by first running multiple random walks and then using the number of co-occurrences of each pair of nodes in observed random walk segments to obtain a low-dimensional representation of nodes on some Euclidean space. The aim of this paper is to rigorously understand the performance of two major algorithms, DeepWalk and node2vec, in recovering communities for canonical network models with ground truth communities. Depending on the sparsity of the graph, we find the length of the random walk segments required such that the corresponding observed co-occurrence window is able to perform almost exact recovery of the underlying community assignments. We prove that, given some fixed co-occurrence window, node2vec using random walks with a low non-backtracking probability can succeed for much sparser networks compared to DeepWalk using simple random walks. Moreover, if the sparsity parameter is low, we provide evidence that these algorithms might not succeed in almost exact recovery. The analysis requires developing general tools for path counting on random networks having an underlying low-rank structure, which are of independent interest.

cs.SI↗

arXiv2021

Root finding algorithms and persistence of Jordan centrality in growing random trees

We consider models of growing random trees $\{\mathcal{T}_f(n):n\geq 1\}$ with model dynamics driven by an attachment function $f:\mathbb{Z}_+\to \mathbb{R}_+$. At each stage a new vertex enters the system and connects to a vertex $v$ in the current tree with probability proportional to $f(\text{degree}(v))$. The main goal of this study is to understand the performance of root finding algorithms. A large body of work (e.g. the work of Bubeck, Devroye and Lugosi or Jog and Loh) has emerged in the last few years in using techniques based on the Jordan centrality measure and its variants to develop root finding algorithms. Given an unlabelled unrooted tree, one computes the Jordan centrality for each vertex in the tree and for a fixed budget $K$ outputs the optimal $K$ vertices (as measured by Jordan centrality). Under general conditions on the attachment function $f$, we derive necessary and sufficient bounds on the budget $K(ε)$ in order to recover the root with probability at least $1-ε$. For canonical examples such as linear preferential attachment and uniform attachment, these general results give matching upper and lower bounds for the budget. We also prove persistence of the optimal $K$ Jordan centers for any $K$, i.e. the existence of an almost surely finite random time $n^*$ such that for $n \geq n^*$ the identity of the $K$-optimal Jordan centers in $\{\mathcal{T}_f(n):n\geq n^*\}$ does not change, thus describing robustness properties of this measure. Key technical ingredients in the proofs of independent interest include sufficient conditions for the existence of exponential moments for limits of (appropriately normalized) continuous time branching processes within which the models $\{\mathcal{T}_f(n):n\geq n^*\}$ can be embedded, as well as rates of convergence results to these limits.

math.PR↗

arXiv2021

Multiscale genesis of a tiny giant for percolation on scale-free random graphs

We study the critical behavior for percolation on inhomogeneous random networks on $n$ vertices, where the weights of the vertices follow a power-law distribution with exponent $τ\in (2,3)$. Such networks, often referred to as scale-free networks, exhibit critical behavior when the percolation probability tends to zero at an appropriate rate, as $n\to\infty$. We identify the critical window for a host of scale-free random graph models such as the Norros-Reittu model, Chung-Lu model and generalized random graphs. Surprisingly, there exists a finite time inside the critical window, after which, we see a sudden emergence of a tiny giant component. This is a novel behavior which is in contrast with the critical behavior in other known universality classes with $τ\in (3,4)$ and $τ>4$. Precisely, for edge-retention probabilities $π_n = λn^{-(3-τ)/2}$, there is an explicitly computable $λ_c>0$ such that the critical window is of the form $λ\in (0,λ_c),$ where the largest clusters have size of order $n^β$ with $β=(τ^2-4τ+5)/[2(τ-1)]\in[\sqrt{2}-1, \tfrac{1}{2})$ and have non-degenerate scaling limits, while in the supercritical regime $λ> λ_c$, a unique `tiny giant' component of size $\sqrt{n}$ emerges. For $λ\in (0,λ_c),$ the scaling limit of the maximum component sizes can be described in terms of components of a one-dimensional inhomogeneous percolation model on $\mathbb{Z}_+$ studied in a seminal work by Durrett and Kesten. For $λ>λ_c$, we prove that the sudden emergence of the tiny giant is caused by a phase transition inside a smaller core of vertices of weight $Ω(\sqrt{n})$.

math.PR↗

arXiv2021

Scaling limits of random graph models at criticality: Universality and the basin of attraction of the Erdős-Rényi random graph

A wide array of random graph models have been postulated to understand properties of observed networks. Typically these models have a parameter $t$ and a critical time $t_c$ when a giant component emerges. It is conjectured that for a large class of models, the nature of this emergence is similar to that of the Erdős-Rényi random graph, in the sense that (a) the sizes of the maximal components in the critical regime scale like $n^{2/3}$, and (b) the structure of the maximal components at criticality (rescaled by $n^{-1/3}$) converges to random fractals. To date, (a) has been proven for a number of models using different techniques. This paper develops a general program for proving (b) that requires three ingredients: (i) in the critical scaling window, components merge approximately like the multiplicative coalescent, (ii) scaling exponents of susceptibility functions are the same as that of the Erdős-Rényi random graph, and (iii) macroscopic averaging of distances between vertices in the barely subcritical regime. We show that these apply to two fundamental random graph models: the configuration model and inhomogeneous random graphs with a finite ground space. For these models, we also obtain new results for component sizes at criticality and structural properties in the barely subcritical regime.

math.PR↗

arXiv2021

Persistence of hubs in growing random networks

We consider models of evolving networks $\{\mathcal{G}_n:n\geq 0\}$ modulated by two parameters: an attachment function $f:\mathbb{N}_0\to\mathbb{R}_+$ and a (possibly random) attachment sequence $\{m_i:i\geq 1\}$. Starting with a single vertex, at each discrete step $i\geq 1$ a new vertex $v_i$ enters the system with $m_i\geq 1$ edges which it sequentially connects to a pre-existing vertex $v\in \mathcal{G}_{i-1}$ with probability proportional to $f(\operatorname{degree}(v))$. We consider the problem of emergence of persistent hubs: existence of a finite (a.s.) time $n^*$ such that for all $n\geq n^*$ the identity of the maximal degree vertex (or in general the $K$ largest degree vertices for $K\geq 1$) does not change. We obtain general conditions on $f$ and $\{m_i:i\geq 1\}$ under which a persistent hub emerges, and also those under which a persistent hub fails to emerge. In the case of lack of persistence, for the specific case of trees ($m_i\equiv 1$ for all $i$), we derive asymptotics for the maximal degree and the index of the maximal degree vertex (time at which the vertex with current maximal degree entered the system) to understand the movement of the maximal degree vertex as the network evolves. A key role in the analysis is played by an inverse rate weighted martingale constructed from a continuous time embedding of the discrete time model. Asymptotics for this martingale, including concentration inequalities and moderate deviations, play a major role in the analysis of the model.

math.PR↗

arXiv2020

Community modulated recursive trees and population dependent branching processes

We consider random recursive trees that are grown via community modulated schemes that involve random attachment or degree based attachment. The aim of this paper is to derive general techniques based on continuous time embedding to study such models. The associated continuous time embeddings are not branching processes: individual reproductive rates at each time depend on the composition of the entire population at that time. Using stochastic analytic techniques we show that various key macroscopic statistics of the continuous time embedding stabilize, allowing asymptotics for a host of functionals of the original models to be derived.

math.PR↗

arXiv2020

Rare event asymptotics for exploration processes for random graphs

Much work in the study of large deviations for random graph models is focused on the dense regime where the theory of graphons has emerged as a principal tool. These tools do not give a good approach to large deviation problems for random graph models in the sparse regime. The aim of this paper is to study an approach for large deviation problems in this regime by establishing Large Deviation Principles (LDP) on suitable path spaces for certain exploration processes of the associated random graph sequence. Our work focuses on the study of one particular class of random graph models, namely the configuration model; however the general approach of using exploration processes for studying large deviation properties of sparse random graph models has broader applicability. The goal is to study asymptotics of probabilities of non-typical behavior in the large network limit. The first key step for this is to establish a LDP for an exploration process associated with the configuration model. A suitable exploration process here turns out to be an infinite dimensional Markov process with transition probability rates that diminish to zero in certain parts of the state space. Large deviation properties of such Markovian models is challenging due to poor regularity behavior of the associated local rate functions. Next, using the rate function in the LDP for the exploration process we formulate a calculus of variations problem associated with the asymptotics of component degree distributions. The second key ingredient in our study is a careful analysis of the infinite dimensional Euler-Lagrange equations associated with this calculus of variations problem. Exact solutions are identified which then provide explicit formulas for decay rates of probabilities of non-typical component degree distributions and related quantities. Please see the paper for the complete abstract.

math.PR↗

arXiv2020

Near Equilibrium Fluctuations for Supermarket Models with Growing Choices

We consider the supermarket model in the usual Markovian setting where jobs arrive at rate $n λ_n$ for some $λ_n > 0$, with $n$ parallel servers each processing jobs in its queue at rate 1. An arriving job joins the shortest among $d_n \le n$ randomly selected service queues. We show that when $d_n \to \infty$ and $λ_n \to λ\in (0, \infty)$, under natural conditions on the initial queues, the state occupancy process converges in probability, in a suitable path space, to the unique solution of an infinite system of constrained ordinary differential equations parametrized by $λ$. Our main interest is in the study of fluctuations of the state process about its near equilibrium state in the critical regime, namely when $λ_n \to 1$. Previous papers have considered the regime $\frac{d_n}{\sqrt{n}\log n} \to \infty$ while the objective of the current work is to develop diffusion approximations for the state occupancy process that allow for all possible rates of growth of $d_n$. In particular we consider the three canonical regimes (a) ${d_n}/{\sqrt{n}} \to 0$; (b) ${d_n}/{\sqrt{n}} \to c\in (0,\infty)$ and, (c) ${d_n}/{\sqrt{n}} \to \infty$. In all three regimes we show, by establishing suitable functional limit theorems, that (under conditions on $λ_n$) fluctuations of the state process about its near equilibrium are of order $n^{-1/2}$ and are governed asymptotically by a one dimensional Brownian motion. The forms of the limit processes in the three regimes are quite different; in the first case we get a linear diffusion; in the second case we get a diffusion with an exponential drift; and in the third case we obtain a reflected diffusion in a half space. In the special case ${d_n}/({\sqrt{n}\log n}) \to \infty$ our work gives alternative proofs for the universality results established by Mukherjee et al in 2018.

math.PR↗

arXiv2020

A probabilistic approach to the leader problem in random graphs

We study the fixation time of the identity of the leader, i.e., the most massive component, in the general setting of Aldous's multiplicative coalescent [4, 5], which in an asymptotic sense describes the evolution of the component sizes of a wide array of near-critical coalescent processes, including the classical Erdős-Rényi process. We show tightness of the fixation time in the "Brownian" regime, explicitly determining the median value of the fixation time to within an optimal $O(1)$ window. This generalizes Łuczak's result [31] for the Erdős-Rényi random graph using completely different techniques. In the heavy-tailed case, in which the limit of the component sizes can be encoded using a thinned pure-jump Lévy process, we prove that only one-sided tightness holds. This shows a genuine difference in the possible behavior in the two regimes. The solution to the leader problem in the setting of the Erdős-Rényi random graph played an important role in the study of the scaling limit of the minimal spanning tree on the complete graph [2]. We believe that analogous results, such as those proved herein, will be useful in establishing universality of the intrinsic geometry of the minimal spanning tree across a large class of models.

math.PR↗

arXiv2020

Universality for critical heavy-tailed network models: Metric structure of maximal components

We study limits of the largest connected components (viewed as metric spaces) obtained by critical percolation on uniformly chosen graphs and configuration models with heavy-tailed degrees. For rank-one inhomogeneous random graphs, such results were derived by Bhamidi, van der Hofstad, Sen [Probab. Theory Relat. Fields 2018]. We develop general principles under which the identical scaling limits as the rank-one case can be obtained. Of independent interest, we derive refined asymptotics for various susceptibility functions and the maximal diameter in the barely subcritical regime.

math.PR↗

arXiv2020

Intertemporal Community Detection in Human Mobility Networks

We introduce a community detection method that finds clusters in network time-series by introducing an algorithm that finds significantly interconnected nodes across time. These connections are either increasing, decreasing, or constant over time. Significance of nodal connectivity within a set is judged using the Weighted Configuration Null Model at each time-point, then a novel significance-testing scheme is used to assess connectivity at all time points and the direction of its time-trend. We apply this method to bikeshare networks in New York City and Chicago and taxicab pickups and dropoffs in New York to find and illustrate patterns in human mobility in urban zones. Results show stark geographical patterns in clusters that are growing and declining in relative usage across time and potentially elucidate latent economic or demographic trends.

physics.soc-ph↗

arXiv2020

Demarcating Geographic Regions using Community Detection in Commuting Networks with Significant Self-Loops

We develop a method to identify statistically significant communities in a weighted network with a high proportion of self-looping weights. We use this method to find overlapping agglomerations of U.S. counties by representing inter-county commuting as a weighted network. We identify three types of communities; non-nodal, nodal and monads, which correspond to different types of regions. The results suggest that traditional regional delineations that rely on ad hoc thresholds do not account for important and pervasive connections that extend far beyond expected metropolitan boundaries or megaregions.

physics.soc-ph↗

arXiv2020

Survival and extinction of epidemics on random graphs with general degrees

In this paper, we establish the necessary and sufficient criterion for the contact process on Galton-Watson trees (resp. random graphs) to exhibit the phase of extinction (resp. short survival). We prove that the survival threshold $λ_1$ for a Galton-Watson tree is strictly positive if and only if its offspring distribution $ξ$ has an exponential tail, i.e., $\mathbb{E} e^{cξ}<\infty$ for some $c>0$, settling a conjecture by Huang and Durrett [12]. On the random graph with degree distribution $μ$, we show that if $μ$ has an exponential tail, then for small enough $λ$ the contact process with the all-infected initial condition survives for $n^{1+o(1)}$-time w.h.p. (short survival), while for large enough $λ$ it runs over $e^{Θ(n)}$-time w.h.p. (long survival). When $μ$ is subexponential, we prove that the contact process w.h.p. displays long survival for any fixed $λ>0$.

math.PR↗

arXiv2019

Large Deviation Principle for the Exploration Process of the Configuration Model

The configuration model is a sequence of random graphs constructed such that in the large network limit the degree distribution converges to a pre-specified probability distribution. The component structure of such random graphs can be obtained from an infinite dimensional Markov chain referred to as the exploration process. We establish a large deviation principle for the exploration process associated with the configuration model. Proofs rely on a representation of the exploration process as a system of stochastic differential equations driven by Poisson random measures and variational formulas for moments of nonnegative functionals of Poisson random measures. Uniqueness results for certain controlled systems of deterministic equations play a key role in the analysis. Applications of the large deviation results, for studying asymptotic behavior of the degree sequence in large components of the random graphs, are discussed.

math.PR↗