arXiv subjects

Soumendu Sundar Mukherjee

Publications and source records attributed to Soumendu Sundar Mukherjee.

At least 37 records · Page 2Linked to original sources

arXiv2023

A dynamic mean-field statistical model of academic collaboration

There is empirical evidence that collaboration in academia has increased significantly during the past few decades, perhaps due to the breathtaking advancements in communication and technology during this period. Multi-author articles have become more frequent than single-author ones. Interdisciplinary collaboration is also on the rise. Although there have been several studies on the dynamical aspects of collaboration networks, systematic statistical models which theoretically explain various empirically observed features of such networks have been lacking. In this work, we propose a dynamic mean-field model and an associated estimation framework for academic collaboration networks. We primarily focus on how the degree of collaboration of a typical author, rather than the local structure of her collaboration network, changes over time. We consider several popular indices of collaboration from the literature and study their dynamics under the proposed model. In particular, we obtain exact formulae for the expectations and temporal rates of change of these indices. Through extensive simulation experiments, we demonstrate that the proposed model has enough flexibility to capture various phenomena characteristic of real-world collaboration networks. Using metadata on papers from the arXiv repository, we empirically study the mean-field collaboration dynamics in disciplines such as Computer Science, Mathematics and Physics.

stat.ME↗

arXiv2023

On a generalisation of the coupon collector problem

We consider a generalisation of the classical coupon collector problem. We define a super-coupon to be any $s$-subset of a universe of $n$ coupons. In each round, a random $r$-subset from the universe is drawn and all its $s$-subsets are marked as collected. We show that the time to collect all super-coupons is $\binom{r}{s}^{-1}\binom{n}{s} \log \binom{n}{s}(1 + o(1))$ on average and has a Gumbel limit after a suitable normalisation. In a similar vein, we show that for any $α\in (0, 1)$, the expected time to collect $(1 - α)$ proportion of all super-coupons is $\binom{r}{s}^{-1}\binom{n}{s} \log \big(\frac{1}α\big)(1 + o(1))$. The $r = s$ case of this model is equivalent to the classical coupon collector model. We also consider a temporally dependent model where the $r$-subsets are drawn according to the following Markovian dynamics: the $r$-subset at round $k + 1$ is formed by replacing a random coupon from the $r$-subset drawn at round $k$ with another random coupon from outside this $r$-subset. We link the time it takes to collect all super-coupons in the $r = s$ case of this model to the cover time of random walk on a certain finite regular graph and conjecture that in general, it takes $\frac{r}{s} \binom{r}{s}^{-1}\binom{n}{s}\log\binom{n}{s}(1 + o(1))$ time on average to collect all super-coupons.

math.PR↗

arXiv2023

Learning Networks from Gaussian Graphical Models and Gaussian Free Fields

We investigate the problem of estimating the structure of a weighted network from repeated measurements of a Gaussian Graphical Model (GGM) on the network. In this vein, we consider GGMs whose covariance structures align with the geometry of the weighted network on which they are based. Such GGMs have been of longstanding interest in statistical physics, and are referred to as the Gaussian Free Field (GFF). In recent years, they have attracted considerable interest in the machine learning and theoretical computer science. In this work, we propose a novel estimator for the weighted network (equivalently, its Laplacian) from repeated measurements of a GFF on the network, based on the Fourier analytic properties of the Gaussian distribution. In this pursuit, our approach exploits complex-valued statistics constructed from observed data, that are of interest on their own right. We demonstrate the effectiveness of our estimator with concrete recovery guarantees and bounds on the required sample complexity. In particular, we show that the proposed statistic achieves the parametric rate of estimation for fixed network size. In the setting of networks growing with sample size, our results show that for Erdos-Renyi random graphs $G(d,p)$ above the connectivity threshold, we demonstrate that network recovery takes place with high probability as soon as the sample size $n$ satisfies $n \gg d^4 \log d \cdot p^{-2}$.

math.ST↗

arXiv2023

Wasserstein Projection Pursuit of Non-Gaussian Signals

We consider the general dimensionality reduction problem of locating in a high-dimensional data cloud, a $k$-dimensional non-Gaussian subspace of interesting features. We use a projection pursuit approach -- we search for mutually orthogonal unit directions which maximise the 2-Wasserstein distance of the empirical distribution of data-projections along these directions from a standard Gaussian. Under a generative model, where there is a underlying (unknown) low-dimensional non-Gaussian subspace, we prove rigorous statistical guarantees on the accuracy of approximating this unknown subspace by the directions found by our projection pursuit approach. Our results operate in the regime where the data dimensionality is comparable to the sample size, and thus supplement the recent literature on the non-feasibility of locating interesting directions via projection pursuit in the complementary regime where the data dimensionality is much larger than the sample size.

cs.LG↗

arXiv2022

Morphological Network: How Far Can We Go with Morphological Neurons?

Morphological neurons, that is morphological operators such as dilation and erosion with learnable structuring elements, have intrigued researchers for quite some time because of the power these operators bring to the table despite their simplicity. These operators are known to be powerful nonlinear tools, but for a given problem coming up with a sequence of operations and their structuring element is a non-trivial task. So, the existing works have mainly focused on this part of the problem without delving deep into their applicability as generic operators. A few works have tried to utilize morphological neurons as a part of classification (and regression) networks when the input is a feature vector. However, these methods mainly focus on a specific problem, without going into generic theoretical analysis. In this work, we have theoretically analyzed morphological neurons and have shown that these are far more powerful than previously anticipated. Our proposed morphological block, containing dilation and erosion followed by their linear combination, represents a sum of hinge functions. Existing works show that hinge functions perform quite well in classification and regression problems. Two morphological blocks can even approximate any continuous function. However, to facilitate the theoretical analysis that we have done in this paper, we have restricted ourselves to the 1D version of the operators, where the structuring element operates on the whole input. Experimental evaluations also indicate the effectiveness of networks built with morphological neurons, over similarly structured neural networks.

cs.LG↗

arXiv2022

On $*$-Convergence of Schur-Hadamard Products of Independent Nonsymmetric Random Matrices

Let $\{x_α\}_{α\in \mathbb{Z}}$ and $\{y_α\}_{α\in \mathbb{Z}}$ be two independent collections of zero mean, unit variance random variables with uniformly bounded moments of all orders. Consider a nonsymmetric Toeplitz matrix $X_n = ((x_{i - j}))_{1 \le i, j \le n}$ and a Hankel matrix $Y_n = ((y_{i + j}))_{1 \le i, j \le n}$, and let $M_n = X_n \odot Y_n$ be their elementwise/Schur-Hadamard product. In this article, we show that almost surely, $n^{-1/2}M_n$, as an element of the $*$-probability space $(\mathcal{M}_n(\mathbb{C}), \frac{1}{n}\mathrm{tr})$, converges in $*$-distribution to a circular variable. With i.i.d. Rademacher entries, this construction gives a matrix model for circular variables with only $O(n)$ bits of randomness. We also consider a dependent setup where $\{x_α\}$ and $\{y_β\}$ are independent strongly multiplicative systems (à la Gaposhkin [7]) satisfying an additional \emph{admissibility} condition, and have uniformly bounded moments of all orders -- a nontrivial example of such a system being $\{\sqrt{2}\sin(2^n πU)\}_{n \in \mathbb{Z}_+}$, where $U \sim \mathrm{Uniform}(0, 1)$. In this case, we show in-expectation and in-probability convergence of the $*$-moments of $n^{-1/2}M_n$ to those of a circular variable. Finally, we generalise our results to Schur-Hadamard products of structured random matrices of the form $X_n = ((x_{L_X(i, j)}))_{1 \le i, j \le n}$ and $Y_n = ((y_{L_Y(i, j)}))_{1 \le i, j \le n}$, under certain assumptions on the \emph{link-functions} $L_X$ and $L_Y$, most notably the injectivity of the map $(i, j) \mapsto (L_X(i, j), L_Y(i, j))$. Based on numerical evidence, we conjecture that the circular law $μ_{\mathrm{circ}}$, i.e. the uniform measure on the unit disk of $\mathbb{C}$, which is also the Brown measure of a circular variable, is in fact the limiting spectral measure of $n^{-1/2}M_n$.

math.PR↗

arXiv2022

Some characterization results on classical and free Poisson thinning

Poisson thinning is an elementary result in probability, which is of great importance in the theory of Poisson point processes. In this article, we record a couple of characterization results on Poisson thinning. We also consider several free probability analogues of Poisson thinning, which we collectively dub as \emph{free Poisson thinning}, and prove characterization results for them, similar to the classical case. One of these free Poisson thinning procedures arises naturally as a high-dimensional asymptotic analogue of Cochran's theorem from multivariate statistics on the "Wishart-ness" of quadratic functions of Gaussian random matrices. We note the implications of our characterization results in the context of Cochran's theorem. We also prove a free probability analogue of Craig's theorem, another well-known result in multivariate statistics on the independence of quadratic functions of Gaussian random matrices.

math.PR↗

arXiv2022

Concentration inequalities for correlated network-valued processes with applications to community estimation and changepoint analysis

Network-valued time series are currently a common form of network data. However, the study of the aggregate behavior of network sequences generated from network-valued stochastic processes is relatively rare. Most of the existing research focuses on the simple setup where the networks are independent (or conditionally independent) across time, and all edges are updated synchronously at each time step. In this paper, we study the concentration properties of the aggregated adjacency matrix and the corresponding Laplacian matrix associated with network sequences generated from lazy network-valued stochastic processes, where edges update asynchronously, and each edge follows a lazy stochastic process for its updates independent of the other edges. We demonstrate the usefulness of these concentration results in proving consistency of standard estimators in community estimation and changepoint estimation problems. We also conduct a simulation study to demonstrate the effect of the laziness parameter, which controls the extent of temporal correlation, on the accuracy of community and changepoint estimation.

math.ST↗

arXiv2022

Learning with latent group sparsity via heat flow dynamics on networks

Group or cluster structure on explanatory variables in machine learning problems is a very general phenomenon, which has attracted broad interest from practitioners and theoreticians alike. In this work we contribute an approach to learning under such group structure, that does not require prior information on the group identities. Our paradigm is motivated by the Laplacian geometry of an underlying network with a related community structure, and proceeds by directly incorporating this into a penalty that is effectively computed via a heat flow-based local network dynamics. In fact, we demonstrate a procedure to construct such a network based on the available data. Notably, we dispense with computationally intensive pre-processing involving clustering of variables, spectral or otherwise. Our technique is underpinned by rigorous theorems that guarantee its effective performance and provide bounds on its sample complexity. In particular, in a wide range of settings, it provably suffices to run the heat flow dynamics for time that is only logarithmic in the problem dimensions. We explore in detail the interfaces of our approach with key statistical physics models in network science, such as the Gaussian Free Field and the Stochastic Block Model. We validate our approach by successful applications to real-world data from a wide array of application domains, including computer science, genetics, climatology and economics. Our work raises the possibility of applying similar diffusion-based techniques to classical learning tasks, exploiting the interplay between geometric, dynamical and stochastic structures underlying the data.

stat.ME↗

arXiv2021

Changepoint Analysis of Topic Proportions in Temporal Text Data

Changepoint analysis deals with unsupervised detection and/or estimation of time-points in time-series data, when the distribution generating the data changes. In this article, we consider \emph{offline} changepoint detection in the context of large scale textual data. We build a specialised temporal topic model with provisions for changepoints in the distribution of topic proportions. As full likelihood based inference in this model is computationally intractable, we develop a computationally tractable approximate inference procedure. More specifically, we use sample splitting to estimate topic polytopes first and then apply a likelihood ratio statistic together with a modified version of the wild binary segmentation algorithm of Fryzlewicz et al. (2014). Our methodology facilitates automated detection of structural changes in large corpora without the need of manual processing by domain experts. As changepoints under our model correspond to changes in topic structure, the estimated changepoints are often highly interpretable as marking the surge or decline in popularity of a fashionable topic. We apply our procedure on two large datasets: (i) a corpus of English literature from the period 1800-1922 (Underwoodet al., 2015); (ii) abstracts from the High Energy Physics arXiv repository (Clementet al., 2019). We obtain some historically well-known changepoints and discover some new ones.

cs.CL↗

arXiv2021

Distribution of Eigenvalues of Matrix Ensembles arising from Wigner and Palindromic Toeplitz Blocks

Random Matrix Theory (RMT) has successfully modeled diverse systems, from energy levels of heavy nuclei to zeros of $L$-functions; this correspondence has allowed RMT to successfully predict many number theoretic behaviors. However there are some operations which to date have no RMT analogue. Our motivation is to find an RMT analogue of Rankin-Selberg convolution, which constructs a new $L$-functions from an input pair. We report one such attempt; while it does not appear to model convolution, it does create new ensembles with properties hybridizing those of its constituents. For definiteness we concentrate on the ensemble of palindromic real symmetric Toeplitz (PST) matrices and the ensemble of real symmetric matrices, whose limiting spectral measures are the Gaussian and semi-circular distributions, respectively; these were chosen as they are the two extreme cases in terms of moment calculations. For a PST matrix $A$ and a real symmetric matrix $B$, we construct an ensemble of random real symmetric block matrices whose first row is $\lbrace A, B \rbrace$ and whose second row is $\lbrace B, A \rbrace$. By Markov's Method of Moments and the use of free probability, we show this ensemble converges weakly and almost surely to a new, universal distribution with a hybrid of Gaussian and semi-circular behaviors. We extend this construction by considering an iterated concatenation of matrices from an arbitrary pair of random real symmetric sub-ensembles with different limiting spectral measures. We prove that finite iterations converge to new, universal distributions with hybrid behavior, and that infinite iterations converge to the limiting spectral measure of the dominant component matrix.

math.PR↗

arXiv2020

High dimensional PCA: a new model selection criterion

Given a random sample from a multivariate population, estimating the number of large eigenvalues of the population covariance matrix is an important problem in Statistics with wide applications in many areas. In the context of Principal Component Analysis (PCA), the linear combinations of the original variables having the largest amounts of variation are determined by this number. In this paper, we study the high dimensional asymptotic regime where the number of variables grows at the same rate as the number of observations, and use the spiked covariance model proposed in Johnstone (2001), under which the problem reduces to model selection. Our focus is on the Akaike Information Criterion (AIC) which is known to be strongly consistent from the work of Bai et al. (2018). However, Bai et al. (2018) requires a certain "gap condition" ensuring the dominant eigenvalues to be above a threshold strictly larger than the BBP threshold (Baik et al. (2005), both quantities depending on the limiting ratio of the number of variables and observations. It is well-known that, below the BBP threshold, a spiked covariance structure becomes indistinguishable from one with no spikes. Thus the strong consistency of AIC requires some extra signal strength. In this paper, we investigate whether consistency continues to hold even if the "gap" is made smaller. We show that strong consistency under arbitrarily small gap is achievable if we alter the penalty term of AIC suitably depending on the target gap. Furthermore, another intuitive alteration of the penalty can indeed make the gap exactly zero, although we can only achieve weak consistency in this case. We compare the two newly-proposed estimators with other existing estimators in the literature via extensive simulation studies, and show, by suitably calibrating our proposals, that a significant improvement in terms of mean-squared error is achievable.

math.ST↗

arXiv2020

Consistent detection and optimal localization of all detectable change points in piecewise stationary arbitrarily sparse network-sequences

We consider the offline change point detection and localization problem in the context of piecewise stationary networks, where the observable is a finite sequence of networks. We develop algorithms involving some suitably modified CUSUM statistics based on adaptively trimmed adjacency matrices of the observed networks for both detection and localization of single or multiple change points present in the input data. We provide rigorous theoretical analysis and finite sample estimates evaluating the performance of the proposed methods when the input (finite sequence of networks) is generated from an inhomogeneous random graph model, where the change points are characterized by the change in the mean adjacency matrix. We show that the proposed algorithms can detect (resp. localize) all change points, where the change in the expected adjacency matrix is above the minimax detectability (resp. localizability) threshold, consistently without any a priori assumption about (a) a lower bound for the sparsity of the underlying networks, (b) an upper bound for the number of change points, and (c) a lower bound for the separation between successive change points, provided either the minimum separation between successive pairs of change points or the average degree of the underlying networks goes to infinity arbitrarily slowly. We also prove that the above condition is necessary to have consistency.

stat.ME↗

arXiv2020

Exact Tests for Offline Changepoint Detection in Multichannel Binary and Count Data with Application to Networks

We consider offline detection of a single changepoint in binary and count time-series. We compare exact tests based on the cumulative sum (CUSUM) and the likelihood ratio (LR) statistics, and a new proposal that combines exact two-sample conditional tests with multiplicity correction, against standard asymptotic tests based on the Brownian bridge approximation to the CUSUM statistic. We see empirically that the exact tests are much more powerful in situations where normal approximations driving asymptotic tests are not trustworthy: (i) small sample settings; (ii) sparse parametric settings; (iii) time-series with changepoint near the boundary. We also consider a multichannel version of the problem, where channels can have different changepoints. Controlling the False Discovery Rate (FDR), we simultaneously detect changes in multiple channels. This "local" approach is shown to be more advantageous than multivariate global testing approaches when the number of channels with changepoints is much smaller than the total number of channels. As a natural application, we consider network-valued time-series and use our approach with (a) edges as binary channels and (b) node-degrees or other local subgraph statistics as count channels. The local testing approach is seen to be much more informative than global network changepoint algorithms.

stat.ME↗

arXiv2019

Graphon Estimation from Partially Observed Network Data

We consider estimating the edge-probability matrix of a network generated from a graphon model when the full network is not observed---only some overlapping subgraphs are. We extend the neighbourhood smoothing (NBS) algorithm of Zhang et al. (2017) to this missing-data set-up and show experimentally that, for a wide range of graphons, the extended NBS algorithm achieves significantly smaller error rates than standard graphon estimation algorithms such as vanilla neighbourhood smoothing (NBS), universal singular value thresholding (USVT), blockmodel approximation, matrix completion, etc. We also show that the extended NBS algorithm is much more robust to missing data.

stat.ML↗

arXiv2019

When random initializations help: a study of variational inference for community detection

Variational approximation has been widely used in large-scale Bayesian inference recently, the simplest kind of which involves imposing a mean field assumption to approximate complicated latent structures. Despite the computational scalability of mean field, theoretical studies of its loss function surface and the convergence behavior of iterative updates for optimizing the loss are far from complete. In this paper, we focus on the problem of community detection for a simple two-class Stochastic Blockmodel (SBM) with equal class sizes. Using batch co-ordinate ascent (BCAVI) for updates, we show different convergence behavior with respect to different initializations. When the parameters are known or estimated within a reasonable range and held fixed, we characterize conditions under which an initialization can converge to the ground truth. On the other hand, when the parameters need to be estimated iteratively, a random initialization will converge to an uninformative local optimum.

math.ST↗

arXiv2017

On clustering network-valued data

Community detection, which focuses on clustering nodes or detecting communities in (mostly) a single network, is a problem of considerable practical interest and has received a great deal of attention in the research community. While being able to cluster within a network is important, there are emerging needs to be able to cluster multiple networks. This is largely motivated by the routine collection of network data that are generated from potentially different populations. These networks may or may not have node correspondence. When node correspondence is present, we cluster networks by summarizing a network by its graphon estimate, whereas when node correspondence is not present, we propose a novel solution for clustering such networks by associating a computationally feasible feature vector to each network based on trace of powers of the adjacency matrix. We illustrate our methods using both simulated and real data sets, and theoretical justifications are provided in terms of consistency.

stat.ML↗

arXiv2017

Two provably consistent divide and conquer clustering algorithms for large networks

In this article, we advance divide-and-conquer strategies for solving the community detection problem in networks. We propose two algorithms which perform clustering on a number of small subgraphs and finally patches the results into a single clustering. The main advantage of these algorithms is that they bring down significantly the computational cost of traditional algorithms, including spectral clustering, semi-definite programs, modularity based methods, likelihood based methods etc., without losing on accuracy and even improving accuracy at times. These algorithms are also, by nature, parallelizable. Thus, exploiting the facts that most traditional algorithms are accurate and the corresponding optimization problems are much simpler in small problems, our divide-and-conquer methods provide an omnibus recipe for scaling traditional algorithms up to large networks. We prove consistency of these algorithms under various subgraph selection procedures and perform extensive simulations and real-data analysis to understand the advantages of the divide-and-conquer approach in various settings.

stat.ML↗