SearcharxivSearch

arXiv subjects

Nelly Litvak

Publications and source records attributed to Nelly Litvak.

At least 19 recordsLinked to original sources

Likelihood-based anomaly detection in preferential attachment networks

Preferential attachment (PA) network is a widely used model for capturing the growth dynamics of real-world networks, in which newly arriving vertices are more likely to connect to existing vertices with higher degrees. In this paper, we consider a setting in which an anomalous vertex appears at some time point and receives edges with an additional attachment advantage governed by a parameter $\beta$, while ordinary vertices continue to follow the PA mechanism with parameter $\delta$. Detecting such anomalies is challenging due to the high variability in degree growth in PA networks and the limited information available when the anomaly arrives late in the network evolution. We propose an iterative parameter estimation procedure together with a likelihood-based detection framework. Simulation results show that the proposed procedure provides accurate estimation of the parameters $\beta$ and $\delta$. Detection performance depends on the time of anomaly occurrence: anomalies arising at midway stages of the network evolution are detected most reliably, whereas very early and late anomalies remain challenging, particularly when the parameter $\beta$ is small.

cs.SI

Power-law hypothesis and (un)fairness of PageRank on undirected multi-type PAMs

The preferential attachment model (PAM) describes the sequential growth of a network based on the "rich-get-richer" principle. Several versions of it have become established for modeling, e.g., citation networks, capturing a power-law degree distribution. Directed versions of the preferential attachment model where the edges are directed from the new to the old vertices have been the subject of extensive research. They have been shown to exhibit remarkable properties such as heavier tails for the limiting graph-normalized PageRank than for the in-degrees. By contrast, for the undirected version, we recently showed that PageRank has similar tails as the degree. In the present paper, we discuss the PageRank asymptotics for a multi-type version of the undirected PAM (here vertices have different colors), complementing previous results of Antunes, Bhamidi, Banerjee and Pipiras on the asymptotics of PageRank on similar directed multi-type or colored PAMs. Our studies are motivated by the aim to go beyond the rigid rule of edge orientation in directed preferential attachment models. As the main result, for the case of a finite set of colors, we show that the power-law hypothesis for PageRank is fulfilled also for the colored undirected PAM, where, by contrast to the directed case, the power-law exponent is color-dependent for some choices of the initial color distribution and the attractiveness function. For the specific case of a two-type model, we discuss implications of our results on fairness in sampling underrepresented nodes from the network.

math.PR

The stochastic block model has the overlap graph property for modularity

The overlap gap property (OGP) is a statement about the geometry of near-optimal solutions. Exhibiting OGP implies failure of a class of local algorithms; and has been observed to coincide with conjectured algorithmic limits in problems with statistical computational gap. We consider the Stochastic Block Model (SBM), where the graph has a planted partition with $k$ equal-size blocks which form the `communities', and where, for parameters $p>q$, vertices within the same community connect with probability $p$, while vertices in different communities connect with probability $q$, independently across pairs of vertices. Modularity--based clustering algorithms have become ubiquitous in applications. This article studies theoretical limits of local algorithms based on the modularity score on the SBM. We establish that modularity exhibits OGP on the SBM. This rules out a class of local algorithms based on modularity for recovery in the SBM, and shows slow mixing time for a related Markov Chain. Theoretically this is one of the few instances where OGP has been established for a `planted' model, as most such analyses to date consider the `null' model. As part of our analysis, we extend a result by Bickel and Chen 2009, who established that with high probability, the modularity optimal partition of SBM is $o(n)$ local moves away from the planted partition, where $n$ is the graph size. We show that, with high probability, any partition with modularity score sufficiently near the optimal value is close to the planted partition.

math.PR

Linking PageRank, Time Reversal, and Policy Evaluation

We establish a connection between policy evaluation in Markov decision processes and PageRank in network analysis. For a fixed policy, we show that the value function of a discounted Markov decision process can be obtained, up to an explicit rescaling, from the PageRank vector of a suitably defined time-reversed Markov chain. In this correspondence, the discount factor plays the role of the teleportation parameter, while rewards induce the restart distribution. Beyond the irreducible case, invoking quasi-stationary distributions and Doob $h$-transforms, we prove a general decomposition theorem showing that policy evaluation for arbitrary finite MDPs reduces to a collection of PageRank problems on the recurrent and transient components of the policy-induced Markov chain. This framework naturally extends to undiscounted MDPs with terminal states and to transition-dependent rewards. We conclude by showing efficiency of our approach on a numerical example of a sticky random walk on large deterministic and random graphs.

math.OC

Planted clique recovery in random geometric graphs

We investigate the problem of identifying planted cliques in random geometric graphs, focusing on two distinct algorithmic approaches: the first based on vertex degrees (VD) and the other on common neighbors (CN). We analyze the performance of these methods under varying regimes of key parameters, namely the average degree of the graph and the size of the planted clique. We demonstrate that exact recovery is achieved with high probability as the graph size increases, in a specific set of parameters. Notably, our results reveal that the CN-algorithm significantly outperforms the VD-algorithm. In particular, in the connectivity regime, tiny planted cliques (even edges) are correctly identified by the CN-algorithm, yielding a significant impact on anomaly detection. Finally, our results are confirmed by a series of numerical experiments, showing that the devised algorithms are effective in practice.

math.PR

The friendship paradox for trees

We analyse the friendship paradox on finite and infinite trees. In particular, we monitor the vertices for which the friendship-bias is positive, neutral and negative, respectively. For an arbitrary finite tree, we show that the number of positive vertices is at least as large as the number of negative vertices, a property we refer to as significance, and derive a lower bound in terms of the branching points in the tree. For an infinite Galton-Watson tree, we compute the densities of the positive and the negative vertices and show that either may dominate the other, depending on the offspring distribution. We also compute the densities of the edges having two given types of vertices at their ends, and give conditions in terms of the offspring distribution under which these types are positively or negatively correlated.

math.PR

Degrees in Preferential Attachment Networks with an Anomaly

We consider a preferential attachment model that incorporates an anomaly. Our goal is to understand the evolution of the network before and after the occurrence of the anomaly by studying the influence of the anomaly on the structural properties of the network. The anomaly is such that after its arrival it attracts newly added edges with fixed probability. We investigate the growth of degrees in the network, finding that the anomaly's degree increases almost linearly. We also provide a heuristic derivation for the exponent of the limiting degree distributions of ordinary vertices, and study the degree growth of the oldest vertex. We show that when the anomaly enters early, the degree distribution is altered significantly, while a late anomaly has minimal impact. Our analysis provides deeper insights into the evolution of preferential attachment networks with an anomalous vertex.

physics.soc-ph

Value of risk-contact data from digital contact monitoring apps in infectious disease modeling

In this paper, we present a simple method to integrate risk-contact data, obtained via digital contact monitoring (DCM) apps, in conventional compartmental transmission models. During the recent COVID-19 pandemic, many such data have been collected for the first time via newly developed DCM apps. However, it is unclear what the added value of these data is, unlike that of traditionally collected data via, e.g., surveys during non-epidemic times. The core idea behind our method is to express the number of infectious individuals as a function of the proportion of contacts that were with infected individuals and use this number as a starting point to initialize the remaining compartments of the model. As an important consequence, using our method, we can estimate key indicators such as the effective reproduction number using only two types of daily aggregated contact information, namely the average number of contacts and the average number of those contacts that were with an infected individual. We apply our method to the recent COVID-19 epidemic in the Netherlands, using self-reported data from the health surveillance app COVID RADAR and proximity-based data from the contact tracing app CoronaMelder. For both data sources, our corresponding estimates of the effective reproduction number agree both in time and magnitude with estimates based on other more detailed data sources such as daily numbers of cases and hospitalizations. This suggests that the use of DCM data in transmission models, regardless of the precise data type and for example via our method, offers a promising alternative for estimating the state of an epidemic, especially when more detailed data are not available.

q-bio.PE

Power-law hypothesis for PageRank on undirected graphs

Based on observations in the web-graph, the power-law hypothesis states that PageRank has a power-law distribution with the same exponent as the in-degree. While this hypothesis has been analytically verified for many random graph models, such as directed configuration models and generalized random graphs, surprisingly it has recently been disproven for the directed preferential attachment model. In this paper, we prove that in undirected networks, the graph-normalized PageRank is always upper bounded by the degree. Furthermore, we prove that the corresponding (asymptotic) lower bound holds true under reasonable assumptions on the local weak limit, but not in general, and we provide a counterexample. Our result shows that PageRank always has a lighter tail than the degree, which contrasts the case of the directed preferential attachment model, where PageRank has a heavier tail instead. We end the paper with a discussion, where we extend our results to directed networks with a bounded ratio of in- and out-degrees, and reflect on our methods by contrasting the undirected and directed preferential attachment model.

math.PR

Learning the mechanisms of network growth

We propose a novel model-selection method for dynamic networks. Our approach involves training a classifier on a large body of synthetic network data. The data is generated by simulating nine state-of-the-art random graph models for dynamic networks, with parameter range chosen to ensure exponential growth of the network size in time. We design a conceptually novel type of dynamic features that count new links received by a group of vertices in a particular time interval. The proposed features are easy to compute, analytically tractable, and interpretable. Our approach achieves a near-perfect classification of synthetic networks, exceeding the state-of-the-art by a large margin. Applying our classification method to real-world citation networks gives credibility to the claims in the literature that models with preferential attachment, fitness and aging fit real-world citation networks best, although sometimes, the predicted model does not involve vertex fitness.

cs.SI

Fairness Rising from the Ranks: HITS and PageRank on Homophilic Networks

In this paper, we investigate the conditions under which link analysis algorithms prevent minority groups from reaching high ranking slots. We find that the most common link-based algorithms using centrality metrics, such as PageRank and HITS, can reproduce and even amplify bias against minority groups in networks. Yet, their behavior differs: one one hand, we empirically show that PageRank mirrors the degree distribution for most of the ranking positions and it can equalize representation of minorities among the top ranked nodes; on the other hand, we find that HITS amplifies pre-existing bias in homophilic networks through a novel theoretical analysis, supported by empirical results. We find the root cause of bias amplification in HITS to be the level of homophily present in the network, modeled through an evolving network model with two communities. We illustrate our theoretical analysis on both synthetic and real datasets and we present directions for future work.

cs.SI

Detecting hyperbolic geometry in networks: why triangles are not enough

In the past decade, geometric network models have received vast attention in the literature. These models formalize the natural idea that similar vertices are likely to connect. Because of that, these models are able to adequately capture many common structural properties of real-world networks, such as self-invariance and high clustering. Indeed, many real-world networks can be accurately modeled by positioning vertices of a network graph in hyperbolic spaces. Nevertheless, if one observes only the network connections, the presence of geometry is not always evident. Currently, triangle counts and clustering coefficients are the standard statistics to signal the presence of geometry. In this paper we show that triangle counts or clustering coefficients are insufficient because they fail to detect geometry induced by hyperbolic spaces. We therefore introduce a novel triangle-based statistic, which weighs triangles based on their strength of evidence for geometry. We show analytically, as well as on synthetic and real-world data, that this is a powerful statistic to detect hyperbolic geometry in networks.

physics.soc-ph

The Projection Method: a Unified Formalism for Community Detection

We present the class of projection methods for community detection that generalizes many popular community detection methods. In this framework, we represent each clustering (partition) by a vector on a high-dimensional hypersphere. A community detection method is a projection method if it can be described by the following two-step approach: 1) the graph is mapped to a query vector on the hypersphere; and 2) the query vector is projected on the set of clustering vectors. This last projection step is performed by minimizing the distance between the query vector and the clustering vector, over the set of clusterings. We prove that optimizing Markov stability, modularity, the likelihood of planted partition models and correlation clustering fit this framework. A consequence of this equivalence is that algorithms for each of these methods can be modified to perform the projection step in our framework. In addition, we show that these different methods suffer from the same granularity problem: they have parameters that control the granularity of the resulting clustering, but choosing these to obtain clusterings of the desired granularity is nontrivial. We provide a general heuristic to address this granularity problem, which can be applied to any projection method. Finally, we show how, given a generator of graphs with community structure, we can optimize a projection method for this generator in order to obtain a community detection method that performs well on this generator.

cs.SI

Look back, look around: a systematic analysis of effective predictors for new outlinks in focused Web crawling

Small and medium enterprises rely on detailed Web analytics to be informed about their market and competition. Focused crawlers meet this demand by crawling and indexing specific parts of the Web. Critically, a focused crawler must quickly find new pages that have not yet been indexed. Since a new page can be discovered only by following a new outlink, predicting new outlinks is very relevant in practice. In the literature, many feature designs have been proposed for predicting changes in the Web. In this work we provide a structured analysis of this problem, using new outlinks as our running prediction target. Specifically, we unify earlier feature designs in a taxonomic arrangement of features along two dimensions: static versus dynamic features, and features of a page versus features of the network around it. Within this taxonomy, complemented by our new (mainly, dynamic network) features, we identify best predictors for new outlinks. Our main conclusion is that most informative features are the recent history of new outlinks on a page itself, and of its content-related pages. Hence, we propose a new 'look back, look around' (LBLA) model, that uses only these features. With the obtained predictions, we design a number of scoring functions to guide a focused crawler to pages with most new outlinks, and compare their performance. The LBLA approach proved extremely effective, outperforming other models including those that use a most complete set of features. One of the learners we use, is the recent NGBoost method that assumes a Poisson distribution for the number of new outlinks on a page, and learns its parameters. This connects the two so far unrelated avenues in the literature: predictions based on features of a page, and those based on probabilistic modelling. All experiments were carried out on an original dataset, made available by a commercial focused crawler.

cs.LG

Red Light Green Light Method for Solving Large Markov Chains

Discrete-time discrete-state finite Markov chains are versatile mathematical models for a wide range of real-life stochastic processes. One of most common tasks in studies of Markov chains is computation of the stationary distribution. Without loss of generality, and drawing our motivation from applications to large networks, we interpret this problem as one of computing the stationary distribution of a random walk on a graph. We propose a new controlled, easily distributed algorithm for this task, briefly summarized as follows: at the beginning, each node receives a fixed amount of cash (positive or negative), and at each iteration, some nodes receive `green light' to distribute their wealth or debt proportionally to the transition probabilities of the Markov chain; the stationary probability of a node is computed as a ratio of the cash distributed by this node to the total cash distributed by all nodes together. Our method includes as special cases a wide range of known, very different, and previously disconnected methods including power iterations, Gauss-Southwell, and online distributed algorithms. We prove exponential convergence of our method, demonstrate its high efficiency, and derive scheduling strategies for the green-light, that achieve convergence rate faster than state-of-the-art algorithms.

math.NA

When Less Is More: Systematic Analysis of Cascade-Based Community Detection

Information diffusion, spreading of infectious diseases, and spreading of rumors are fundamental processes occurring in real-life networks. In many practical cases, one can observe when nodes become infected, but the underlying network, over which a contagion or information propagates, is hidden. Inferring properties of the underlying network is important since these properties can be used for constraining infections, forecasting, viral marketing, and so on. Moreover, for many applications, it is sufficient to recover only coarse high-level properties of this network rather than all its edges. This article conducts a systematic and extensive analysis of the following problem: Given only the infection times, find communities of highly interconnected nodes. This task significantly differs from the well-studied community detection problem since we do not observe a graph to be clustered. We carry out a thorough comparison between existing and new approaches on several large datasets and cover methodological challenges specific to this problem. One of the main conclusions is that the most stable performance and the most significant improvement on the current state-of-the-art are achieved by our proposed simple heuristic approaches agnostic to a particular graph structure and epidemic model. We also show that some well-known community detection algorithms can be enhanced by including edge weights based on the cascade data.

cs.SI

The role of inter-regional mobility in forecasting SARS-CoV-2 transmission

In this paper, we present a method to forecast the spread of SARS-CoV-2 across regions with a focus on the role of mobility. Mobility has previously been shown to play a significant role in the spread of the virus, particularly between regions. Here, we investigate under which epidemiological circumstances incorporating mobility into transmission models yields improvements in the accuracy of forecasting, where we take the situation in the Netherlands during and after the first wave of transmission in 2020 as a case study. We assess the quality of forecasting on the detailed level of municipalities, instead of on a nation-wide level. To model transmissions, we use a simple mobility-enhanced SEIR compartmental model with subpopulations corresponding to the Dutch municipalities. We use commuter information to quantify mobility, and develop a method based on maximum likelihood estimation to determine the other relevant parameters. We show that taking inter-regional mobility into account generally leads to an improvement in forecast quality. However, at times when policies are in place that aim to reduce contacts or travel, this improvement is very small.

math.OC