arXiv subjects

Shai Carmi

Publications and source records attributed to Shai Carmi.

At least 19 recordsLinked to original sources

arXiv2018

Learning Binary Latent Variable Models: A Tensor Eigenpair Approach

Latent variable models with hidden binary units appear in various applications. Learning such models, in particular in the presence of noise, is a challenging computational problem. In this paper we propose a novel spectral approach to this problem, based on the eigenvectors of both the second order moment matrix and third order moment tensor of the observed data. We prove that under mild non-degeneracy conditions, our method consistently estimates the model parameters at the optimal parametric rate. Our tensor-based method generalizes previous orthogonal tensor decomposition approaches, where the hidden units were assumed to be either statistically independent or mutually exclusive. We illustrate the consistency of our method on simulated data and demonstrate its usefulness in learning a common model for population mixtures in genetics.

stat.ML↗

arXiv2015

A note on the distribution of admixture segment lengths and ancestry proportions under pulse and two-wave admixture models

Admixed populations are formed by the merging of two or more ancestral populations, and the ancestry of each locus in an admixed genome derives from either source. Consider a simple "pulse" admixture model, where populations A and B merged t generations ago without subsequent gene flow. We derive the distribution of the proportion of an admixed chromosome that has A (or B) ancestry, as a function of the chromosome length L, t, and the initial contribution of the A source, m. We demonstrate that these results can be used for inference of the admixture parameters. For more complex admixture models, we derive an expression in Laplace space for the distribution of ancestry proportions that depends on having the distribution of the lengths of segments of each ancestry. We obtain explicit results for the special case of a "two-wave" admixture model, where population A contributed additional migrants in one of the generations between the present and the initial admixture event. Specifically, we derive formulas for the distribution of A and B segment lengths and numerical results for the distribution of ancestry proportions. We show that for recent admixture, data generated under a two-wave model can hardly be distinguished from that generated under a pulse model.

q-bio.PE↗

arXiv2015

The SMC' is a highly accurate approximation to the ancestral recombination graph

Two sequentially Markov coalescent models (SMC and SMC') are available as tractable approximations to the ancestral recombination graph (ARG). We present a Markov process describing coalescence at two fixed points along a pair of sequences evolving under the SMC'. Using our Markov process, we derive a number of new quantities related to the pairwise SMC', thereby analytically quantifying for the first time the similarity between the SMC' and ARG. We use our process to show that the joint distribution of pairwise coalescence times at recombination sites under the SMC' is the same as it is marginally under the ARG, which demonstrates that the SMC' is, in a particular well-defined, intuitive sense, the most appropriate first-order sequentially Markov approximation to the ARG. Finally, we use these results to show that population size estimates under the pairwise SMC are asymptotically biased, while under the pairwise SMC' they are approximately asymptotically unbiased.

q-bio.PE↗

arXiv2014

A renewal theory approach to IBD sharing

A long genomic segment inherited by a pair of individuals from a single, recent common ancestor is said to be identical-by-descent (IBD). Shared IBD segments have numerous applications in genetics, from demographic inference to phasing, imputation, pedigree reconstruction, and disease mapping. Here, we provide a theoretical analysis of IBD sharing under Markovian approximations of the coalescent with recombination. We describe a general framework for the IBD process along the chromosome under the Markovian models (SMC/SMC'), as well as introduce and justify a new model, which we term the renewal approximation, under which lengths of successive segments are independent. Then, considering the infinite-chromosome limit of the IBD process, we recover previous results (for SMC) and derive new results (for SMC') for the mean number of shared segments longer than a cutoff and the fraction of the chromosome found in such segments. We then use renewal theory to derive an expression (in Laplace space) for the distribution of the number of shared segments and demonstrate implications for demographic inference. We also compute (again, in Laplace space) the distribution of the fraction of the chromosome in shared segments, from which we obtain explicit expressions for the first two moments. Finally, we generalize all results to populations with a variable effective size.

q-bio.PE↗

arXiv2013

The variance of identity-by-descent sharing in the Wright-Fisher model

Widespread sharing of long, identical-by-descent (IBD) genetic segments is a hallmark of populations that have experienced recent genetic drift. Detection of these IBD segments has recently become feasible, enabling a wide range of applications from phasing and imputation to demographic inference. Here, we study the distribution of IBD sharing in the Wright-Fisher model. Specifically, using coalescent theory, we calculate the variance of the total sharing between random pairs of individuals. We then investigate the cohort-averaged sharing: the average total sharing between one individual and the rest of the cohort. We find that for large cohorts, the cohort-averaged sharing is distributed approximately normally. Surprisingly, the variance of this distribution does not vanish even for large cohorts, implying the existence of "hyper-sharing" individuals. The presence of such individuals has consequences for the design of sequencing studies, since, if they are selected for whole-genome sequencing, a larger fraction of the cohort can be subsequently imputed. We calculate the expected gain in power of imputation by IBD, and subsequently, in power to detect an association, when individuals are either randomly selected or specifically chosen to be the hyper-sharing individuals. Using our framework, we also compute the variance of an estimator of the population size that is based on the mean IBD sharing and the variance in the sharing between inbred siblings. Finally, we study IBD sharing in an admixture pulse model, and show that in the Ashkenazi Jewish population the admixture fraction is correlated with the cohort-averaged sharing.

q-bio.PE↗

arXiv2013

Random walk with priorities in communication-like networks

We study a model for a random walk of two classes of particles (A and B). Where both species are present in the same site, the motion of A's takes precedence over that of B's. The model was originally proposed and analyzed in Maragakis et al., Phys. Rev. E 77, 020103 (2008); here we provide additional results. We solve analytically the diffusion coefficients of the two species in lattices for a number of protocols. In networks, we find that the probability of a B particle to be free decreases exponentially with the node degree. In scale-free networks, this leads to localization of the B's at the hubs and arrest of their motion. To remedy this, we investigate several strategies to avoid trapping of the B's: moving an A instead of the hindered B; allowing a trapped B to hop with a small probability; biased walk towards non-hub nodes; and limiting the capacity of nodes. We obtain analytic results for lattices and networks, and discuss the advantages and shortcomings of the possible strategies.

cond-mat.dis-nn↗

arXiv2011

A fractional Feynman-Kac equation for weak ergodicity breaking

Continuous-time random walk (CTRW) is a model of anomalous sub-diffusion in which particles are immobilized for random times between successive jumps. A power-law distribution of the waiting times, $ψ(τ) τ^{-(1+α)}$, leads to sub-diffusion ($ ~t^α$) for 0<α<1. In closed systems, the long stagnation periods cause time-averages to divert from the corresponding ensemble averages, which is a manifestation of weak ergodicity breaking. The time-average of a general observable $\bar{U} = \int_0^t U[x(τ)]dτ/ t$ is a functional of the path and is described by the well known Feynman-Kac equation if the motion is Brownian. Here, we derive forward and backward fractional Feynman-Kac equations for functionals of CTRW in a binding potential. We use our equations to study two specific time-averages: the fraction of time spent by a particle in half box, and the time-average of the particle's position in a harmonic field. In both cases, we obtain the probability density function of the time-averages for $t \rightarrow \infty$ and the first two moments. Our results show that both the occupation fraction and the time-averaged position are random variables even for long-times, except for α=1 when they are identical to their ensemble averages. Using the fractional Feynman-Kac equation, we also study the dynamics leading to weak ergodicity breaking, namely the convergence of the fluctuations to their asymptotic values.

cond-mat.stat-mech↗

arXiv2010

On distributions of functionals of anomalous diffusion paths

Functionals of Brownian motion have diverse applications in physics, mathematics, and other fields. The probability density function (PDF) of Brownian functionals satisfies the Feynman-Kac formula, which is a Schrodinger equation in imaginary time. In recent years there is a growing interest in particular functionals of non-Brownian motion, or anomalous diffusion, but no equation existed for their PDF. Here, we derive a fractional generalization of the Feynman-Kac equation for functionals of anomalous paths based on sub-diffusive continuous-time random walk. We also derive a backward equation and a generalization to Levy flights. Solutions are presented for a wide number of applications including the occupation time in half space and in an interval, the first passage time, the maximal displacement, and the hitting probability. We briefly discuss other fractional Schrodinger equations that recently appeared in the literature.

cond-mat.stat-mech↗

arXiv2010

Epidemic threshold for the SIS model on networks

We derive an analytical expression for the critical infection rate r_c of the susceptible-infectious-susceptible (SIS) disease spreading model on random networks. To obtain r_c, we first calculate the probability of reinfection, pi, defined as the probability of a node to reinfect the node that had earlier infected it. We then derive r_c from pi using percolation theory. We show that pi is governed by two effects: (i) The requirement from an infecting node to recover prior to its reinfection, which depends on the disease spreading parameters; and (ii) The competition between nodes that simultaneously try to reinfect the same ancestor, which depends on the network topology.

cond-mat.dis-nn↗

arXiv2009

From non-Brownian Functionals to a Fractional Schrödinger Equation

We derive backward and forward fractional Schrödinger type of equations for the distribution of functionals of the path of a particle undergoing anomalous diffusion. Fractional substantial derivatives introduced by Friedrich and co-workers [PRL {\bf 96}, 230601 (2006)] provide the correct fractional framework for the problem at hand. In the limit of normal diffusion we recover the Feynman-Kac treatment of Brownian functionals. For applications, we calculate the distribution of occupation times in half space and show how statistics of anomalous functionals is related to weak ergodicity breaking.

cond-mat.stat-mech↗

arXiv2009

Energy-landscape network approach to the glass transition

We study the energy-landscape network of Lennard-Jones clusters as a model of a glass forming system. We find the stable basins and the first order saddles connecting them, and identify them with the network nodes and links, respectively. We analyze the network properties and model the system's evolution. Using the model, we explore the system's response to varying cooling rates, and reproduce many of the glass transition properties. We also find that the static network structure gives rise to a critical temperature where a percolation transition breaks down the space of configurations into disconnected components. Finally, we discuss the possibility of studying the system mathematically with a trap-model generalized to networks.

cond-mat.soft↗

arXiv2009

Asymptotic behavior of the Kleinberg model

We study Kleinberg navigation (the search of a target in a d-dimensional lattice, where each site is connected to one other random site at distance r, with probability proportional to r^{-a}) by means of an exact master equation for the process. We show that the asymptotic scaling behavior for the delivery time T to a target at distance L scales as (ln L)^2 when a=d, and otherwise as L^x, with x=(d-a)/(d+1-a) for a d+1. These values of x exceed the rigorous lower-bounds established by Kleinberg. We also address the situation where there is a finite probability for the message to get lost along its way and find short delivery times (conditioned upon arrival) for a wide range of a's.

cond-mat.stat-mech↗

arXiv2008

Partition of Networks into Basins of Attraction

We study partition of networks into basins of attraction based on a steepest ascent search for the node of highest degree. Each node is associated with, or "attracted" to its neighbor of maximal degree, as long as the degree is increasing. A node that has no neighbors of higher degree is a peak, attracting all the nodes in its basin. Maximally random scale-free networks exhibit different behavior based on their degree distribution exponent $γ$: for small $γ$ (broad distribution) networks are dominated by a giant basin, whereas for large $γ$ (narrow distribution) there are numerous basins, with peaks attracting mainly their nearest neighbors. We derive expressions for the first two moments of the number of basins. We also obtain the complete distribution of basin sizes for a class of hierarchical deterministic scale-free networks that resemble random nets. Finally, we generalize the problem to regular networks and lattices where all degrees are equal, and thus the attractiveness of a node must be determined by an assigned weight, rather than the degree. We derive the complete distribution of basins of attraction resulting from randomly assigned weights in one-dimensional chains.

cond-mat.dis-nn↗

arXiv2008

Transport in networks with multiple sources and sinks

We investigate the electrical current and flow (number of parallel paths) between two sets of n sources and n sinks in complex networks. We derive analytical formulas for the average current and flow as a function of n. We show that for small n, increasing n improves the total transport in the network, while for large n bottlenecks begin to form. For the case of flow, this leads to an optimal n* above which the transport is less efficient. For current, the typical decrease in the length of the connecting paths for large n compensates for the effect of the bottlenecks. We also derive an expression for the average flow as a function of n under the common limitation that transport takes place between specific pairs of sources and sinks.

cs.DM↗

arXiv2008

Trapping in complex networks

We investigate the trapping problem in Erdos-Renyi (ER) and Scale-Free (SF) networks. We calculate the evolution of the particle density $ρ(t)$ of random walkers in the presence of one or multiple traps with concentration $c$. We show using theory and simulations that in ER networks, while for short times $ρ(t) \propto \exp(-Act)$, for longer times $ρ(t)$ exhibits a more complex behavior, with explicit dependence on both the number of traps and the size of the network. In SF networks we reveal the significant impact of the trap's location: $ρ(t)$ is drastically different when a trap is placed on a random node compared to the case of the trap being on the node with the maximum connectivity. For the latter case we find $ρ(t)\propto\exp\left[-At/N^\frac{γ-2}{γ-1}\av{k}\right]$ for all $γ>2$, where $γ$ is the exponent of the degree distribution $P(k)\propto k^{-γ}$.

cond-mat.dis-nn↗

arXiv2007

Priority diffusion model in lattices and complex networks

We introduce a model for diffusion of two classes of particles ($A$ and $B$) with priority: where both species are present in the same site the motion of $A$'s takes precedence over that of $B$'s. This describes realistic situations in wireless and communication networks. In regular lattices the diffusion of the two species is normal but the $B$ particles are significantly slower, due to the presence of the $A$ particles. From the fraction of sites where the $B$ particles can move freely, which we compute analytically, we derive the diffusion coefficients of the two species. In heterogeneous networks the fraction of sites where $B$ is free decreases exponentially with the degree of the sites. This, coupled with accumulation of particles in high-degree nodes leads to trapping of the low priority particles in scale-free networks.

cond-mat.stat-mech↗

arXiv2007

Limited path percolation in complex networks

We study the stability of network communication after removal of $q=1-p$ links under the assumption that communication is effective only if the shortest path between nodes $i$ and $j$ after removal is shorter than $a\ell_{ij} (a\geq1)$ where $\ell_{ij}$ is the shortest path before removal. For a large class of networks, we find a new percolation transition at $\tilde{p}_c=(κ_o-1)^{(1-a)/a}$, where $κ_o\equiv < k^2>/< k>$ and $k$ is the node degree. Below $\tilde{p}_c$, only a fraction $N^δ$ of the network nodes can communicate, where $δ\equiv a(1-|\log p|/\log{(κ_o-1)}) < 1$, while above $\tilde{p}_c$, order $N$ nodes can communicate within the limited path length $a\ell_{ij}$. Our analytical results are supported by simulations on Erdős-Rényi and scale-free network models. We expect our results to influence the design of networks, routing algorithms, and immunization strategies, where short paths are most relevant.

cond-mat.stat-mech↗

arXiv2006

Anomalous electrical and frictionless flow conductance in complex networks

We study transport properties such as electrical and frictionless flow conductance on scale-free and Erdos-Renyi networks. We consider the conductance G between two arbitrarily chosen nodes where each link has the same unit resistance. Our theoretical analysis for scale-free networks predicts a broad range of values of G, with a power-law tail distribution Φ_{SF}(G) \sim G^{g_G}, where g_G = 2λ- 1, where λis the decay exponent for the scale-free network degree distribution. We confirm our predictions by simulations of scale-free networks solving the Kirchhoff equations for the conductance between a pair of nodes. The power-law tail in Φ_{SF}(G) leads to large values of G, thereby significantly improving the transport in scale-free networks, compared to Erdos-Renyi networks where the tail of the conductivity distribution decays exponentially. Based on a simple physical 'transport backbone' picture we suggest that the conductances of scale-free and Erdos-Renyi networks can be approximated by ck_Ak_B/(k_A+k_B) for any pair of nodes A and B with degrees k_A and k_B. Thus, a single quantity c, which depends on the average degree of the network, characterizes transport on both scale-free and Erdos-Renyi networks. We determine that c tends to 1 for increasing , and it is larger for scale-free networks. We compare the electrical results with a model for frictionless transport, where conductance is defined as the number of link-independent paths between A and B, and find that a similar picture holds. The effects of distance on the value of conductance are considered for both models, and some differences emerge. Finally, we use a recent data set for the AS (autonomous system) level of the Internet and confirm that our results are valid in this real-world example.

cond-mat.dis-nn↗