arXiv subjects

Yitong Yin

Publications and source records attributed to Yitong Yin.

At least 55 records · Page 3Linked to original sources

arXiv2019

Distributed Metropolis Sampler with Optimal Parallelism

The Metropolis-Hastings algorithm is a fundamental Markov chain Monte Carlo (MCMC) method for sampling and inference. With the advent of Big Data, distributed and parallel variants of MCMC methods are attracting increased attention. In this paper, we give a distributed algorithm that can correctly simulate sequential single-site Metropolis chains without any bias in a fully asynchronous message-passing model. Furthermore, if a natural Lipschitz condition is satisfied by the Metropolis filters, our algorithm can simulate $N$-step Metropolis chains within $O(N/n+\log n)$ rounds of asynchronous communications, where $n$ is the number of variables. For sequential single-site dynamics, whose mixing requires $Ω(n\log n)$ steps, this achieves an optimal linear speedup. For several well-studied important graphical models, including proper graph coloring, hardcore model, and Ising model, our condition for linear speedup is weaker than the respective uniqueness (mixing) conditions. The novel idea in our algorithm is to resolve updates in advance: the local Metropolis filters can often be executed correctly before the full information about neighboring spins is available. This achieves optimal parallelism without introducing any bias.

cs.DS↗

arXiv2018

Dynamic Sampling from Graphical Models

In this paper, we study the problem of sampling from a graphical model when the model itself is changing dynamically with time. This problem derives its interest from a variety of inference, learning, and sampling settings in machine learning, computer vision, statistical physics, and theoretical computer science. While the problem of sampling from a static graphical model has received considerable attention, theoretical works for its dynamic variants have been largely lacking. The main contribution of this paper is an algorithm that can sample dynamically from a broad class of graphical models over discrete random variables. Our algorithm is parallel and Las Vegas: it knows when to stop and it outputs samples from the exact distribution. We also provide sufficient conditions under which this algorithm runs in time proportional to the size of the update, on general graphical models as well as well-studied specific spin systems. In particular we obtain, for the Ising model (ferromagnetic or anti-ferromagnetic) and for the hardcore model the first dynamic sampling algorithms that can handle both edge and vertex updates (addition, deletion, change of functions), both efficient within regimes that are close to the respective uniqueness regimes, beyond which, even for the static and approximate sampling, no local algorithms were known or the problem itself is intractable. Our dynamic sampling algorithm relies on a local resampling algorithm and a new "equilibrium" property that is shown to be satisfied by our algorithm at each step, and enables us to prove its correctness. This equilibrium property is robust enough to guarantee the correctness of our algorithm, helps us improve bounds on fast convergence on specific models, and should be of independent interest.

cs.DS↗

arXiv2018

Distributed Symmetry Breaking in Sampling (Optimal Distributed Randomly Coloring with Fewer Colors)

We examine the problem of almost-uniform sampling proper $q$-colorings of a graph whose maximum degree is $Δ$. A famous result, discovered independently by Jerrum(1995) and Salas and Sokal(1997), is that, assuming $q > (2+δ) Δ$, the Glauber dynamics (a.k.a. single-site dynamics) for this problem has mixing time $O(n \log n)$, where $n$ is the number of vertices, and thus provides a nearly linear time sampling algorithm for this problem. A natural question is the extent to which this algorithm can be parallelized. Previous work Feng, Sun and Yin [PODC'17] has shown that a $O(Δ\log n)$ time parallelized algorithm is possible, and that $Ω(\log n)$ time is necessary. We give a distributed sampling algorithm, which we call the Lazy Local Metropolis Algorithm, that achieves an optimal parallelization of this classic algorithm. It improves its predecessor, the Local Metropolis algorithm of Feng, Sun and Yin [PODC'17], by introducing a step of distributed symmetry breaking that helps the mixing of the distributed sampling algorithm. For sampling almost-uniform proper $q$-colorings of graphs $G$ on $n$ vertices, we show that the Lazy Local Metropolis algorithm achieves an optimal $O(\log n)$ mixing time if either of the following conditions is true for an arbitrary constant $δ>0$: $\bullet$ $q\ge(2+δ)Δ$, on general graphs with maximum degree $Δ$; $\bullet$ $q \geq (α^* + δ)Δ$, where $α^* \approx 1.763$ satisfies $α^* = \mathrm{e}^{1/α^*}$, on graphs with sufficiently large maximum degree $Δ\ge Δ_0(δ)$ and girth at least $9$.

cs.DS↗

arXiv2018

On Local Distributed Sampling and Counting

In classic distributed graph problems, each instance on a graph specifies a space of feasible solutions (e.g. all proper ($Δ+1$)-list-colorings of the graph), and the task of distributed algorithm is to construct a feasible solution using local information. We study distributed sampling and counting problems, in which each instance specifies a joint distribution of feasible solutions. The task of distributed algorithm is to sample from this joint distribution, or to locally measure the volume of the probability space via the marginal probabilities. The latter task is also known as inference, which is a local counterpart of counting. For self-reducible classes of instances, the following equivalences are established in the LOCAL model up to polylogarithmic factors: $\bullet$ For all joint distributions, approximate inference and approximate sampling are computationally equivalent. $\bullet$ For all joint distributions defined by local constraints, exact sampling is reducible to either one of the above tasks. $\bullet$ If further, sequentially constructing a feasible solution is trivial locally, then all above tasks are easy if and only if the joint distribution exhibits strong spatial mixing. Combining with the state of the arts of strong spatial mixing, we obtain efficient sampling algorithms in the LOCAL model for various important sampling problems, including: an $O(\sqrtΔ\log^3n)$-round algorithm for exact sampling matchings in graphs with maximum degree $Δ$, and an $O(\log^3n)$-round algorithm for sampling according to the hardcore model (weighted independent sets) in the uniqueness regime, which along with the $Ω(\mathrm{diam})$ lower bound in arXiv:1702.00142 for sampling according to the hardcore model in the non-uniqueness regime, gives the first computational phase transition for distributed sampling.

cs.DS↗

arXiv2017

Counting hypergraph matchings up to uniqueness threshold

We study the problem of approximately counting matchings in hypergraphs of bounded maximum degree and maximum size of hyperedges. With an activity parameter $λ$, each matching $M$ is assigned a weight $λ^{|M|}$. The counting problem is formulated as computing a partition function that gives the sum of the weights of all matchings in a hypergraph. This problem unifies two extensively studied statistical physics models in approximate counting: the hardcore model (graph independent sets) and the monomer-dimer model (graph matchings). For this model, the critical activity $λ_c= \frac{d^d}{k (d-1)^{d+1}}$ is the threshold for the uniqueness of Gibbs measures on the infinite $(d+1)$-uniform $(k+1)$-regular hypertree. Consider hypergraphs of maximum degree at most $k+1$ and maximum size of hyperedges at most $d+1$. We show that when $λ< λ_c$, there is an FPTAS for computing the partition function; and when $λ= λ_c$, there is a PTAS for computing the log-partition function. These algorithms are based on the decay of correlation (strong spatial mixing) property of Gibbs distributions. When $λ> 2λ_c$, there is no PRAS for the partition function or the log-partition function unless NP$=$RP. Towards obtaining a sharp transition of computational complexity of approximate counting, we study the local convergence from a sequence of finite hypergraphs to the infinite lattice with specified symmetry. We show a surprising connection between the local convergence and the reversibility of a natural random walk. This leads us to a barrier for the hardness result: The non-uniqueness of infinite Gibbs measure is not realizable by any finite gadgets.

cs.DS↗

arXiv2017

Simple average-case lower bounds for approximate near-neighbor from isoperimetric inequalities

We prove an $Ω(d/\log \frac{sw}{nd})$ lower bound for the average-case cell-probe complexity of deterministic or Las Vegas randomized algorithms solving approximate near-neighbor (ANN) problem in $d$-dimensional Hamming space in the cell-probe model with $w$-bit cells, using a table of size $s$. This lower bound matches the highest known worst-case cell-probe lower bounds for any static data structure problems. This average-case cell-probe lower bound is proved in a general framework which relates the cell-probe complexity of ANN to isoperimetric inequalities in the underlying metric space. A tighter connection between ANN lower bounds and isoperimetric inequalities is established by a stronger richness lemma proved by cell-sampling techniques.

cs.DS↗

arXiv2016

Convergence of MCMC and Loopy BP in the Tree Uniqueness Region for the Hard-Core Model

We study the hard-core model defined on independent sets of an input graph where the independent sets are weighted by a parameter $λ>0$. For constant $Δ$, previous work of Weitz (2006) established an FPTAS for the partition function for graphs of maximum degree $Δ$ when $λ< λ_c(Δ)$. The threshold $λ_c(Δ)$ is the critical point for the phase transition for uniqueness/non-uniqueness on the infinite $Δ$-regular trees. Sly (2010) showed that there is no FPRAS, unless NP=RP, when $λ>λ_c(Δ)$. The running time of Weitz's algorithm is exponential in $\log(Δ)$. Here we present an FPRAS for the partition function whose running time is $O^*(n^2)$. We analyze the simple single-site Glauber dynamics for sampling from the associated Gibbs distribution. We prove there exists a constant $Δ_0$ such that for all graphs with maximum degree $Δ\geqΔ_0$ and girth $\geq 7$, the mixing time of the Glauber dynamics is $O(n\log(n))$ when $λ<λ_c(Δ)$. Our work complements that of Weitz which applies for constant $Δ$ whereas our work applies for all $Δ\geq Δ_0$. We utilize loopy BP (belief propagation), a widely-used inference algorithm. A novel aspect of our work is using the principal eigenvector for the BP operator to design a distance function which contracts in expectation for pairs of states that behave like the BP fixed point. We also prove that the Glauber dynamics behaves locally like loopy BP. As a byproduct we obtain that the Glauber dynamics converges, after a short burn-in period, close to the BP fixed point, and this implies that the fixed point of loopy BP is a close approximation to the Gibbs distribution. Using these connections we establish that loopy BP quickly converges to the Gibbs distribution when the girth $\geq 6$ and $λ<λ_c(Δ)$.

cs.DM↗

arXiv2016

Randomized approximate nearest neighbor search with limited adaptivity

We study the fundamental problem of approximate nearest neighbor search in $d$-dimensional Hamming space $\{0,1\}^d$. We study the complexity of the problem in the famous cell-probe model, a classic model for data structures. We consider algorithms in the cell-probe model with limited adaptivity, where the algorithm makes $k$ rounds of parallel accesses to the data structure for a given $k$. For any $k\ge 1$, we give a simple randomized algorithm solving the approximate nearest neighbor search using $k$ rounds of parallel memory accesses, with $O(k(\log d)^{1/k})$ accesses in total. We also give a more sophisticated randomized algorithm using $O(k+(\frac{1}{k}\log d)^{O(1/k)})$ memory accesses in $k$ rounds for large enough $k$. Both algorithms use data structures of size polynomial in $n$, the number of points in the database. For the lower bound, we prove an $Ω(\frac{1}{k}(\log d)^{1/k})$ lower bound for the total number of memory accesses required by any randomized algorithm solving the approximate nearest neighbor search within $k\le\frac{\log\log d}{2\log\log\log d}$ rounds of parallel memory accesses on any data structures of polynomial size. This lower bound shows that our first algorithm is asymptotically optimal for any constant round $k$. And our second algorithm approaches the asymptotically optimal tradeoff between rounds and memory accesses, in a sense that the lower bound of memory accesses for any $k_1$ rounds can be matched by the algorithm within $k_2=O(k_1)$ rounds. In the extreme, for some large enough $k=Θ\left(\frac{\log\log d}{\log\log\log d}\right)$, our second algorithm matches the $Θ\left(\frac{\log\log d}{\log\log\log d}\right)$ tight bound for fully adaptive algorithms for approximate nearest neighbor search due to Chakrabarti and Regev.

cs.DS↗

arXiv2015

Sampling colorings almost uniformly in sparse random graphs

The problem of sampling proper $q$-colorings from uniform distribution has been extensively studied. Most of existing samplers require $q\ge αΔ+β$ for some constants $α$ and $β$, where $Δ$ is the maximum degree of the graph. The problem becomes more challenging when the underlying graph has unbounded degree since even the decision of $q$-colorability becomes nontrivial in this situation. The Erdős-Rényi random graph $\mathcal{G}(n,d/n)$ is a typical class of such graphs and has received a lot of recent attention. In this case, the performance of a sampler is usually measured by the relation between $q$ and the average degree $d$. We are interested in the fully polynomial-time almost uniform sampler (FPAUS) and the state-of-the-art with such sampler for proper $q$-coloring on $\mathcal{G}(n,d/n)$ requires that $q\ge 5.5d$. In this paper, we design an FPAUS for proper $q$-colorings on $\mathcal{G}(n,d/n)$ by requiring that $q\ge 3d+O(1)$, which improves the best bound for the problem so far. Our sampler is based on the spatial mixing property of $q$-coloring on random graphs. The core of the sampler is a deterministic algorithm to estimate the marginal probability on blocks, which is computed by a novel block version of recursion for $q$-coloring on unbounded degree graphs.

cs.DS↗

arXiv2015

Spatial mixing and approximate counting for Potts model on graphs with bounded average degree

We propose a notion of contraction function for a family of graphs and establish its connection to the strong spatial mixing for spin systems. More specifically, we show that for anti-ferromagnetic Potts model on families of graphs characterized by a specific contraction function, the model exhibits strong spatial mixing, and if further the graphs exhibit certain local sparsity which are very natural and easy to satisfy by typical sparse graphs, then we also have FPTAS for computing the partition function. This new characterization of strong spatial mixing of multi-spin system does not require maximum degree of the graphs to be bounded, but instead it relates the decay of correlation of the model to a notion of effective average degree measured by the contraction of a function on the family of graphs. It also generalizes other notion of effective average degree which may determine the strong spatial mixing, such as the connective constant, whose connection to strong spatial mixing is only known for very simple models and is not extendable to general spin systems. As direct consequences: (1) we obtain FPTAS for the partition function of $q$-state anti-ferromagnetic Potts model with activity $0\leβ<1$ on graphs of maximum degree bounded by $d$ when $q> 3(1-β)d+1$, improving the previous best bound $β> 3(1-β)d$ and asymptotically approaching the inapproximability threshold $q=(1-β)d$, and (2) we obtain an efficient sampler (in the same sense of fully polynomial-time almost uniform sampler, FPAUS) for the Potts model on Erdős-Rényi random graph $\mathcal{G}(n,d/n)$ with sufficiently large constant $d$, provided that $q> 3(1-β)d+4$. In particular when $β=0$, the sampler becomes an FPAUS for for proper $q$-coloring in $\mathcal{G}(n,d/n)$ with $q> 3d+4$, improving the current best bound $q> 5.5d$ for FPAUS for $q$-coloring in $\mathcal{G}(n,d/n)$.

cs.DS↗

arXiv2015

Expander Chunked Codes

Chunked codes are efficient random linear network coding (RLNC) schemes with low computational cost, where the input packets are encoded into small chunks (i.e., subsets of the coded packets). During the network transmission, RLNC is performed within each chunk. In this paper, we first introduce a simple transfer matrix model to characterize the transmission of chunks, and derive some basic properties of the model to facilitate the performance analysis. We then focus on the design of overlapped chunked codes, a class of chunked codes whose chunks are non-disjoint subsets of input packets, which are of special interest since they can be encoded with negligible computational cost and in a causal fashion. We propose expander chunked (EC) codes, the first class of overlapped chunked codes that have an analyzable performance,where the construction of the chunks makes use of regular graphs. Numerical and simulation results show that in some practical settings, EC codes can achieve rates within 91 to 97 percent of the optimum and outperform the state-of-the-art overlapped chunked codes significantly.

cs.IT↗

arXiv2014

Spatial mixing and approximation algorithms for graphs with bounded connective constant

The hard core model in statistical physics is a probability distribution on independent sets in a graph in which the weight of any independent set I is proportional to lambda^(|I|), where lambda > 0 is the vertex activity. We show that there is an intimate connection between the connective constant of a graph and the phenomenon of strong spatial mixing (decay of correlations) for the hard core model; specifically, we prove that the hard core model with vertex activity lambda < lambda_c(Delta + 1) exhibits strong spatial mixing on any graph of connective constant Delta, irrespective of its maximum degree, and hence derive an FPTAS for the partition function of the hard core model on such graphs. Here lambda_c(d) := d^d/(d-1)^(d+1) is the critical activity for the uniqueness of the Gibbs measure of the hard core model on the infinite d-ary tree. As an application, we show that the partition function can be efficiently approximated with high probability on graphs drawn from the random graph model G(n,d/n) for all lambda < e/d, even though the maximum degree of such graphs is unbounded with high probability. We also improve upon Weitz's bounds for strong spatial mixing on bounded degree graphs (Weitz, 2006) by providing a computationally simple method which uses known estimates of the connective constant of a lattice to obtain bounds on the vertex activities lambda for which the hard core model on the lattice exhibits strong spatial mixing. Using this framework, we improve upon these bounds for several lattices including the Cartesian lattice in dimensions 3 and higher. Our techniques also allow us to relate the threshold for the uniqueness of the Gibbs measure on a general tree to its branching factor (Lyons, 1989).

cs.DM↗

arXiv2014

Spatial mixing and the connective constant: Optimal bounds

We study the problem of deterministic approximate counting of matchings and independent sets in graphs of bounded connective constant. More generally, we consider the problem of evaluating the partition functions of the monomer-dimer model (which is defined as a weighted sum over all matchings where each matching is given a weight $γ^{|V| - 2 |M|}$ in terms of a fixed parameter gamma called the monomer activity) and the hard core model (which is defined as a weighted sum over all independent sets where an independent set I is given a weight $λ^{|I|}$ in terms of a fixed parameter lambda called the vertex activity). The connective constant is a natural measure of the average degree of a graph which has been studied extensively in combinatorics and mathematical physics, and can be bounded by a constant even for certain unbounded degree graphs such as those sampled from the sparse Erdős-Rényi model $G(n, d/n)$. Our main technical contribution is to prove the best possible rates of decay of correlations in the natural probability distributions induced by both the hard core model and the monomer-dimer model in graphs with a given bound on the connective constant. These results on decay of correlations are obtained using a new framework based on the so-called message approach that has been extensively used recently to prove such results for bounded degree graphs. We then use these optimal decay of correlations results to obtain FPTASs for the two problems on graphs of bounded connective constant. Our techniques also allow us to improve upon known bounds for decay of correlations for the hard core model on various regular lattices, including those obtained by Restrepo, Shin, Vigoda and Tetali (2011) for the special case of Z^2 using sophisticated numerically intensive methods tailored to that special case.

cs.DS↗

arXiv2014

Certificates in Data Structures

We study certificates in static data structures. In the cell-probe model, certificates are the cell probes which can uniquely identify the answer to the query. As a natural notion of nondeterministic cell probes, lower bounds for certificates in data structures immediately imply deterministic cell-probe lower bounds. In spite of this extra power brought by nondeterminism, we prove that two widely used tools for cell-probe lower bounds: richness lemma of Miltersen et al. and direct-sum richness lemma of Patrascu and Thorup, both hold for certificates in data structures with even better parameters. Applying these lemmas and adopting existing reductions, we obtain certificate lower bounds for a variety of static data structure problems. These certificate lower bounds are at least as good as the highest known cell-probe lower bounds for the respective problems. In particular, for approximate near neighbor (ANN) problem in Hamming distance, our lower bound improves the state of the art. When the space is strictly linear, our lower bound for ANN in d-dimensional Hamming space becomes t=Omega(d), which along with the recent breakthrough for polynomial evaluation of Larsen, are the only two t=Omega(d) lower bounds ever proved for any problems in the cell-probe model.

cs.DS↗

arXiv2014

Approximate Capacities of Two-Dimensional Codes by Spatial Mixing

We apply several state-of-the-art techniques developed in recent advances of counting algorithms and statistical physics to study the spatial mixing property of the two-dimensional codes arising from local hard (independent set) constraints, including: hard-square, hard-hexagon, read/write isolated memory (RWIM), and non-attacking kings (NAK). For these constraints, the strong spatial mixing would imply the existence of polynomial-time approximation scheme (PTAS) for computing the capacity. It was previously known for the hard-square constraint the existence of strong spatial mixing and PTAS. We show the existence of strong spatial mixing for hard-hexagon and RWIM constraints by establishing the strong spatial mixing along self-avoiding walks, and consequently we give PTAS for computing the capacities of these codes. We also show that for the NAK constraint, the strong spatial mixing does not hold along self-avoiding walks.

cs.IT↗

arXiv2014

Spatial Mixing of Coloring Random Graphs

We study the strong spatial mixing (decay of correlation) property of proper $q$-colorings of random graph $G(n, d/n)$ with a fixed $d$. The strong spatial mixing of coloring and related models have been extensively studied on graphs with bounded maximum degree. However, for typical classes of graphs with bounded average degree, such as $G(n, d/n)$, an easy counterexample shows that colorings do not exhibit strong spatial mixing with high probability. Nevertheless, we show that for $q\geαd+β$ with $α>2$ and sufficiently large $β=O(1)$, with high probability proper $q$-colorings of random graph $G(n, d/n)$ exhibit strong spatial mixing with respect to an arbitrarily fixed vertex. This is the first strong spatial mixing result for colorings of graphs with unbounded maximum degree. Our analysis of strong spatial mixing establishes a block-wise correlation decay instead of the standard point-wise decay, which may be of interest by itself, especially for graphs with unbounded degree.

cs.DS↗

arXiv2012

Approximate Counting via Correlation Decay on Planar Graphs

We show for a broad class of counting problems, correlation decay (strong spatial mixing) implies FPTAS on planar graphs. The framework for the counting problems considered by us is the Holant problems with arbitrary constant-size domain and symmetric constraint functions. We define a notion of regularity on the constraint functions, which covers a wide range of natural and important counting problems, including all multi-state spin systems, counting graph homomorphisms, counting weighted matchings or perfect matchings, the subgraphs world problem transformed from the ferromagnetic Ising model, and all counting CSPs and Holant problems with symmetric constraint functions of constant arity. The core of our algorithm is a fixed-parameter tractable algorithm which computes the exact values of the Holant problems with regular constraint functions on graphs of bounded treewidth. By utilizing the locally tree-like property of apex-minor-free families of graphs, the parameterized exact algorithm implies an FPTAS for the Holant problem on these graph families whenever the Gibbs measure defined by the problem exhibits strong spatial mixing. We further extend the recursive coupling technique to Holant problems and establish strong spatial mixing for the ferromagnetic Potts model and the subgraphs world problem. As consequences, we have new deterministic approximation algorithms on planar graphs and all apex-minor-free graphs for several counting problems.

cs.DS↗

arXiv2011

Approximate Counting via Correlation Decay in Spin Systems

We give the first deterministic fully polynomial-time approximation scheme (FPTAS) for computing the partition function of a two-state spin system on an arbitrary graph, when the parameters of the system satisfy the uniqueness condition on infinite regular trees. This condition is of physical significance and is believed to be the right boundary between approximable and inapproximable. The FPTAS is based on the correlation decay technique introduced by Bandyopadhyay and Gamarnik [SODA 06] and Weitz [STOC 06]. The classic correlation decay is defined with respect to graph distance. Although this definition has natural physical meanings, it does not directly support an FPTAS for systems on arbitrary graphs, because for graphs with unbounded degrees, the local computation that provides a desirable precision by correlation decay may take super-polynomial time. We introduce a notion of computationally efficient correlation decay, in which the correlation decay is measured in a refined metric instead of graph distance. We use a potential method to analyze the amortized behavior of this correlation decay and establish a correlation decay that guarantees an inverse-polynomial precision by polynomial-time local computation. This gives us an FPTAS for spin systems on arbitrary graphs. This new notion of correlation decay properly reflects the algorithmic aspect of the spin systems, and may be used for designing FPTAS for other counting problems.

cs.DS↗