arXiv subjects

Andrea Montanari

Publications and source records attributed to Andrea Montanari.

At least 199 records · Page 11Linked to original sources

arXiv2007

Estimating Random Variables from Random Sparse Observations

Let X_1,...., X_n be a collection of iid discrete random variables, and Y_1,..., Y_m a set of noisy observations of such variables. Assume each observation Y_a to be a random function of some a random subset of the X_i's, and consider the conditional distribution of X_i given the observations, namely μ_i(x_i)\equiv\prob\{X_i=x_i|Y\} (a posteriori probability). We establish a general relation between the distribution of μ_i, and the fixed points of the associated density evolution operator. Such relation holds asymptotically in the large system limit, provided the average number of variables an observation depends on is bounded. We discuss the relevance of our result to a number of applications, ranging from sparse graph codes, to multi-user detection, to group testing.

cs.IT↗

arXiv2007

A simple one dimensional glassy Kac model

We define a new family of random spin models with one-dimensional structure, finite-range multi-spin interactions, and bounded average degree (number of interactions in which each spin participates). Unfrustrated ground states can be described as solutions of a sparse, band diagonal linear system, thus allowing for efficient numerical analysis. In the limit of infinite interaction range, we recover the so-called XORSAT (diluted p-spin) model, that is known to undergo a random first order phase transition as the average degree is increased. Here we investigate the most important consequences of a large but finite interaction range: (i) Fluctuation-induced corrections to thermodynamic quantities; (ii) The need of an inhomogeneous (position dependent) order parameter; (iii) The emergence of a finite mosaic length scale. In particular, we study the correlation length divergence at the (mean-field) glass transition.

cond-mat.stat-mech↗

arXiv2007

Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View

These are the notes for a set of lectures delivered by the two authors at the Les Houches Summer School on `Complex Systems' in July 2006. They provide an introduction to the basic concepts in modern (probabilistic) coding theory, highlighting connections with statistical mechanics. We also stress common concepts with other disciplines dealing with similar problems that can be generically referred to as `large graphical models'. While most of the lectures are devoted to the classical channel coding problem over simple memoryless channels, we present a discussion of more complex channel models. We conclude with an overview of the main open challenges in the field.

cs.IT↗

arXiv2007

Analytic determination of dynamical and mosaic length scales in a Kac glass model

We consider a disordered spin model with multi-spin interactions undergoing a glass transition. We introduce a dynamic and a static length scales and compute them in the Kac limit (long--but--finite range interactions). They diverge at the dynamic and static phase transition with exponents (respectively) -1/4 and -1. The two length scales are approximately equal well above the mode coupling transition. Their discrepancy increases rapidly as this transition is approached. We argue that this signals a crossover from mode coupling to activated dynamics.

cond-mat.dis-nn↗

arXiv2007

Gibbs States and the Set of Solutions of Random Constraint Satisfaction Problems

An instance of a random constraint satisfaction problem defines a random subset S (the set of solutions) of a large product space (the set of assignments). We consider two prototypical problem ensembles (random k-satisfiability and q-coloring of random regular graphs), and study the uniform measure with support on S. As the number of constraints per variable increases, this measure first decomposes into an exponential number of pure states ("clusters"), and subsequently condensates over the largest such states. Above the condensation point, the mass carried by the n largest states follows a Poisson-Dirichlet process. For typical large instances, the two transitions are sharp. We determine for the first time their precise location. Further, we provide a formal definition of each phase transition in terms of different notions of correlation between distinct variables in the problem. The degree of correlation naturally affects the performances of many search/sampling algorithms. Empirical evidence suggests that local Monte Carlo Markov Chain strategies are effective up to the clustering phase transition, and belief propagation up to the condensation point. Finally, refined message passing techniques (such as survey propagation) may beat also this threshold.

cond-mat.stat-mech↗

arXiv2006

Counting good truth assignments of random k-SAT formulae

We present a deterministic approximation algorithm to compute logarithm of the number of `good' truth assignments for a random k-satisfiability (k-SAT) formula in polynomial time (by `good' we mean that violate a small fraction of clauses). The relative error is bounded above by an arbitrarily small constant epsilon with high probability as long as the clause density (ratio of clauses to variables) alpha<alpha_{u}(k) = 2k^{-1}\log k(1+o(1)). The algorithm is based on computation of marginal distribution via belief propagation and use of an interpolation procedure. This scheme substitutes the traditional one based on approximation of marginal probabilities via MCMC, in conjunction with self-reduction, which is not easy to extend to the present problem. We derive 2k^{-1}\log k (1+o(1)) as threshold for uniqueness of the Gibbs distribution on satisfying assignment of random infinite tree k-SAT formulae to establish our results, which is of interest in its own right.

cs.DM↗

arXiv2006

How to Find Good Finite-Length Codes: From Art Towards Science

We explain how to optimize finite-length LDPC codes for transmission over the binary erasure channel. Our approach relies on an analytic approximation of the erasure probability. This is in turn based on a finite-length scaling result to model large scale erasures and a union bound involving minimal stopping sets to take into account small error events. We show that the performances of optimized ensembles as observed in simulations are well described by our approximation. Although we only address the case of transmission over the binary erasure channel, our method should be applicable to a more general setting.

cs.IT↗

arXiv2006

Belief Propagation Based Multi--User Detection

We apply belief propagation (BP) to multi--user detection in a spread spectrum system, under the assumption of Gaussian symbols. We prove that BP is both convergent and allows to estimate the correct conditional expectation of the input symbols. It is therefore an optimal --minimum mean square error-- detection algorithm. This suggests the possibility of designing BP detection algorithms for more general systems. As a byproduct we rederive the Tse-Hanly formula for minimum mean square error without any recourse to random matrix theory.

cs.IT↗

arXiv2006

Analysis of Belief Propagation for Non-Linear Problems: The Example of CDMA (or: How to Prove Tanaka's Formula)

We consider the CDMA (code-division multiple-access) multi-user detection problem for binary signals and additive white gaussian noise. We propose a spreading sequences scheme based on random sparse signatures, and a detection algorithm based on belief propagation (BP) with linear time complexity. In the new scheme, each user conveys its power onto a finite number of chips l, in the large system limit. We analyze the performances of BP detection and prove that they coincide with the ones of optimal (symbol MAP) detection in the l->\infty limit. In the same limit, we prove that the information capacity of the system converges to Tanaka's formula for random `dense' signatures, thus providing the first rigorous justification of this formula. Apart from being computationally convenient, the new scheme allows for optimization in close analogy with irregular low density parity check code ensembles.

cs.IT↗

arXiv2005

Two Lectures on Iterative Coding and Statistical Mechanics

These are the notes for two lectures delivered at the Les Houches summer school Mathematical Statistical Mechanics, held in July 2005. I review some basic notions on sparse graph error correcting codes with emphasis on `modern' aspects, such as, iterative belief propagation decoding. Relations with statistical mechanics, inference and random combinatorial optimization are stressed, as well as some general mathematical ideas and open problems.

cond-mat.stat-mech↗

arXiv2005

Reconstruction on trees and spin glass transition

Consider an information source generating a symbol at the root of a tree network whose links correspond to noisy communication channels, and broadcasting it through the network. We study the problem of reconstructing the transmitted symbol from the information received at the leaves. In the large system limit, reconstruction is possible when the channel noise is smaller than a threshold. We show that this threshold coincides with the dynamical (replica symmetry breaking) glass transition for an associated statistical physics problem. Motivated by this correspondence, we derive a variational principle which implies new rigorous bounds on the reconstruction threshold. Finally, we apply a standard numerical procedure used in statistical physics, to predict the reconstruction thresholds in various channels. In particular, we prove a bound on the reconstruction problem for the antiferromagnetic ``Potts'' channels, which implies, in the noiseless limit, new results on random proper colorings of infinite regular trees. This relation to the reconstruction problem also offers interesting perspective for putting on a clean mathematical basis the theory of glasses on random graphs.

cond-mat.stat-mech↗

arXiv2005

The Generalized Area Theorem and Some of its Consequences

There is a fundamental relationship between belief propagation and maximum a posteriori decoding. The case of transmission over the binary erasure channel was investigated in detail in a companion paper. This paper investigates the extension to general memoryless channels (paying special attention to the binary case). An area theorem for transmission over general memoryless channels is introduced and some of its many consequences are discussed. We show that this area theorem gives rise to an upper-bound on the maximum a posteriori threshold for sparse graph codes. In situations where this bound is tight, the extrinsic soft bit estimates delivered by the belief propagation decoder coincide with the correct a posteriori probabilities above the maximum a posteriori threshold. More generally, it is conjectured that the fundamental relationship between the maximum a posteriori and the belief propagation decoder which was observed for transmission over the binary erasure channel carries over to the general case. We finally demonstrate that in order for the design rate of an ensemble to approach the capacity under belief propagation decoding the component codes have to be perfectly matched, a statement which is well known for the special case of transmission over the binary erasure channel.

cs.IT↗

arXiv2005

Why We Can Not Surpass Capacity: The Matching Condition

We show that iterative coding systems can not surpass capacity using only quantities which naturally appear in density evolution. Although the result in itself is trivial, the method which we apply shows that in order to achieve capacity the various components in an iterative coding system have to be perfectly matched. This generalizes the perfect matching condition which was previously known for the case of transmission over the binary erasure channel to the general class of binary-input memoryless output-symmetric channels. Potential applications of this perfect matching condition are the construction of capacity-achieving degree distributions and the determination of the number required iterations as a function of the multiplicative gap to capacity.

cs.IT↗

arXiv2005

On the dynamics of the glass transition on Bethe lattices

The Glauber dynamics of disordered spin models with multi-spin interactions on sparse random graphs (Bethe lattices) is investigated. Such models undergo a dynamical glass transition upon decreasing the temperature or increasing the degree of constrainedness. Our analysis is based upon a detailed study of large scale rearrangements which control the slow dynamics of the system close to the dynamical transition. Particular attention is devoted to the neighborhood of a zero temperature tricritical point. Both the approach and several key results are conjectured to be valid in a considerably more general context.

cond-mat.dis-nn↗

arXiv2005

How to Compute Loop Corrections to Bethe Approximation

We introduce a method for computing corrections to Bethe approximation for spin models on arbitrary lattices. Unlike cluster variational methods, the new approach takes into account fluctuations on all length scales. The derivation of the leading correction is explained and applied to two simple examples: the ferromagnetic Ising model on d-dimensional lattices, and the spin glass on random graphs (both in their high-temperature phases). In the first case we rederive the well-known Ginzburg criterion and the upper critical dimension. In the second, we compute finite-size corrections to the free energy.

cond-mat.stat-mech↗

arXiv2005

Maxwell Construction: The Hidden Bridge between Iterative and Maximum a Posteriori Decoding

There is a fundamental relationship between belief propagation and maximum a posteriori decoding. A decoding algorithm, which we call the Maxwell decoder, is introduced and provides a constructive description of this relationship. Both, the algorithm itself and the analysis of the new decoder are reminiscent of the Maxwell construction in thermodynamics. This paper investigates in detail the case of transmission over the binary erasure channel, while the extension to general binary memoryless channels is discussed in a companion paper.

cs.IT↗

arXiv2005

Tight bounds for LDPC and LDGM codes under MAP decoding

A new method for analyzing low density parity check (LDPC) codes and low density generator matrix (LDGM) codes under bit maximum a posteriori probability (MAP) decoding is introduced. The method is based on a rigorous approach to spin glasses developed by Francesco Guerra. It allows to construct lower bounds on the entropy of the transmitted message conditional to the received one. Based on heuristic statistical mechanics calculations, we conjecture such bounds to be tight. The result holds for standard irregular ensembles when used over binary input output symmetric channels. The method is first developed for Tanner graph ensembles with Poisson left degree distribution. It is then generalized to `multi-Poisson' graphs, and, by a completion procedure, to arbitrary degree distribution.

cs.IT↗

arXiv2005

From Large Scale Rearrangements to Mode Coupling Phenomenology

We consider the equilibrium dynamics of Ising spin models with multi-spin interactions on sparse random graphs (Bethe lattices). Such models undergo a mean field glass transition upon increasing the graph connectivity or lowering the temperature. Focusing on the low temperature limit, we identify the large scale rearrangements responsible for the dynamical slowing-down near the transition. We are able to characterize exactly the dynamics near criticality by analyzing the statistical properties of such rearrangements. Our approach can be generalized to a large variety of glassy models on sparse random graphs, ranging from satisfiability to kinetically constrained models.

cond-mat.stat-mech↗