arXiv subjects

Michael Chertkov

Publications and source records attributed to Michael Chertkov.

At least 163 records · Page 9Linked to original sources

arXiv2010

Worst Configurations (Instantons) for Compressed Sensing over Reals: a Channel Coding Approach

We consider the Linear Programming (LP) solution of the Compressed Sensing (CS) problem over reals, also known as the Basis Pursuit (BasP) algorithm. The BasP allows interpretation as a channel-coding problem, and it guarantees error-free reconstruction with a properly chosen measurement matrix and sufficiently sparse error vectors. In this manuscript, we examine how the BasP performs on a given measurement matrix and develop an algorithm to discover the sparsest vectors for which the BasP fails. The resulting algorithm is a generalization of our previous results on finding the most probable error-patterns degrading performance of a finite size Low-Density Parity-Check (LDPC) code in the error-floor regime. The BasP fails when its output is different from the actual error-pattern. We design a CS-Instanton Search Algorithm (ISA) generating a sparse vector, called a CS-instanton, such that the BasP fails on the CS-instanton, while the BasP recovery is successful for any modification of the CS-instanton replacing a nonzero element by zero. We also prove that, given a sufficiently dense random input for the error-vector, the CS-ISA converges to an instanton in a small finite number of steps. The performance of the CS-ISA is illustrated on a randomly generated $120\times 512$ matrix. For this example, the CS-ISA outputs the shortest instanton (error vector) pattern of length 11.

cs.IT↗

arXiv2009

Distributed control of reactive power flow in a radial distribution circuit with high photovoltaic penetration

We show how distributed control of reactive power can serve to regulate voltage and minimize resistive losses in a distribution circuit that includes a significant level of photovoltaic (PV) generation. To demonstrate the technique, we consider a radial distribution circuit with a single branch consisting of sequentially-arranged residential-scale loads that consume both real and reactive power. In parallel, some loads also have PV generation capability. We postulate that the inverters associated with each PV system are also capable of limited reactive power generation or consumption, and we seek to find the optimal dispatch of each inverter's reactive power to both maintain the voltage within an acceptable range and minimize the resistive losses over the entire circuit. We assume the complex impedance of the distribution circuit links and the instantaneous load and PV generation at each load are known. We compare the results of the optimal dispatch with a suboptimal local scheme that does not require any communication. On our model distribution circuit, we illustrate the feasibility of high levels of PV penetration and a significant (20% or higher) reduction in losses.

math.OC↗

arXiv2009

Non-Equilibrium Thermodynamics and Topology of Currents

In many experimental situations, a physical system undergoes stochastic evolution which may be described via random maps between two compact spaces. In the current work, we study the applicability of large deviations theory to time-averaged quantities which describe such stochastic maps, in particular time-averaged currents and density functionals. We derive the large deviations principle for these quantities, as well as for global topological currents, and formulate variational, thermodynamic relations to establish large deviation properties of the topological currents. We illustrate the theory with a nontrivial example of a Heisenberg spin-chain with a topological driving of the Wess-Zumino type. The Cramér functional of the topological current is found explicitly in the instanton gas regime for the spin-chain model in the weak-noise limit. In the context of the Morse theory, we discuss a general reduction of continuous stochastic models with weak noise to effective Markov chains describing transitions between stable fixed points.

cond-mat.stat-mech↗

arXiv2009

Message Passing for Integrating and Assessing Renewable Generation in a Redundant Power Grid

A simplified model of a redundant power grid is used to study integration of fluctuating renewable generation. The grid consists of large number of generator and consumer nodes. The net power consumption is determined by the difference between the gross consumption and the level of renewable generation. The gross consumption is drawn from a narrow distribution representing the predictability of aggregated loads, and we consider two different distributions representing wind and solar resources. Each generator is connected to D consumers, and redundancy is built in by connecting R of these consumers to other generators. The lines are switchable so that at any instance each consumer is connected to a single generator. We explore the capacity of the renewable generation by determining the level of "firm" generation capacity that can be displaced for different levels of redundancy R. We also develop message-passing control algorithm for finding switch settings where no generator is overloaded.

physics.soc-ph↗

arXiv2009

Message Passing for Optimization and Control of Power Grid: Model of Distribution System with Redundancy

We use a power grid model with $M$ generators and $N$ consumption units to optimize the grid and its control. Each consumer demand is drawn from a predefined finite-size-support distribution, thus simulating the instantaneous load fluctuations. Each generator has a maximum power capability. A generator is not overloaded if the sum of the loads of consumers connected to a generator does not exceed its maximum production. In the standard grid each consumer is connected only to its designated generator, while we consider a more general organization of the grid allowing each consumer to select one generator depending on the load from a pre-defined consumer-dependent and sufficiently small set of generators which can all serve the load. The model grid is interconnected in a graph with loops, drawn from an ensemble of random bipartite graphs, while each allowed configuration of loaded links represent a set of graph covering trees. Losses, the reactive character of the grid and the transmission-level connections between generators (and many other details relevant to realistic power grid) are ignored in this proof-of-principles study. We focus on the asymptotic limit and we show that the interconnects allow significant expansion of the parameter domains for which the probability of a generator overload is asymptotically zero. Our construction explores the formal relation between the problem of grid optimization and the modern theory of sparse graphical models. We also design heuristic algorithms that achieve the asymptotically optimal selection of loaded links. We conclude discussing the ability of this approach to include other effects, such as a more realistic modeling of the power grid and related optimization and control algorithms.

cond-mat.stat-mech↗

arXiv2009

Orbit-Product Representation and Correction of Gaussian Belief Propagation

We present a new view of Gaussian belief propagation (GaBP) based on a representation of the determinant as a product over orbits of a graph. We show that the GaBP determinant estimate captures totally backtracking orbits of the graph and consider how to correct this estimate. We show that the missing orbits may be grouped into equivalence classes corresponding to backtrackless orbits and the contribution of each equivalence class is easily determined from the GaBP solution. Furthermore, we demonstrate that this multiplicative correction factor can be interpreted as the determinant of a backtrackless adjacency matrix of the graph with edge weights based on GaBP. Finally, an efficient method is proposed to compute a truncated correction factor including all backtrackless orbits up to a specified length.

math.ST↗

arXiv2009

Bethe Free Energy Approach to LDPC Decoding on Memory Channels

We address the problem of the joint sequence detection in partial-response (PR) channels and decoding of low-density parity-check (LDPC) codes. We model the PR channel and the LDPC code as a combined inference problem. We present for the first time the derivation of the belief propagation (BP) equations that allow the simultaneous detection and decoding of a LDPC codeword in a PR channel. To accomplish this we follow an approach from statistical mechanics, in which the Bethe free energy is minimized with respect to the beliefs on the nodes of the PR-LDPC graph. The equations obtained are explicit and are optimal for decoding LDPC codes on PR channels with polynomial $h(D) = 1 - a D^n$ (a real, n positive integer) in the sense that they provide the exact inference of the marginal probabilities on the nodes in a graph free of loops. A simple algorithmic solution to the set of BP equations is proposed and evaluated using numerical simulations, yielding bit-error rate performances that surpass those of turbo equalization.

cs.IT↗

arXiv2009

Instanton-based Techniques for Analysis and Reduction of Error Floors of LDPC Codes

We describe a family of instanton-based optimization methods developed recently for the analysis of the error floors of low-density parity-check (LDPC) codes. Instantons are the most probable configurations of the channel noise which result in decoding failures. We show that the general idea and the respective optimization technique are applicable broadly to a variety of channels, discrete or continuous, and variety of sub-optimal decoders. Specifically, we consider: iterative belief propagation (BP) decoders, Gallager type decoders, and linear programming (LP) decoders performing over the additive white Gaussian noise channel (AWGNC) and the binary symmetric channel (BSC). The instanton analysis suggests that the underlying topological structures of the most probable instanton of the same code but different channels and decoders are related to each other. Armed with this understanding of the graphical structure of the instanton and its relation to the decoding failures, we suggest a method to construct codes whose Tanner graphs are free of these structures, and thus have less significant error floors.

cs.IT↗

arXiv2008

Fermions and Loops on Graphs. I. Loop Calculus for Determinant

This paper is the first in the series devoted to evaluation of the partition function in statistical models on graphs with loops in terms of the Berezin/fermion integrals. The paper focuses on a representation of the determinant of a square matrix in terms of a finite series, where each term corresponds to a loop on the graph. The representation is based on a fermion version of the Loop Calculus, previously introduced by the authors for graphical models with finite alphabets. Our construction contains two levels. First, we represent the determinant in terms of an integral over anti-commuting Grassman variables, with some reparametrization/gauge freedom hidden in the formulation. Second, we show that a special choice of the gauge, called BP (Bethe-Peierls or Belief Propagation) gauge, yields the desired loop representation. The set of gauge-fixing BP conditions is equivalent to the Gaussian BP equations, discussed in the past as efficient (linear scaling) heuristics for estimating the covariance of a sparse positive matrix.

cond-mat.stat-mech↗

arXiv2008

Fermions and Loops on Graphs. II. Monomer-Dimer Model as Series of Determinants

We continue the discussion of the fermion models on graphs that started in the first paper of the series. Here we introduce a Graphical Gauge Model (GGM) and show that : (a) it can be stated as an average/sum of a determinant defined on the graph over $\mathbb{Z}_{2}$ (binary) gauge field; (b) it is equivalent to the Monomer-Dimer (MD) model on the graph; (c) the partition function of the model allows an explicit expression in terms of a series over disjoint directed cycles, where each term is a product of local contributions along the cycle and the determinant of a matrix defined on the remainder of the graph (excluding the cycle). We also establish a relation between the MD model on the graph and the determinant series, discussed in the first paper, however, considered using simple non-Belief-Propagation choice of the gauge. We conclude with a discussion of possible analytic and algorithmic consequences of these results, as well as related questions and challenges.

cond-mat.stat-mech↗

arXiv2008

Irreversible Monte Carlo Algorithms for Efficient Sampling

Equilibrium systems evolve according to Detailed Balance (DB). This principe guided development of the Monte-Carlo sampling techniques, of which Metropolis-Hastings (MH) algorithm is the famous representative. It is also known that DB is sufficient but not necessary. We construct irreversible deformation of a given reversible algorithm capable of dramatic improvement of sampling from known distribution. Our transformation modifies transition rates keeping the structure of transitions intact. To illustrate the general scheme we design an Irreversible version of Metropolis-Hastings (IMH) and test it on example of a spin cluster. Standard MH for the model suffers from the critical slowdown, while IMH is free from critical slowdown.

cond-mat.stat-mech↗

arXiv2008

Loop Calculus and Belief Propagation for q-ary Alphabet: Loop Tower

Loop Calculus introduced in [Chertkov, Chernyak '06] constitutes a new theoretical tool that explicitly expresses the symbol Maximum-A-Posteriori (MAP) solution of a general statistical inference problem via a solution of the Belief Propagation (BP) equations. This finding brought a new significance to the BP concept, which in the past was thought of as just a loop-free approximation. In this paper we continue a discussion of the Loop Calculus. We introduce an invariant formulation which allows to generalize the Loop Calculus approach to a q-are alphabet.

cs.IT↗

arXiv2008

Provably efficient instanton search algorithm for LP decoding of LDPC codes over the BSC

We consider Linear Programming (LP) decoding of a fixed Low-Density Parity-Check (LDPC) code over the Binary Symmetric Channel (BSC). The LP decoder fails when it outputs a pseudo-codeword which is not a codeword. We design an efficient algorithm termed the Instanton Search Algorithm (ISA) which, given a random input, generates a set of flips called the BSC-instanton. We prove that: (a) the LP decoder fails for any set of flips with support vector including an instanton; (b) for any input, the algorithm outputs an instanton in the number of steps upper-bounded by twice the number of flips in the input. Repeated sufficient number of times, the ISA outcomes the number of unique instantons of different sizes.

cs.IT↗

arXiv2008

Exactness of Belief Propagation for Some Graphical Models with Loops

It is well known that an arbitrary graphical model of statistical inference defined on a tree, i.e. on a graph without loops, is solved exactly and efficiently by an iterative Belief Propagation (BP) algorithm convergent to unique minimum of the so-called Bethe free energy functional. For a general graphical model on a loopy graph the functional may show multiple minima, the iterative BP algorithm may converge to one of the minima or may not converge at all, and the global minimum of the Bethe free energy functional is not guaranteed to correspond to the optimal Maximum-Likelihood (ML) solution in the zero-temperature limit. However, there are exceptions to this general rule, discussed in \cite{05KW} and \cite{08BSS} in two different contexts, where zero-temperature version of the BP algorithm finds ML solution for special models on graphs with loops. These two models share a key feature: their ML solutions can be found by an efficient Linear Programming (LP) algorithm with a Totally-Uni-Modular (TUM) matrix of constraints. Generalizing the two models we consider a class of graphical models reducible in the zero temperature limit to LP with TUM constraints. Assuming that a gedanken algorithm, g-BP, funding the global minimum of the Bethe free energy is available we show that in the limit of zero temperature g-BP outputs the ML solution. Our consideration is based on equivalence established between gapless Linear Programming (LP) relaxation of the graphical model in the $T\to 0$ limit and respective LP version of the Bethe-Free energy minimization.

cond-mat.stat-mech↗

arXiv2008

Self-Similarity and Universality in Rayleigh-Taylor, Boussinesq Turbulence

We report and discuss case study simulations of the Rayleigh-Taylor instability in the Boussinesq, incompressible regime developed to turbulence. Our main focus is on a statistical analysis of density and velocity fluctuations inside of the already developed and growing in size mixing zone. Novel observations reported in the manuscript concern self-similarity of the velocity and density fluctuations spectra inside of the mixing zone snapshot, independence of the spectra of the horizontal slice level, and universality showing itself in a virtual independence of the internal structure of the mixing zone, measured in the re-scaled spatial units, of the initial interface perturbations.

physics.flu-dyn↗

arXiv2008

Belief Propagation and Beyond for Particle Tracking

We describe a novel approach to statistical learning from particles tracked while moving in a random environment. The problem consists in inferring properties of the environment from recorded snapshots. We consider here the case of a fluid seeded with identical passive particles that diffuse and are advected by a flow. Our approach rests on efficient algorithms to estimate the weighted number of possible matchings among particles in two consecutive snapshots, the partition function of the underlying graphical model. The partition function is then maximized over the model parameters, namely diffusivity and velocity gradient. A Belief Propagation (BP) scheme is the backbone of our algorithm, providing accurate results for the flow parameters we want to learn. The BP estimate is additionally improved by incorporating Loop Series (LS) contributions. For the weighted matching problem, LS is compactly expressed as a Cauchy integral, accurately estimated by a saddle point approximation. Numerical experiments show that the quality of our improved BP algorithm is comparable to the one of a fully polynomial randomized approximation scheme, based on the Markov Chain Monte Carlo (MCMC) method, while the BP-based scheme is substantially faster than the MCMC scheme.

cs.IT↗

arXiv2008

Belief Propagation and Loop Series on Planar Graphs

We discuss a generic model of Bayesian inference with binary variables defined on edges of a planar graph. The Loop Calculus approach of [1, 2] is used to evaluate the resulting series expansion for the partition function. We show that, for planar graphs, truncating the series at single-connected loops reduces, via a map reminiscent of the Fisher transformation [3], to evaluating the partition function of the dimer matching model on an auxiliary planar graph. Thus, the truncated series can be easily re-summed, using the Pfaffian formula of Kasteleyn [4]. This allows to identify a big class of computationally tractable planar models reducible to a dimer model via the Belief Propagation (gauge) transformation. The Pfaffian representation can also be extended to the full Loop Series, in which case the expansion becomes a sum of Pfaffian contributions, each associated with dimer matchings on an extension to a subgraph of the original graph. Algorithmic consequences of the Pfaffian representation, as well as relations to quantum and non-planar models, are discussed.

cond-mat.stat-mech↗

arXiv2007

Non-equilibrium thermodynamics for functionals of current and density

We study a stochastic many-body system maintained in an non-equilibrium steady state. Probability distribution functional of the time-integrated current and density is shown to attain a large-deviation form in the long-time asymptotics. The corresponding Current-Density Cramer Functional (CDCF) is explicitly derived for irreversible Langevin dynamics and discrete-space Markov chains. We also show that the Cramer functionals of other linear functionals of density and current, like work generated by a force, are related to CDCF in a way reminiscent of variational relations between different thermodynamic potentials. The general formalism is illustrated with a model example.

cond-mat.stat-mech↗