arXiv subjects

Rémi Monasson

Publications and source records attributed to Rémi Monasson.

At least 37 records · Page 2Linked to original sources

arXiv2019

Learning protein constitutive motifs from sequence data

Statistical analysis of evolutionary-related protein sequences provides insights about their structure, function, and history. We show that Restricted Boltzmann Machines (RBM), designed to learn complex high-dimensional data and their statistical features, can efficiently model protein families from sequence information. We here apply RBM to twenty protein families, and present detailed results for two short protein domains, Kunitz and WW, one long chaperone protein, Hsp70, and synthetic lattice proteins for benchmarking. The features inferred by the RBM are biologically interpretable: they are related to structure (such as residue-residue tertiary contacts, extended secondary motifs ($α$-helix and $β$-sheet) and intrinsically disordered regions), to function (such as activity and ligand specificity), or to phylogenetic identity. In addition, we use RBM to design new protein sequences with putative properties by composing and turning up or down the different modes at will. Our work therefore shows that RBM are a versatile and practical tool to unveil and exploit the genotype-phenotype relationship for protein families.

q-bio.QM↗

arXiv2019

Learning Compositional Representations of Interacting Systems with Restricted Boltzmann Machines: Comparative Study of Lattice Proteins

A Restricted Boltzmann Machine (RBM) is an unsupervised machine-learning bipartite graphical model that jointly learns a probability distribution over data and extracts their relevant statistical features. As such, RBM were recently proposed for characterizing the patterns of coevolution between amino acids in protein sequences and for designing new sequences. Here, we study how the nature of the features learned by RBM changes with its defining parameters, such as the dimensionality of the representations (size of the hidden layer) and the sparsity of the features. We show that for adequate values of these parameters, RBM operate in a so-called compositional phase in which visible configurations sampled from the RBM are obtained by recombining these features. We then compare the performance of RBM with other standard representation learning algorithms, including Principal or Independent Component Analysis, autoencoders (AE), variational auto-encoders (VAE), and their sparse variants. We show that RBM, due to the stochastic mapping between data configurations and representations, better capture the underlying interactions in the system and are significantly more robust with respect to sample size than deterministic methods such as PCA or ICA. In addition, this stochastic mapping is not prescribed a priori as in VAE, but learned from data, which allows RBM to show good performance even with shallow architectures. All numerical results are illustrated on synthetic lattice-protein data, that share similar statistical features with real protein sequences, and for which ground-truth interactions are known.

cs.LG↗

arXiv2017

Statistical Physics and Representations in Real and Artificial Neural Networks

This document presents the material of two lectures on statistical physics and neural representations, delivered by one of us (R.M.) at the Fundamental Problems in Statistical Physics XIV summer school in July 2017. In a first part, we consider the neural representations of space (maps) in the hippocampus. We introduce an extension of the Hopfield model, able to store multiple spatial maps as continuous, finite-dimensional attractors. The phase diagram and dynamical properties of the model are analyzed. We then show how spatial representations can be dynamically decoded using an effective Ising model capturing the correlation structure in the neural data, and compare applications to data obtained from hippocampal multi-electrode recordings and by (sub)sampling our attractor model. In a second part, we focus on the problem of learning data representations in machine learning, in particular with artificial neural networks. We start by introducing data representations through some illustrations. We then analyze two important algorithms, Principal Component Analysis and Restricted Boltzmann Machines, with tools from statistical physics.

physics.data-an↗

arXiv2017

Emergence of Compositional Representations in Restricted Boltzmann Machines

Extracting automatically the complex set of features composing real high-dimensional data is crucial for achieving high performance in machine--learning tasks. Restricted Boltzmann Machines (RBM) are empirically known to be efficient for this purpose, and to be able to generate distributed and graded representations of the data. We characterize the structural conditions (sparsity of the weights, low effective temperature, nonlinearities in the activation functions of hidden units, and adaptation of fields maintaining the activity in the visible layer) allowing RBM to operate in such a compositional phase. Evidence is provided by the replica analysis of an adequate statistical ensemble of random RBMs and by RBM trained on the handwritten digits dataset MNIST.

physics.data-an↗

arXiv2016

Inference of principal components of noisy correlation matrices with prior information

The problem of infering the top component of a noisy sample covariance matrix with prior information about the distribution of its entries is considered, in the framework of the spiked covariance model. Using the replica method of statistical physics the computation of the overlap between the top components of the sample and population covariance matrices is formulated as an explicit optimization problem for any kind of entry-wise prior information. The approach is illustrated on the case of top components including large entries, and the corresponding phase diagram is shown. The calculation predicts that the maximal sampling noise level at which the recovery of the top population component remains possible is higher than its counterpart in the spiked covariance model with no prior information.

cond-mat.stat-mech↗

arXiv2016

Benchmarking inverse statistical approaches for protein structure and design with exactly solvable models

Inverse statistical approaches to determine protein structure and function from Multiple Sequence Alignments (MSA) are emerging as powerful tools in computational biology. However the underlying assumptions of the relationship between the inferred effective Potts Hamiltonian and real protein structure and energetics remain untested so far. Here we use lattice protein model (LP) to benchmark those inverse statistical approaches. We build MSA of highly stable sequences in target LP structures, and infer the effective pairwise Potts Hamiltonians from those MSA. We find that inferred Potts Hamiltonians reproduce many important aspects of 'true' LP structures and energetics. Careful analysis reveals that effective pairwise couplings in inferred Potts Hamiltonians depend not only on the energetics of the native structure but also on competing folds; in particular, the coupling values reflect both positive design (stabilization of native conformation) and negative design (destabilization of competing folds). In addition to providing detailed structural information, the inferred Potts models used as protein Hamiltonian for design of new sequences are able to generate with high probability completely new sequences with the desired folds, which is not possible using independent-site models. Those are remarkable results as the effective LP Hamiltonians used to generate MSA are not simple pairwise models due to the competition between the folds. Our findings elucidate the reasons for the success of inverse approaches to the modelling of proteins from sequence data, and their limitations.

q-bio.BM↗

arXiv2015

On the entropy of protein families

Proteins are essential components of living systems, capable of performing a huge variety of tasks at the molecular level, such as recognition, signalling, copy, transport, ... The protein sequences realizing a given function may largely vary across organisms, giving rise to a protein family. Here, we estimate the entropy of those families based on different approaches, including Hidden Markov Models used for protein databases and inferred statistical models reproducing the low-order (1-and 2-point) statistics of multi-sequence alignments. We also compute the entropic cost, that is, the loss in entropy resulting from a constraint acting on the protein, such as the fixation of one particular amino-acid on a specific site, and relate this notion to the escape probability of the HIV virus. The case of lattice proteins, for which the entropy can be computed exactly, allows us to provide another illustration of the concept of cost, due to the competition of different folds. The relevance of the entropy in relation to directed evolution experiments is stressed.

q-bio.BM↗

arXiv2015

Estimating the principal components of correlation matrices from all their empirical eigenvectors

We consider the problem of estimating the principal components of a population correlation matrix from a limited number of measurement data. Using a combination of random matrix and information-theoretic tools, we show that all the eigenmodes of the sample correlation matrices are informative, and not only the top ones. We show how this information can be exploited when prior information about the principal component, such as whether it is localized or not, is available by mapping the estimation problem onto the search for the ground state of a spin-glass-like effective Hamiltonian encoding the prior. Results are illustrated numerically on the spiked covariance model.

cond-mat.stat-mech↗

arXiv2015

Learning probabilities from random observables in high dimensions: the maximum entropy distribution and others

We consider the problem of learning a target probability distribution over a set of $N$ binary variables from the knowledge of the expectation values (with this target distribution) of $M$ observables, drawn uniformly at random. The space of all probability distributions compatible with these $M$ expectation values within some fixed accuracy, called version space, is studied. We introduce a biased measure over the version space, which gives a boost increasing exponentially with the entropy of the distributions and with an arbitrary inverse `temperature' $Γ$. The choice of $Γ$ allows us to interpolate smoothly between the unbiased measure over all distributions in the version space ($Γ=0$) and the pointwise measure concentrated at the maximum entropy distribution ($Γ\to \infty$). Using the replica method we compute the volume of the version space and other quantities of interest, such as the distance $R$ between the target distribution and the center-of-mass distribution over the version space, as functions of $α=(\log M)/N$ and $Γ$ for large $N$. Phase transitions at critical values of $α$ are found, corresponding to qualitative improvements in the learning of the target distribution and to the decrease of the distance $R$. However, for fixed $α$, the distance $R$ does not vary with $Γ$, which means that the maximum entropy distribution is not closer to the target distribution than any other distribution compatible with the observable values. Our results are confirmed by Monte Carlo sampling of the version space for small system sizes ($N\le 10$).

cond-mat.stat-mech↗

arXiv2014

Crosstalk and transitions between multiple spatial maps in an attractor neural network model of the hippocampus: Collective motion of the activity (II)

The dynamics of a neural model for hippocampal place cells storing spatial maps is studied. In the absence of external input, depending on the number of cells and on the values of control parameters (number of environments stored, level of neural noise, average level of activity, connectivity of place cells), a 'clump' of spatially-localized activity can diffuse, or remains pinned due to crosstalk between the environments. In the single-environment case, the macroscopic coefficient of diffusion of the clump and its effective mobility are calculated analytically from first principles, and corroborated by numerical simulations. In the multi-environment case the heights and the widths of the pinning barriers are analytically characterized with the replica method; diffusion within one map is then in competition with transitions between different maps. Possible mechanisms enhancing mobility are proposed and tested.

q-bio.NC↗

arXiv2013

Cross-talk and transitions between multiple spatial maps in an attractor neural network model of the hippocampus: phase diagram (I)

We study the stable phases of an attractor neural network model, with binary units, for hippocampal place cells encoding 1D or 2D spatial maps or environments. Using statistical mechanics tools we show that, below critical values for the noise in the neural response and for the number of environments, the network activity is spatially localized in one environment. We calculate the number of stored environments. For high noise and loads the network activity extends over space, either uniformly or with spatial heterogeneities due to the cross-talk between the maps, and memory of environments is lost. Analytical predictions are corroborated by numerical simulations.

cond-mat.stat-mech↗

arXiv2011

Adaptive cluster expansion for the inverse Ising problem: convergence, algorithm and tests

We present a procedure to solve the inverse Ising problem, that is to find the interactions between a set of binary variables from the measure of their equilibrium correlations. The method consists in constructing and selecting specific clusters of variables, based on their contributions to the cross-entropy of the Ising model. Small contributions are discarded to avoid overfitting and to make the computation tractable. The properties of the cluster expansion and its performances on synthetic data are studied. To make the implementation easier we give the pseudo-code of the algorithm.

cond-mat.dis-nn↗

arXiv2011

On the trajectories and performance of Infotaxis, an information-based greedy search algorithm

We present a continuous-space version of Infotaxis, a search algorithm where a searcher greedily moves to maximize the gain in information about the position of the target to be found. Using a combination of analytical and numerical tools we study the nature of the trajectories in two and three dimensions. The probability that the search is successful and the running time of the search are estimated. A possible extension to non-greedy search is suggested.

physics.bio-ph↗

arXiv2011

Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data

We introduce a procedure to infer the interactions among a set of binary variables, based on their sampled frequencies and pairwise correlations. The algorithm builds the clusters of variables contributing most to the entropy of the inferred Ising model, and rejects the small contributions due to the sampling noise. Our procedure successfully recovers benchmark Ising models even at criticality and in the low temperature phase, and is applied to neurobiological data.

physics.data-an↗

arXiv2008

Small-correlation expansions for the inverse Ising problem

We present a systematic small-correlation expansion to solve the inverse Ising problem: find a set of couplings and fields corresponding to a given set of correlations and magnetizations. Couplings are calculated up to the third order in the correlations for generic magnetizations, and to the seventh order in the case of zero magnetizations; in addition we show how to sum some useful classes of diagrams exactly. The resulting expansion outperforms existing algorithms on the Sherrington-Kirkpatrick spin-glass model.

cond-mat.dis-nn↗

arXiv2006

Criticality and Universality in the Unit-Propagation Search Rule

The probability Psuccess(alpha, N) that stochastic greedy algorithms successfully solve the random SATisfiability problem is studied as a function of the ratio alpha of constraints per variable and the number N of variables. These algorithms assign variables according to the unit-propagation (UP) rule in presence of constraints involving a unique variable (1-clauses), to some heuristic (H) prescription otherwise. In the infinite N limit, Psuccess vanishes at some critical ratio alpha\_H which depends on the heuristic H. We show that the critical behaviour is determined by the UP rule only. In the case where only constraints with 2 and 3 variables are present, we give the phase diagram and identify two universality classes: the power law class, where Psuccess[alpha\_H (1+epsilon N^{-1/3}), N] ~ A(epsilon)/N^gamma; the stretched exponential class, where Psuccess[alpha\_H (1+epsilon N^{-1/3}), N] ~ exp[-N^{1/6} Phi(epsilon)]. Which class is selected depends on the characteristic parameters of input data. The critical exponent gamma is universal and calculated; the scaling functions A and Phi weakly depend on the heuristic H and are obtained from the solutions of reaction-diffusion equations for 1-clauses. Computation of some non-universal corrections allows us to match numerical results with good precision. The critical behaviour for constraints with >3 variables is given. Our results are interpreted in terms of dynamical graph percolation and we argue that they should apply to more general situations where UP is used.

cond-mat.stat-mech↗

arXiv2006

Critical behaviour of combinatorial search algorithms, and the unitary-propagation universality class

The probability P(alpha, N) that search algorithms for random Satisfiability problems successfully find a solution is studied as a function of the ratio alpha of constraints per variable and the number N of variables. P is shown to be finite if alpha lies below an algorithm--dependent threshold alpha\_A, and exponentially small in N above. The critical behaviour is universal for all algorithms based on the widely-used unitary propagation rule: P[ (1 + epsilon) alpha\_A, N] ~ exp[-N^(1/6) Phi(epsilon N^(1/3)) ]. Exponents are related to the critical behaviour of random graphs, and the scaling function Phi is exactly calculated through a mapping onto a diffusion-and-death problem.

cond-mat.stat-mech↗

arXiv2005

Field theoretic approach to metastability in the contact process

A quantum field theoretic formulation of the dynamics of the Contact Process on a regular graph of degree z is introduced. A perturbative calculation in powers of 1/z of the effective potential for the density of particles phi(t) and an instantonic field psi(t) emerging from the quantum formalism is performed. Corrections to the mean-field distribution of densities of particles in the out-of-equilibrium stationary state are derived in powers of 1/z. Results for typical (e.g. average density) and rare fluctuation (e.g. lifetime of the metastable state) properties are in very good agreement with numerical simulations carried out on D-dimensional hypercubic (z=2D) and Cayley lattices.

cond-mat.stat-mech↗