arXiv subjects

Adriano Barra

Publications and source records attributed to Adriano Barra.

At least 55 records · Page 3Linked to original sources

arXiv2018

Dreaming neural networks: rigorous results

Recently a daily routine for associative neural networks has been proposed: the network Hebbian-learns during the awake state (thus behaving as a standard Hopfield model), then, during its sleep state, optimizing information storage, it consolidates pure patterns and removes spurious ones: this forces the synaptic matrix to collapse to the projector one (ultimately approaching the Kanter-Sompolinksy model). This procedure keeps the learning Hebbian-based (a biological must) but, by taking advantage of a (properly stylized) sleep phase, still reaches the maximal critical capacity (for symmetric interactions). So far this emerging picture (as well as the bulk of papers on unlearning techniques) was supported solely by mathematically-challenging routes, e.g. mainly replica-trick analysis and numerical simulations: here we rely extensively on Guerra's interpolation techniques developed for neural networks and, in particular, we extend the generalized stochastic stability approach to the case. Confining our description within the replica symmetric approximation (where the previous ones lie), the picture painted regarding this generalization (and the previously existing variations on theme) is here entirely confirmed. Further, still relying on Guerra's schemes, we develop a systematic fluctuation analysis to check where ergodicity is broken (an analysis entirely absent in previous investigations). We find that, as long as the network is awake, ergodicity is bounded by the Amit-Gutfreund-Sompolinsky critical line (as it should), but, as the network sleeps, sleeping destroys spin glass states by extending both the retrieval as well as the ergodic region: after an entire sleeping session the solely surviving regions are retrieval and ergodic ones and this allows the network to achieve the perfect retrieval regime (the number of storable patterns equals the number of neurons in the network).

cond-mat.dis-nn↗

arXiv2018

A novel derivation of the Marchenko-Pastur law through analog bipartite spin-glasses

In this work we consider the {\em analog bipartite spin-glass} (or {\em real-valued restricted Boltzmann machine} in a neural network jargon), whose variables (those quenched as well as those dynamical) share standard Gaussian distributions. First, via Guerra's interpolation technique, we express its quenched free energy in terms of the natural order parameters of the theory (namely the self- and two-replica overlaps), then, we re-obtain the same result by using the replica-trick: a mandatory tribute, given the special occasion. Next, we show that the quenched free energy of this model is the functional generator of the moments of the correlation matrix among the weights connecting the two layers of the spin-glass (i.e., the Wishart matrix in random matrix theory or the Hebbian coupling in neural networks): as weights are quenched stochastic variables, this plays as a novel tool to inspect random matrices. In particular, we find that the Stieltjes transform of the spectral density of the correlation matrix is determined by the (replica-symmetric) quenched free energy of the bipartite spin-glass model. In this setup, we re-obtain the Marchenko-Pastur law in a very simple way.

cond-mat.dis-nn↗

arXiv2018

Dreaming neural networks: forgetting spurious memories and reinforcing pure ones

The standard Hopfield model for associative neural networks accounts for biological Hebbian learning and acts as the harmonic oscillator for pattern recognition, however its maximal storage capacity is $α\sim 0.14$, far from the theoretical bound for symmetric networks, i.e. $α=1$. Inspired by sleeping and dreaming mechanisms in mammal brains, we propose an extension of this model displaying the standard on-line (awake) learning mechanism (that allows the storage of external information in terms of patterns) and an off-line (sleep) unlearning$\&$consolidating mechanism (that allows spurious-pattern removal and pure-pattern reinforcement): this obtained daily prescription is able to saturate the theoretical bound $α=1$, remaining also extremely robust against thermal noise. Both neural and synaptic features are analyzed both analytically and numerically. In particular, beyond obtaining a phase diagram for neural dynamics, we focus on synaptic plasticity and we give explicit prescriptions on the temporal evolution of the synaptic matrix. We analytically prove that our algorithm makes the Hebbian kernel converge with high probability to the projection matrix built over the pure stored patterns. Furthermore, we obtain a sharp and explicit estimate for the "sleep rate" in order to ensure such a convergence. Finally, we run extensive numerical simulations (mainly Monte Carlo sampling) to check the approximations underlying the analytical investigations (e.g., we developed the whole theory at the so called replica-symmetric level, as standard in the Amit-Gutfreund-Sompolinsky reference framework) and possible finite-size effects, finding overall full agreement with the theory.

cs.NE↗

arXiv2018

The Relativistic Hopfield network: rigorous results

The relativistic Hopfield model constitutes a generalization of the standard Hopfield model that is derived by the formal analogy between the statistical-mechanic framework embedding neural networks and the Lagrangian mechanics describing a fictitious single-particle motion in the space of the tuneable parameters of the network itself. In this analogy the cost-function of the Hopfield model plays as the standard kinetic-energy term and its related Mattis overlap (naturally bounded by one) plays as the velocity. The Hamiltonian of the relativisitc model, once Taylor-expanded, results in a P-spin series with alternate signs: the attractive contributions enhance the information-storage capabilities of the network, while the repulsive contributions allow for an easier unlearning of spurious states, conferring overall more robustness to the system as a whole. Here we do not deepen the information processing skills of this generalized Hopfield network, rather we focus on its statistical mechanical foundation. In particular, relying on Guerra's interpolation techniques, we prove the existence of the infinite volume limit for the model free-energy and we give its explicit expression in terms of the Mattis overlaps. By extremizing the free energy over the latter we get the generalized self-consistent equations for these overlaps, as well as a picture of criticality that is further corroborated by a fluctuation analysis. These findings are in full agreement with the available previous results.

math-ph↗

arXiv2018

An evolutionary game model for behavioral gambit of loyalists: Global awareness and risk-aversion

We study the phase diagram of a minority game where three classes of agents are present. Two types of agents play a risk-loving game that we model by the standard Snowdrift Game. The behaviour of the third type of agents is coded by {\em indifference} w.r.t. the game at all: their dynamics is designed to account for risk-aversion as an innovative behavioral gambit. From this point of view, the choice of this solitary strategy is enhanced when innovation starts, while is depressed when it becomes the majority option. This implies that the payoff matrix of the game becomes dependent on the global awareness of the agents measured by the relevance of the population of the indifferent players. The resulting dynamics is non-trivial with different kinds of phase transition depending on a few model parameters. The phase diagram is studied on regular as well as complex networks.

physics.soc-ph↗

arXiv2018

A relativistic extension of Hopfield neural networks via the mechanical analogy

We propose a modification of the cost function of the Hopfield model whose salient features shine in its Taylor expansion and result in more than pairwise interactions with alternate signs, suggesting a unified framework for handling both with deep learning and network pruning. In our analysis, we heavily rely on the Hamilton-Jacobi correspondence relating the statistical model with a mechanical system. In this picture, our model is nothing but the relativistic extension of the original Hopfield model (whose cost function is a quadratic form in the Mattis magnetization which mimics the non-relativistic Hamiltonian for a free particle). We focus on the low-storage regime and solve the model analytically by taking advantage of the mechanical analogy, thus obtaining a complete characterization of the free energy and the associated self-consistency equations in the thermodynamic limit. On the numerical side, we test the performances of our proposal with MC simulations, showing that the stability of spurious states (limiting the capabilities of the standard Hebbian construction) is sensibly reduced due to presence of unlearning contributions in this extended framework.

cond-mat.dis-nn↗

arXiv2018

Complex Reaction Kinetics in Chemistry: A unified picture suggested by Mechanics in Physics

Complex biochemical pathways or regulatory enzyme kinetics can be reduced to chains of elementary reactions, which can be described in terms of chemical kinetics. This discipline provides a set of tools for quantifying and understanding the dialogue between reactants, whose framing into a solid and consistent mathematical description is of pivotal importance in the growing field of biotechnology. Among the elementary reactions so far extensively investigated, we recall the socalled Michaelis-Menten scheme and the Hill positive-cooperative kinetics, which apply to molecular binding and are characterized by the absence and the presence, respectively, of cooperative interactions between binding sites, giving rise to qualitative different phenomenologies. However, there is evidence of reactions displaying a more complex, and by far less understood, pattern: these follow the positive-cooperative scenario at small substrate concentration, yet negative-cooperative effects emerge and get stronger as the substrate concentration is increased. In this paper we analyze the structural analogy between the mathematical backbone of (classical) reaction kinetics in Chemistry and that of (classical) mechanics in Physics: techniques and results from the latter shall be used to infer properties on the former.

q-bio.QM↗

arXiv2017

Phase transitions in Restricted Boltzmann Machines with generic priors

We study Generalised Restricted Boltzmann Machines with generic priors for units and weights, interpolating between Boolean and Gaussian variables. We present a complete analysis of the replica symmetric phase diagram of these systems, which can be regarded as Generalised Hopfield models. We underline the role of the retrieval phase for both inference and learning processes and we show that retrieval is robust for a large class of weight and unit priors, beyond the standard Hopfield scenario. Furthermore we show how the paramagnetic phase boundary is directly related to the optimal size of the training set necessary for good generalisation in a teacher-student scenario of unsupervised learning.

cond-mat.dis-nn↗

arXiv2017

Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors

Restricted Boltzmann Machines are described by the Gibbs measure of a bipartite spin glass, which in turn corresponds to the one of a generalised Hopfield network. This equivalence allows us to characterise the state of these systems in terms of retrieval capabilities, both at low and high load. We study the paramagnetic-spin glass and the spin glass-retrieval phase transitions, as the pattern (i.e. weight) distribution and spin (i.e. unit) priors vary smoothly from Gaussian real variables to Boolean discrete variables. Our analysis shows that the presence of a retrieval phase is robust and not peculiar to the standard Hopfield model with Boolean patterns. The retrieval region is larger when the pattern entries and retrieval units get more peaked and, conversely, when the hidden units acquire a broader prior and therefore have a stronger response to high fields. Moreover, at low load retrieval always exists below some critical temperature, for every pattern distribution ranging from the Boolean to the Gaussian case.

cond-mat.dis-nn↗

arXiv2017

Retrieving Infinite Numbers of Patterns in a Spin-Glass Model of Immune Networks

The similarity between neural and immune networks has been known for decades, but so far we did not understand the mechanism that allows the immune system, unlike associative neural networks, to recall and execute a large number of memorized defense strategies {\em in parallel}. The explanation turns out to lie in the network topology. Neurons interact typically with a large number of other neurons, whereas interactions among lymphocytes in immune networks are very specific, and described by graphs with finite connectivity. In this paper we use replica techniques to solve a statistical mechanical immune network model with `coordinator branches' (T-cells) and `effector branches' (B-cells), and show how the finite connectivity enables the system to manage an extensive number of immune clones simultaneously, even above the percolation threshold. The system exhibits only weak ergodicity breaking, so that both multiple antigen defense and homeostasis can be accomplished.

cond-mat.dis-nn↗

arXiv2017

Neural Networks retrieving Boolean patterns in a sea of Gaussian ones

Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.

math-ph↗

arXiv2016

Complete integrability of information processing by biochemical reactions

Statistical mechanics provides an effective framework to investigate information processing in biochemical reactions. Within such framework far-reaching analogies are established among (anti-) cooperative collective behaviors in chemical kinetics, (anti-)ferromagnetic spin models in statistical mechanics and operational amplifiers/flip-flops in cybernetics. The underlying modeling -- based on spin systems -- has been proved to be accurate for a wide class of systems matching classical (e.g. Michaelis--Menten, Hill, Adair) scenarios in the infinite-size approximation. However, the current research in biochemical information processing has been focusing on systems involving a relatively small number of units, where this approximation is no longer valid. Here we show that the whole statistical mechanical description of reaction kinetics can be re-formulated via a mechanical analogy -- based on completely integrable hydrodynamic-type systems of PDEs -- which provides explicit finite-size solutions, matching recently investigated phenomena (e.g. noise-induced cooperativity, stochastic bi-stability, quorum sensing). The resulting picture, successfully tested against a broad spectrum of data, constitutes a neat rationale for a numerically effective and theoretically consistent description of collective behaviors in biochemical reactions.

cond-mat.stat-mech↗

arXiv2015

Emerging heterogeneities in Italian customs and comparison with nearby countries

In this work we apply techniques and modus operandi typical of Statistical Mechanics to a large dataset about key social quantifiers and compare the resulting behaviours of five European nations, namely France, Germany, Italy, Spain and Switzerland. The social quantifiers considered are $i.$ the evolution of the number of autochthonous marriages (i.e. between two natives) within a given territorial district and $ii.$ the evolution of the number of mixed marriages (i.e. between a native and an immigrant) within a given territorial district. Our investigations are twofold. From a theoretical perspective, we develop novel techniques, complementary to classical methods (e.g. historical series and logistic regression), in order to detect possible collective features underlying the empirical behaviours; from an experimental perspective, we evidence a clear outline for the evolution of the social quantifiers considered. The comparison between experimental results and theoretical predictions is excellent and allows speculating that France, Italy and Spain display a certain degree of {\em internal heterogeneity}, that is not found in Germany and Switzerland; such heterogeneity, quite mild in France and in Spain, is not negligible in Italy and highlights quantitative differences in the customs of Northern and Southern regions. These findings may suggest the persistence of two culturally distinct communities, long-term lasting heritages of different and well-established cultures.

physics.soc-ph↗

arXiv2015

Interpolating the Sherrington-Kirkpatrick replica trick

The interpolation techniques have become, in the past decades, a powerful approach to lighten several properties of spin glasses within a simple mathematical framework. Intrinsically, for their construction, these schemes were naturally implemented into the cavity field technique, or its variants as the stochastic stability or the random overlap structures. However the first and most famous approach to mean field statistical mechanics with quenched disorder is the replica trick. Among the models where these methods have been used (namely, dealing with frustration and complexity), probably the best known is the Sherrington-Kirkpatrick spin glass: In this paper we are pleased to apply the interpolation scheme to the replica trick framework and test it directly to the cited paradigmatic model: interestingly this allows to obtain easily the replica-symmetric control and, synergically with the broken replica bounds, a description of the full RSB scenario, both coupled with several minor theorems. Furthermore, by treating the amount of replicas $n\in(0,1]$ as an interpolating parameter (far from its original interpretation) this can be though of as a quenching temperature close to the one introduce in off-equilibrium approaches and, within this viewpoint, the proof of the attended commutativity of the zero replica and the infinite volume limits can be obtained.

cond-mat.dis-nn↗

arXiv2015

Topological properties of hierarchical networks

Hierarchical networks are attracting a renewal interest for modelling the organization of a number of biological systems and for tackling the complexity of statistical mechanical models beyond mean-field limitations. Here we consider the Dyson hierarchical construction for ferromagnets, neural networks and spin-glasses, recently analyzed from a statistical-mechanics perspective, and we focus on the topological properties of the underlying structures. In particular, we find that such structures are weighted graphs that exhibit high degree of clustering and of modularity, with small spectral gap; the robustness of such features with respect to link removal is also studied. These outcomes are then discussed and related to the statistical mechanics scenario in full consistency. Lastly, we look at these weighted graphs as Markov chains and we show that in the limit of infinite size, the emergence of ergodicity breakdown for the stochastic process mirrors the emergence of meta-stabilities in the corresponding statistical mechanical analysis.

cond-mat.dis-nn↗

arXiv2015

Exact solution of the van der Waals model in the critical region

The celebrated van der Waals model describes simple fluids in the thermodynamic limit and predicts the existence of a critical point associated to the gas-liquid phase transition. However the behaviour of critical isotherms according to the equation of state, where a gas-liquid phase transition occurs, significantly departs from experimental observations. The correct critical isotherms are heuristically re-established via the Maxwell equal areas rule. A long standing open problem in mean field theory is concerned with the analytic description of van der Waals isotherms for a finite size system that is consistent, in the thermodynamic limit, with the Maxwell prescription. Inspired by the theory of nonlinear conservation laws, we propose a novel mean field approach, based on statistical mechanics, that allows to calculate the van der Waals partition function for a system of large but finite number of particles $N$. Our partition function naturally extends to the whole space of thermodynamic variables, reproduces, in the thermodynamic limit $N\to \infty$, the classical results outside the critical region and automatically encodes Maxwell's prescription. We show that isothermal curves evolve in the space of thermodynamic variables like nonlinear breaking waves and the criticality is explained as the mechanism of formation of a classical hydrodynamic shock.

cond-mat.stat-mech↗

arXiv2015

Insights in Economical Complexity in Spain: the hidden boost of migrants in international tradings

We consider extensive data on Spanish international trades and population composition and, through statistical-mechanics and graph-theory driven analysis, we unveil that the social network made of native and foreign-born individuals plays a role in the evolution and in the diversification of trades. Indeed, migrants naturally provide key information on policies and needs in their native countries, hence allowing firm's holders to leverage transactional costs of exports and duties. As a consequence, international trading is affordable for a larger basin of firms and thus results in an increased number of transactions, which, in turn, implies a larger diversification of international traded products. These results corroborate the novel scenario depicted by "Economical Complexity", where the pattern of production and trade of more developed countries is highly diversified. We also address a central question in Economics, concerning the existence of a critical threshold for migrants (within a given territorial district) over which they effectively contribute to boost international trades: in our physically-driven picture, this phenomenon corresponds to the emergence of a phase transition and, tackling the problem from this perspective, results in a novel successful quantitative route. Finally, we can infer that the pattern of interaction between native and foreign-born population exhibits small-world features as small diameter, large clustering, and weak ties working as optimal cut-edge, in complete agreement with findings in "Social Complexity".

physics.soc-ph↗

arXiv2015

On quantum and relativistic mechanical analogues in mean field spin models

Conceptual analogies among statistical mechanics and classical (or quantum) mechanics often appeared in the literature. For classical two-body mean field models, an analogy develops into a proper identification between the free energy of Curie-Weiss type magnetic models and the Hamilton-Jacobi action for a one dimensional mechanical system. Similarly, the partition function plays the role of the wave function in quantum mechanics and satisfies the heat equation that plays, in this context, the role of the Schrodinger equation in quantum mechanics. We show that this identification can be remarkably extended to include a wide family of magnetic models classified by normal forms of suitable real algebraic dispersion curves. In all these cases, the model turns out to be completely solvable as the free energy as well as the order parameter are obtained as solutions of an integrable nonlinear PDE of Hamilton-Jacobi type. We observe that the mechanical analog of these models can be viewed as the relativistic analog of the Curie-Weiss model and this helps to clarify the connection between generalised self-averaging and in statistical thermodynamics and the semi-classical dynamics of viscous conservation laws.

cond-mat.dis-nn↗