arXiv subjects

Alessandro Treves

Publications and source records attributed to Alessandro Treves.

At least 19 recordsLinked to original sources

arXiv2025

Modeling Language Evolution Using a Spin Glass Approach

The evolution of natural languages poses a riddle to any theoretical perspective based on efficiency considerations. If languages are already optimally effective means of organization and communication of thought, why do they change? And if they are driven to become optimally effective in the future, why do they change so slowly, and why do they diversify, rather than converge towards an optimum? We look here at the hypothesis that disorder, rather than efficiency, may play a dominant role. Most traditional approaches to study diachronic language dynamics emphasize lexical data, but it would seem that a crucial contribution to the effectiveness of a thought-coding device is given by its core generative structure, i.e., its syntax. Based on the reduction of syntax to a set of binary syntactic parameters, we introduce here a model of natural language change in which diachronic dynamics stem from disordered interactions between/among parameters, even in the idealized limit of identical external inputs. We show in which region of phase space such dynamics show the glassy features that are observed in natural language across time. In particular, binary syntactic vectors remain trapped in glassy metastable (ie, tendentially stable) states when the degree of asymmetry in the disordered interactions is below a critical value, consistent with studies of spin glasses with asymmetric interactions. We further show that an added Hopfield-type memory term, would indeed, if strong enough, stabilize syntactic configurations, but losing their multiplicity. Finally, using a notion of linguistic distance in syntactic state space we show that a phylogenetic signal may remain among related languages, despite their gradually divergent syntax, exactly as recently pointed out for real-world languages. These statistical results appear to generalize beyond the 94 parameters across 58 languages used here.

q-bio.NC↗

arXiv2024

Guerra interpolation for place cells

Pyramidal cells that emit spikes when the animal is at specific locations of the environment are known as "place cells": these neurons are thought to provide an internal representation of space via "cognitive maps". Here, we consider the Battaglia-Treves neural network model for cognitive map storage and reconstruction, instantiated with McCulloch & Pitts binary neurons. To quantify the information processing capabilities of these networks, we exploit spin-glass techniques based on Guerra's interpolation: in the low-storage regime (i.e., when the number of stored maps scales sub-linearly with the network size and the order parameters self-average around their means) we obtain an exact phase diagram in the noise vs inhibition strength plane (in agreement with previous findings) by adapting the Hamilton-Jacobi PDE-approach. Conversely, in the high-storage regime, we find that -- for mild inhibition and not too high noise -- memorization and retrieval of an extensive number of spatial maps is indeed possible, since the maximal storage capacity is shown to be strictly positive. These results, holding under the replica-symmetry assumption, are obtained by adapting the standard interpolation based on stochastic stability and are further corroborated by Monte Carlo simulations (and replica-trick outcomes for the sake of completeness). Finally, by relying upon an interpretation in terms of hidden units, in the last part of the work, we adapt the Battaglia-Treves model to cope with more general frameworks, such as bats flying in long tunnels.

cond-mat.dis-nn↗

arXiv2023

Grammatical Parameters from a Gene-like Code to Self-Organizing Attractors

Parametric approaches to grammatical diversity range from Chomsky's 1981 classical Principles & Parameters model to minimalist reinterpretations: in some proposals of the latter framework, parameters need not be an extensional list given at the initial state S0 of the mind, but can be constructed through a bio-program in the course of language development. In this contribution we pursue this lead and discuss initial data and ideas relevant for the elaboration of three sets of questions: 1) how can binary parameters be conceivably implemented in cortical and subcortical circuitry in the human brain? 2) how can parameter mutations be taken to occur? 3) given the distribution of parameter values across languages and their implications, can multi-parental models of language phylogenies, departing from ultrametricity, also account for some of the available evidence?

q-bio.NC↗

arXiv2020

Efficiency of local learning rules in threshold-linear associative networks

We derive the Gardner storage capacity for associative networks of threshold linear units, and show that with Hebbian learning they can operate closer to such Gardner bound than binary networks, and even surpass it. This is largely achieved through a sparsification of the retrieved patterns, which we analyze for theoretical and empirical distributions of activity. As reaching the optimal capacity via non-local learning rules like backpropagation requires slow and neurally implausible training procedures, our results indicate that one-shot self-organized Hebbian learning can be just as efficient.

cond-mat.dis-nn↗

arXiv2018

Metric Learning for Phoneme Perception

Metric functions for phoneme perception capture the similarity structure among phonemes in a given language and therefore play a central role in phonology and psycho-linguistics. Various phenomena depend on phoneme similarity, such as spoken word recognition or serial recall from verbal working memory. This study presents a new framework for learning a metric function for perceptual distances among pairs of phonemes. Previous studies have proposed various metric functions, from simple measures counting the number of phonetic dimensions that two phonemes share (place-, manner-of-articulation and voicing), to more sophisticated ones such as deriving perceptual distances based on the number of natural classes that both phonemes belong to. However, previous studies have manually constructed the metric function, which may lead to unsatisfactory account of the empirical data. This study presents a framework to derive the metric function from behavioral data on phoneme perception using learning algorithms. We first show that this approach outperforms previous metrics suggested in the literature in predicting perceptual distances among phoneme pairs. We then study several metric functions derived by the learning algorithms and show how perceptual saliencies of phonological features can be derived from them. For English, we show that the derived perceptual saliencies are in accordance with a previously described order among phonological features and show how the framework extends the results to more features. Finally, we explore how the metric function and perceptual saliencies of phonological features may vary across languages. To this end, we compare results based on two English datasets and a new dataset that we have collected for Hebrew.

cs.LG↗

arXiv2018

Reducing a cortical network to a Potts model yields storage capacity estimates

An autoassociative network of Potts units, coupled via tensor connections, has been proposed and analysed as an effective model of an extensive cortical network with distinct short- and long-range synaptic connections, but it has not been clarified in what sense it can be regarded as an effective model. We draw here the correspondence between the two, which indicates the need to introduce a local feedback term in the reduced model, i.e., in the Potts network. An effective model allows the study of phase transitions. As an example, we study the storage capacity of the Potts network with this additional term, the local feedback $w$, which contributes to drive the activity of the network towards one of the stored patterns. The storage capacity calculation, performed using replica tools, is limited to fully connected networks, for which a Hamiltonian can be defined. To extend the results to the case of intermediate partial connectivity, we also derive the self-consistent signal-to-noise analysis for the Potts network; and finally we discuss implications for semantic memory in humans.

q-bio.NC↗

arXiv2015

Can rodents conceive hyperbolic spaces?

The grid cells discovered in the rodent medial entorhinal cortex have been proposed to provide a metric for Euclidean space, possibly even hardwired in the embryo. Yet one class of models describing the formation of grid unit selectivity is entirely based on developmental self-organization, and as such it predicts that the metric it expresses should reflect the environment to which the animal has adapted. We show that, according to self-organizing models, if raised in a non-Euclidean hyperbolic cage rats should be able to form hyperbolic grids. For a given range of grid spacing relative to the radius of negative curvature of the hyperbolic surface, such grids are predicted to appear as multi-peaked firing maps, in which each peak has seven neighbours instead of the Euclidean six, a prediction that can be tested in experiments. We thus demonstrate that a useful universal neuronal metric, in the sense of a multi-scale ruler and compass that remain unaltered when changing environments, can be extended to other than the standard Euclidean plane.

q-bio.NC↗

arXiv2014

The Self-Organization of Grid Cells in 3D

What sort of grid cells do we expect to see in bats exploring a three-dimensional environment? How long will it take for them to emerge? We address these questions within our self-organization model based on firing-rate adaptation. The model indicates that the answer to the first question may be simple, and to the second one rather complex. The mathematical analysis of the simplified version of the model points at asymptotic states resembling FCC and HCP crystal structures, which are calculated to be very close to each other in terms of cost function. The simulation of the full model, however, shows that the approach to such asymptotic states involves several sub-processes over distinct time scales. The smoothing of the initially irregular multiple fields of individual units and their arrangement into hexagonal grids over certain best planes are observed to occur relatively fast, even in large 3D volumes. The correct mutual orientation of the planes, however, and the coordinated arrangement of different units, take a longer time, with the network showing no sign of convergence towards either a pure FCC or HCP ordering.

q-bio.NC↗

arXiv2012

Grid Alignment in Entorhinal Cortex

The spatial responses of many of the cells recorded in all layers of rodent medial entorhinal cortex (mEC) show a triangular grid pattern, and once established might be based in part on path-integration mechanisms. Grid axes are tightly aligned across simultaneously recorded units. Recent experimental findings have shown that grids can often be better described as elliptical rather than purely circular and that, beyond the mutual alignment of their grid axes, ellipses tend to also orient their long axis along preferred directions. Are grid alignment and ellipse orientation the same phenomenon? Does the grid alignment result from single-unit mechanisms or does it require network interactions? We address these issues by refining our model, to describe specifically the spontaneous emergence of conjunctive grid-by-head-direction cells in layers III, V and VI of mEC. We find that tight alignment can be produced by recurrent collateral interactions, but this requires head-direction modulation. Through a competitive learning process driven by spatial inputs, grid fields then form already aligned, and with randomly distributed spatial phases. In addition, we find that the self-organization process is influenced by the behavior of the simulated rat. The common grid alignment often orients along preferred running directions. The shape of individual grids is distorted towards an ellipsoid arrangement when some speed anisotropy is present in exploration behavior. Speed anisotropy on its own also tends to align grids, even without collaterals, but the alignment is seen to be loose. Finally, the alignment of spatial grid fields in multiple environments shows that the network expresses the same set of grid fields across environments, modulo a coherent rotation and translation. Thus, an efficient metric encoding of space may emerge through spontaneous pattern formation at the single-unit level.

q-bio.NC↗

arXiv2012

Cortical free association dynamics: distinct phases of a latching network

A Potts associative memory network has been proposed as a simplified model of macroscopic cortical dynamics, in which each Potts unit stands for a patch of cortex, which can be activated in one of S local attractor states. The internal neuronal dynamics of the patch is not described by the model, rather it is subsumed into an effective description in terms of graded Potts units, with adaptation effects both specific to each attractor state and generic to the patch. If each unit, or patch, receives effective (tensor) connections from C other units, the network has been shown to be able to store a large number p of global patterns, or network attractors, each with a fraction a of the units active, where the critical load p_c scales roughly like p_c ~ (C S^2)/(a ln(1/a)) (if the patterns are randomly correlated). Interestingly, after retrieving an externally cued attractor, the network can continue jumping, or latching, from attractor to attractor, driven by adaptation effects. The occurrence and duration of latching dynamics is found through simulations to depend critically on the strength of local attractor states, expressed in the Potts model by a parameter w. Here we describe with simulations and then analytically the boundaries between distinct phases of no latching, of transient and sustained latching, deriving a phase diagram in the plane w-T, where T parametrizes thermal noise effects. Implications for real cortical dynamics are briefly reviewed in the conclusions.

nlin.AO↗

arXiv2009

How informative are spatial CA3 representations established by the dentate gyrus?

In the mammalian hippocampus, the dentate gyrus (DG) is characterized by sparse and powerful unidirectional projections to CA3 pyramidal cells, the so-called mossy fibers. Mossy fiber synapses appear to duplicate, in terms of the information they convey, what CA3 cells already receive from entorhinal cortex layer II cells, which project both to the dentate gyrus and to CA3. Computational models of episodic memory have hypothesized that the function of the mossy fibers is to enforce a new, well separated pattern of activity onto CA3 cells, to represent a new memory, prevailing over the interference produced by the traces of older memories already stored on CA3 recurrent collateral connections. Can this hypothesis apply also to spatial representations, as described by recent neurophysiological recordings in rats? To address this issue quantitatively, we estimate the amount of information DG can impart on a new CA3 pattern of spatial activity, using both mathematical analysis and computer simulations of a simplified model. We confirm that, also in the spatial case, the observed sparse connectivity and level of activity are most appropriate for driving memory storage and not to initiate retrieval. Surprisingly, the model also indicates that even when DG codes just for space, much of the information it passes on to CA3 acquires a non-spatial and episodic character, akin to that of a random number generator. It is suggested that further hippocampal processing is required to make full spatial use of DG inputs.

q-bio.NC↗

arXiv2007

Uninformative memories will prevail: the storage of correlated representations and its consequences

Autoassociative networks were proposed in the 80's as simplified models of memory function in the brain, using recurrent connectivity with hebbian plasticity to store patterns of neural activity that can be later recalled. This type of computation has been suggested to take place in the CA3 region of the hippocampus and at several levels in the cortex. One of the weaknesses of these models is their apparent inability to store correlated patterns of activity. We show, however, that a small and biologically plausible modification in the `learning rule' (associating to each neuron a plasticity threshold that reflects its popularity) enables the network to handle correlations. We study the stability properties of the resulting memories (in terms of their resistance to the damage of neurons or synapses), finding a novel property of autoassociative networks: not all memories are equally robust, and the most informative are also the most sensitive to damage. We relate these results to category-specific effects in semantic memory patients, where concepts related to `non-living things' are usually more resistant to brain damage than those related to `living things', a phenomenon suspected to be rooted in the correlation between representations of concepts in the cortex.

cond-mat.dis-nn↗

arXiv2005

The storage capacity of Potts models for semantic memory retrieval

We introduce and analyze a minimal network model of semantic memory in the human brain. The model is a global associative memory structured as a collection of N local modules, each coding a feature, which can take S possible values, with a global sparseness a (the average fraction of features describing a concept). We show that, under optimal conditions, the number c of modules connected on average to a module can range widely between very sparse connectivity (c/N -> 0) and full connectivity (c = N), maintaining a global network storage capacity (the maximum number p of stored and retrievable concepts) that scales like c*S^2/a, with logarithmic corrections consistent with the constraint that each synapse may store up to a fraction of a bit.

cond-mat.dis-nn↗

arXiv2005

Localized activity profiles and storage capacity of rate-based autoassociative networks

We study analytically the effect of metrically structured connectivity on the behavior of autoassociative networks. We focus on three simple rate-based model neurons: threshold-linear, binary or smoothly saturating units. For a connectivity which is short range enough the threshold-linear network shows localized retrieval states. The saturating and binary models also exhibit spatially modulated retrieval states if the highest activity level that they can achieve is above the maximum activity of the units in the stored patterns. In the zero quenched noise limit, we derive an analytical formula for the critical value of the connectivity width below which one observes spatially non-uniform retrieval states. Localization reduces storage capacity, but only by a factor of 2~3. The approach that we present here is generic in the sense that there are no specific assumptions on the single unit input-output function nor on the exact connectivity structure.

cond-mat.dis-nn↗

arXiv2004

An associative network with spatially organized connectivity

We investigate the properties of an autoassociative network of threshold-linear units whose synaptic connectivity is spatially structured and asymmetric. Since the methods of equilibrium statistical mechanics cannot be applied to such a network due to the lack of a Hamiltonian, we approach the problem through a signal-to-noise analysis, that we adapt to spatially organized networks. The conditions are analyzed for the appearance of stable, spatially non-uniform profiles of activity with large overlaps with one of the stored patterns. It is also shown, with simulations and analytic results, that the storage capacity does not decrease much when the connectivity of the network becomes short range. In addition, the method used here enables us to calculate exactly the storage capacity of a randomly connected network with arbitrary degree of dilution.

cond-mat.dis-nn↗

arXiv2003

Disappearance of Spurious States in Analog Associative Memories

We show that symmetric n-mixture states, when they exist, are almost never stable in autoassociative networks with threshold-linear units. Only with a binary coding scheme we could find a limited region of the parameter space in which either 2-mixtures or 3-mixtures are stable attractors of the dynamics.

cond-mat.dis-nn↗

arXiv2002

Replica symmetric evaluation of the information transfer in a two-layer network in presence of continuous and discrete stimuli

In a previous report we have evaluated analytically the mutual information between the firing rates of N independent units and a set of continuous+discrete stimuli, for finite N and in the limit of large noise. Here, we extend the analysis to the case of two interconnected populations, where input units are linked to output ones via gaussian weights and a threshold linear transfer function. We evaluate the information carried by M output units about continuous +discrete correlates by means of saddle point equations under the assumption of replica symmetry. Within this limitation, we analyze the dependence of the information on the ratio M/N, on the selectivity of the input units and on the level of the output noise. We show analytically, and confirm numerically, that in the limit of a linear transfer function and of a small ratio between output and input noise, the output information approaches asymptotically the information carried in input. Finally, we show that the information loss in output does not depend much on the structure of the stimulus, but mainly on the position of the threshold nonlinearity, and on the ratio between input and output noise.

cond-mat.dis-nn↗

arXiv2002

Representational capacity of a set of independent neurons

The capacity with which a system of independent neuron-like units represents a given set of stimuli is studied by calculating the mutual information between the stimuli and the neural responses. Both discrete noiseless and continuous noisy neurons are analyzed. In both cases, the information grows monotonically with the number of neurons considered. Under the assumption that neurons are independent, the mutual information rises linearly from zero, and approaches exponentially its maximum value. We find the dependence of the initial slope on the number of stimuli and on the sparseness of the representation.

cond-mat.dis-nn↗