arXiv subjects

Dani S. Bassett

Publications and source records attributed to Dani S. Bassett.

At least 37 records · Page 2Linked to original sources

arXiv2024

Towards principles of brain network organization and function

The brain is immensely complex, with diverse components and dynamic interactions building upon one another to orchestrate a wide range of functions and behaviors. Understanding patterns of these complex interactions and how they are coordinated to support collective neural activity and function is critical for parsing human and animal behavior, treating mental illness, and developing artificial intelligence. Rapid experimental advances in imaging, recording, and perturbing neural systems across various species now provide opportunities and challenges to distill underlying principles of brain organization and function. Here, we take stock of recent progresses and review methods used in the statistical analysis of brain networks, drawing from fields of statistical physics, network theory and information theory. Our discussion is organized by scale, starting with models of individual neurons and extending to large-scale networks mapped across brain regions. We then examine the organizing principles and constraints that shape the biological structure and function of neural circuits. Finally, we describe current opportunities aimed at improving models in light of recent developments and at bridging across scales to contribute to a better understanding of brain networks.

q-bio.NC↗

arXiv2024

Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck

Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution with minimal redundancy. Finding an optimal measurement is challenging, and has generally required intimate knowledge of the dynamics in the few cases where it has been done. We establish an equivalence between a perfect measurement and a variant of the information bottleneck. As a consequence, we can employ machine learning to optimize measurement processes that efficiently extract information from trajectory data. We obtain approximately optimal measurements for multiple chaotic maps and lay the necessary groundwork for efficient information extraction from general time series.

cs.LG↗

arXiv2024

Information decomposition in complex systems via machine learning

One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of linking variation across scales of a system due to its independence of functional relationship between observables. However, characterizing the manner in which information is distributed across a set of observables is computationally challenging and generally infeasible beyond a handful of measurements. Here we propose a practical and general methodology that uses machine learning to decompose the information contained in a set of measurements by jointly optimizing a lossy compression of each measurement. Guided by the distributed information bottleneck as a learning objective, the information decomposition identifies the variation in the measurements of the system state most relevant to specified macroscale behavior. We focus our analysis on two paradigmatic complex systems: a Boolean circuit and an amorphous material undergoing plastic deformation. In both examples, the large amount of entropy of the system state is decomposed, bit by bit, in terms of what is most related to macroscale behavior. The identification of meaningful variation in data, with the full generality brought by information theory, is made practical for studying the connection between micro- and macroscale structure in complex systems.

cs.LG↗

arXiv2023

Intrinsically motivated graph exploration using network theories of human curiosity

Intrinsically motivated exploration has proven useful for reinforcement learning, even without additional extrinsic rewards. When the environment is naturally represented as a graph, how to guide exploration best remains an open question. In this work, we propose a novel approach for exploring graph-structured data motivated by two theories of human curiosity: the information gap theory and the compression progress theory. The theories view curiosity as an intrinsic motivation to optimize for topological features of subgraphs induced by nodes visited in the environment. We use these proposed features as rewards for graph neural-network-based reinforcement learning. On multiple classes of synthetically generated graphs, we find that trained agents generalize to longer exploratory walks and larger environments than are seen during training. Our method computes more efficiently than the greedy evaluation of the relevant topological properties. The proposed intrinsic motivations bear particular relevance for recommender systems. We demonstrate that next-node recommendations considering curiosity are more predictive of human choices than PageRank centrality in several real-world graph environments.

cs.LG↗

arXiv2023

Human Learning of Hierarchical Graphs

Humans are constantly exposed to sequences of events in the environment. Those sequences frequently evince statistical regularities, such as the probabilities with which one event transitions to another. Collectively, inter-event transition probabilities can be modeled as a graph or network. Many real-world networks are organized hierarchically and understanding how humans learn these networks is an ongoing aim of current investigations. While much is known about how humans learn basic transition graph topology, whether and to what degree humans can learn hierarchical structures in such graphs remains unknown. We investigate how humans learn hierarchical graphs of the Sierpiński family using computer simulations and behavioral laboratory experiments. We probe the mental estimates of transition probabilities via the surprisal effect: a phenomenon in which humans react more slowly to less expected transitions, such as those between communities or modules in the network. Using mean-field predictions and numerical simulations, we show that surprisal effects are stronger for finer-level than coarser-level hierarchical transitions. Surprisal effects at coarser levels of the hierarchy are difficult to detect for limited learning times or in small samples. Using a serial response experiment with human participants (n=$100$), we replicate our predictions by detecting a surprisal effect at the finer-level of the hierarchy but not at the coarser-level of the hierarchy. To further explain our findings, we evaluate the presence of a trade-off in learning, whereby humans who learned the finer-level of the hierarchy better tended to learn the coarser-level worse, and vice versa. Our study elucidates the processes by which humans learn hierarchical sequential events. Our work charts a road map for future investigation of the neural underpinnings and behavioral manifestations of graph learning.

q-bio.NC↗

arXiv2023

Information content of note transitions in the music of J. S. Bach

Music has a complex structure that expresses emotion and conveys information. Humans process that information through imperfect cognitive instruments that produce a gestalt, smeared version of reality. How can we quantify the information contained in a piece of music? Further, what is the information inferred by a human, and how does that relate to (and differ from) the true structure of a piece? To tackle these questions quantitatively, we present a framework to study the information conveyed in a musical piece by constructing and analyzing networks formed by notes (nodes) and their transitions (edges). Using this framework, we analyze music composed by J. S. Bach through the lens of network science and information theory. Regarded as one of the greatest composers in the Western music tradition, Bach's work is highly mathematically structured and spans a wide range of compositional forms, such as fugues and choral pieces. Conceptualizing each composition as a network of note transitions, we quantify the information contained in each piece and find that different kinds of compositions can be grouped together according to their information content and network structure. Moreover, we find that the music networks communicate large amounts of information while maintaining small deviations of the inferred network from the true network, suggesting that they are structured for efficient communication of information. We probe the network structures that enable this rapid and efficient communication of information--namely, high heterogeneity and strong clustering. Taken together, our findings shed new light on the information and network properties of Bach's compositions. More generally, our framework serves as a stepping stone for exploring musical complexities, creativity and the structure of information in a range of complex systems.

physics.soc-ph↗

arXiv2023

The information content of brain states is explained by structural constraints on state energetics

Signal propagation along the structural connectome of the brain induces changes in the patterns of activity. These activity patterns define global brain states and contain information in accordance with their expected probability of occurrence. The structural connectome, in conjunction with the dynamics, determines the set of possible brain states and constrains the transition between accessible states. Yet, precisely how these structural constraints on state-transitions relate to their information content remains unexplored. To address this gap in knowledge, we defined the information content as a function of the activation distribution, where statistically rare values of activation correspond to high information content. With this numerical definition in hand, we studied the spatiotemporal distribution of information content in fMRI data from the Human Connectome Project during different tasks, and report four key findings. First, information content strongly depends on the cognitive task. Second, while information content shows similarities to other measures of brain activity, it is distinct from both Neurosynth maps and task contrast maps generated by a general linear model applied to the fMRI data. Third, the brain's structural wiring constrains the cost to control its state, where the cost to transition into high information content states is larger than that to transition into low information content states. Finally, all state transitions - especially those to high information content states - are less costly than expected from random network null models, thereby indicating the brain's marked efficiency. Taken together, our findings establish an explanatory link between the information contained in a brain state and the energetic cost of attaining that state, thereby laying important groundwork for our understanding of large-scale cognitive computations.

q-bio.NC↗

arXiv2022

Interpretability with full complexity by constraining feature information

Interpretability is a pressing issue for machine learning. Common approaches to interpretable machine learning constrain interactions between features of the input, rendering the effects of those features on a model's output comprehensible but at the expense of model complexity. We approach interpretability from a new angle: constrain the information about the features without restricting the complexity of the model. Borrowing from information theory, we use the Distributed Information Bottleneck to find optimal compressions of each feature that maximally preserve information about the output. The learned information allocation, by feature and by feature value, provides rich opportunities for interpretation, particularly in problems with many features and complex feature interactions. The central object of analysis is not a single trained model, but rather a spectrum of models serving as approximations that leverage variable amounts of information about the inputs. Information is allocated to features by their relevance to the output, thereby solving the problem of feature selection by constructing a learned continuum of feature inclusion-to-exclusion. The optimal compression of each feature -- at every stage of approximation -- allows fine-grained inspection of the distinctions among feature values that are most impactful for prediction. We develop a framework for extracting insight from the spectrum of approximate models and demonstrate its utility on a range of tabular datasets.

cs.LG↗

arXiv2022

Compression supports low-dimensional representations of behavior across neural circuits

Dimensionality reduction, a form of compression, can simplify representations of information to increase efficiency and reveal general patterns. Yet, this simplification also forfeits information, thereby reducing representational capacity. Hence, the brain may benefit from generating both compressed and uncompressed activity, and may do so in a heterogeneous manner across diverse neural circuits that represent low-level (sensory) or high-level (cognitive) stimuli. However, precisely how compression and representational capacity differ across the cortex remains unknown. Here we predict different levels of compression across regional circuits by using random walks on networks to model activity flow and to formulate rate-distortion functions, which are the basis of lossy compression. Using a large sample of youth ($n=1,040$), we test predictions in two ways: by measuring the dimensionality of spontaneous activity from sensorimotor to association cortex, and by assessing the representational capacity for 24 behaviors in neural circuits and 20 cognitive variables in recurrent neural networks. Our network theory of compression predicts the dimensionality of activity ($t=12.13, p<0.001$) and the representational capacity of biological ($r=0.53, p=0.016$) and artificial ($r=0.61, p<0.001$) networks. The model suggests how a basic form of compression is an emergent property of activity flow between distributed circuits that communicate with the rest of the network.

q-bio.NC↗

arXiv2022

Characterizing information loss in a chaotic double pendulum with the Information Bottleneck

A hallmark of chaotic dynamics is the loss of information with time. Although information loss is often expressed through a connection to Lyapunov exponents -- valid in the limit of high information about the system state -- this picture misses the rich spectrum of information decay across different levels of granularity. Here we show how machine learning presents new opportunities for the study of information loss in chaotic dynamics, with a double pendulum serving as a model system. We use the Information Bottleneck as a training objective for a neural network to extract information from the state of the system that is optimally predictive of the future state after a prescribed time horizon. We then decompose the optimally predictive information by distributing a bottleneck to each state variable, recovering the relative importance of the variables in determining future evolution. The framework we develop is broadly applicable to chaotic systems and pragmatic to apply, leveraging data and machine learning to monitor the limits of predictability and map out the loss of information.

cs.LG↗

arXiv2022

Controlling collective dynamical states of mesoscale brain networks with local perturbations

Oscillatory synchrony is hypothesized to support the flow of information between brain regions, with different phase-locked configurations enabling activation of different effective interactions. Along these lines, past work has proposed multistable phase-locking as a means for hardwired brain networks to flexibly support multiple functional patterns, without having to reconfigure their anatomical connections. Given the potential link between interareal communication and phase-locked states, it is thus important to understand how those states might be controlled to achieve rapid alteration of functional connectivity in interareal circuits. Here, we study functional state control in small networks of coupled neural masses that display collective multistability under determinstic conditions, and that display more biologically-realistic irregular oscillations and transient phase-locking when conditions are stochastic. In particular, we investigate the global responses of these mesoscale circuits to external signals that target only a single subunit. Focusing mainly on the more realistic scenario wherein network dynamics are stochastic, we identify conditions under which local inputs (i) can trigger fast transitions to topologically distinct functional connectivity motifs that are temporarily stable ("state switching"), (ii) can smoothly adjust the spatial pattern of phase-relations for a particular set of lead-lag relationships ("state morphing"), and (iii) fail to regulate global phase-locking states. In total, our results add to a growing literature highlighting that the modulation of multistable, interareal coherence patterns could provide a basis for flexible brain network operation.

q-bio.NC↗

arXiv2022

A structurally informed model for modulating functional connectivity

Functional connectivity (FC) between brain regions tracks symptom severity in many neuropsychiatric disorders. Transcranial magnetic stimulation (TMS) directly alters regional activity and indirectly alters FC. Predicting how FC will change following TMS is difficult, but would allow novel therapies that target FC to improve symptoms. We address this challenge by proposing a predictive model that explains how TMS-induced activation can change the strength of FC. Here, we focus on the FC of the frontoparietal (FPS) and default mode (DMS) systems given the importance of their FC in executive function and affective disorders. We fit this model to neuroimaging data in 29 individuals who received TMS to the frontal cortex and evaluated the FC between the FPS and DMS. For each individual, we measured the TMS-induced change in FC between the FPS and DMS (the FC network), and the structural coupling between the stimulated area and the FPS and DMS (the structural context network (SCN)). We find that TMS-induced FC changes are best predicted when the model accounts for white matter fibers from the stimulated area to the two systems. We find that the correlation between these two networks (structure-function coupling) - and therefore the predictability of the TMS-induced modulation - was highest when the SCN contained a dense core of intraconnected regions, indicating that the stimulated area had ample access to an anatomical module. Further, we found that when the core of the SCN overlapped with the FPS and DMS, we observed the greatest change in the strength of their FC. Broadly, our findings explain how the structural connectivity of a stimulated region modulates TMS-induced changes in the brain's functional network. Efforts to account for such structural connections could improve predictions of TMS response, further informing the development of TMS protocols for clinical translation.

q-bio.NC↗

arXiv2022

Learning Dynamic Graphs, Too Slow

The structure of knowledge is commonly described as a network of key concepts and semantic relations between them. A learner of a particular domain can discover this network by navigating the nodes and edges presented by instructional material, such as a textbook, workbook, or other text. While over a long temporal period such exploration processes are certain to discover the whole connected network, little is known about how the learning is affected by the dual pressures of finite study time and human mental errors. Here we model the learning of linear algebra textbooks with finite length random walks over the corresponding semantic networks. We show that if a learner does not keep up with the pace of material presentation, the learning can be an order of magnitude worse than it is in the asymptotic limit. Further, we find that this loss is compounded by three types of mental errors: forgetting, shuffling, and reinforcement. Broadly, our study informs the design of teaching materials from both structural and temporal perspectives.

cond-mat.stat-mech↗

arXiv2022

Synchronization of coupled Kuramoto oscillators competing for resources

Populations of oscillators are present throughout nature. Very often synchronization is observed in such populations if they are allowed to interact. A paradigmatic model for the study of such phenomena has been the Kuramoto model. However, considering real oscillations are rarely isochronous as a function of energy, it is natural to extend the model by allowing the natural frequencies to vary as a function of some dynamical resource supply. Beyond just accounting for a dynamical supply of resources, however, competition over a \emph{shared} resource supply is important in a variety of biological systems. In neuronal systems, for example, resource competition enables the study of neural activity via fMRI. It is reasonable to expect that this dynamical resource allocation should have consequences for the synchronization behavior of the brain. This paper presents a modified Kuramoto dynamics which includes additional dynamical terms that provide a relatively simple model of resource competition among populations of Kuramoto oscillators. We design a mutlilayer system which highlights the impact of the competition dynamics, and we show that in this designed system, correlations can arise between the synchronization states of two populations of oscillators which share no phase-coupling edges. These correlations are interesting in light of the often observed variance between functional and structural connectivity measures in real systems. The model presented here then suggests that some of the observed discrepancy may be explained by the way in which the brain dynamically allocates resources to different regions according to demand. If true, models such as this one provide a theoretical framework for analyzing the differences between structural and functional measures, and possibly implicate dynamical resource allocation as an integral part of the neural computation process.

nlin.AO↗

arXiv2022

Modeling observed gender imbalances in academic citation practices

In multiple academic disciplines, having a perceived gender of `woman' is associated with a lower than expected rate of citations. In some fields, that disparity is driven primarily by the citations of men and is increasing over time despite increasing diversification of the profession. It is likely that complex social interactions and individual ideologies shape these disparities. Computational models of select factors that reproduce empirical observations can help us understand some of the minimal driving forces behind these complex phenomena and therefore aid in their mitigation. Here, we present a simple agent-based model of citation practices within academia, in which academics generate citations based on three factors: their estimate of the collaborative network of the field, how they sample that estimate, and how open they are to learning about their field from other academics. We show that increasing homophily -- or the tendency of people to interact with others more like themselves -- in these three domains is sufficient to reproduce observed biases in citation practices. We find that homophily in sampling an estimate of the field influences total citation rates, and openness to learning from new and unfamiliar authors influences the change in those citations over time. We next model a real-world intervention -- the citation diversity statement -- which has the potential to influence both of these parameters. We determine a parameterization of our model that matches the citation practices of academics who use the citation diversity statement. This parameterization paired with an openness to learning from many new authors can result in citation practices that are equitable and stable over time. Ultimately, our work underscores the importance of homophily in shaping citation practices and provides evidence that specific actions may mitigate biased citation practices in academia.

cs.DL↗

arXiv2022

The Distributed Information Bottleneck reveals the explanatory structure of complex systems

The fruits of science are relationships made comprehensible, often by way of approximation. While deep learning is an extremely powerful way to find relationships in data, its use in science has been hindered by the difficulty of understanding the learned relationships. The Information Bottleneck (IB) is an information theoretic framework for understanding a relationship between an input and an output in terms of a trade-off between the fidelity and complexity of approximations to the relationship. Here we show that a crucial modification -- distributing bottlenecks across multiple components of the input -- opens fundamentally new avenues for interpretable deep learning in science. The Distributed Information Bottleneck throttles the downstream complexity of interactions between the components of the input, deconstructing a relationship into meaningful approximations found through deep learning without requiring custom-made datasets or neural network architectures. Applied to a complex system, the approximations illuminate aspects of the system's nature by restricting -- and monitoring -- the information about different components incorporated into the approximation. We demonstrate the Distributed IB's explanatory utility in systems drawn from applied mathematics and condensed matter physics. In the former, we deconstruct a Boolean circuit into approximations that isolate the most informative subsets of input components without requiring exhaustive search. In the latter, we localize information about future plastic rearrangement in the static structure of a sheared glass, and find the information to be more or less diffuse depending on the system's preparation. By way of a principled scheme of approximations, the Distributed IB brings much-needed interpretability to deep learning and enables unprecedented analysis of information flow through a system.

cs.LG↗

arXiv2022

Curiosity as filling, compressing, and reconfiguring knowledge networks

Due to the significant role that curiosity plays in our lives, several theoretical constructs, such as the information gap theory and compression progress theory, have sought to explain how we engage in its practice. According to the former, curiosity is the drive to acquire information that is missing from our understanding of the world. According to the latter, curiosity is the drive to construct an increasingly parsimonious mental model of the world. To complement the densification processes inherent to these theories, we propose the conformational change theory, wherein we posit that curiosity results in mental models with marked conceptual flexibility. We formalize curiosity as the process of building a growing knowledge network to quantitatively investigate information gap theory, compression progress theory, and the conformational change theory of curiosity. In knowledge networks, gaps can be identified as topological cavities, compression progress can be quantified using network compressibility, and flexibility can be measured as the number of conformational degrees of freedom. We leverage data acquired from the online encyclopedia Wikipedia to determine the degree to which each theory explains the growth of knowledge networks built by individuals and by collectives. Our findings lend support to a pluralistic view of curiosity, wherein intrinsically motivated information acquisition fills knowledge gaps and simultaneously leads to increasingly compressible and flexible knowledge networks. Across individuals and collectives, we determine the contexts in which each theoretical account may be explanatory, thereby clarifying their complementary and distinct explanations of curiosity. Our findings offer a novel network theoretical perspective on intrinsically motivated information acquisition that may harmonize with or compel an expansion of the traditional taxonomy of curiosity.

q-bio.NC↗

arXiv2022

A Neural Programming Language for the Reservoir Computer

From logical reasoning to mental simulation, biological and artificial neural systems possess an incredible capacity for computation. Such neural computers offer a fundamentally novel computing paradigm by representing data continuously and processing information in a natively parallel and distributed manner. To harness this computation, prior work has developed extensive training techniques to understand existing neural networks. However, the lack of a concrete and low-level programming language for neural networks precludes us from taking full advantage of a neural computing framework. Here, we provide such a programming language using reservoir computing -- a simple recurrent neural network -- and close the gap between how we conceptualize and implement neural computers and silicon computers. By decomposing the reservoir's internal representation and dynamics into a symbolic basis of its inputs, we define a low-level neural machine code that we use to program the reservoir to solve complex equations and store chaotic dynamical systems as random access memory (dRAM). Using this representation, we provide a fully distributed neural implementation of software virtualization and logical circuits, and even program a playable game of pong inside of a reservoir computer. Taken together, we define a concrete, practical, and fully generalizable implementation of neural computation.

cond-mat.dis-nn↗