arXiv subjects
Publications and source records attributed to Katherine Morrison.
This is an extended version of our survey article, "Graph rules for recurrent neural network dynamics," to appear in the April 2023 edition of the Notices of the AMS. It includes additional results, derivations, figures, references, and a set of open questions.
This is a brief overview of results from [arXiv:2107.10244, ref 11], on network architectures that produce sequential dynamics in a special family of inhibition-dominated neural networks. It was written for SIAM DSWeb.
Combinatorial threshold-linear networks (CTLNs) are a special class of inhibition-dominated TLNs defined from directed graphs. Like more general TLNs, they display a wide variety of nonlinear dynamics including multistability, limit cycles, quasiperiodic attractors, and chaos. In prior work, we have developed a detailed mathematical theory relating stable and unstable fixed points of CTLNs to graph-theoretic properties of the underlying network. Here we find that a special type of fixed points, corresponding to core motifs, are predictive of both static and dynamic attractors. Moreover, the attractors can be found by choosing initial conditions that are small perturbations of these fixed points. This motivates us to hypothesize that dynamic attractors of a network correspond to unstable fixed points supported on core motifs. We tested this hypothesis on a large family of directed graphs of size $n=5$, and found remarkable agreement. Furthermore, we discovered that core motifs with similar embeddings give rise to nearly identical attractors. This allowed us to classify attractors based on structurally-defined graph families. Our results suggest that graphical properties of the connectivity can be used to predict a network's complex repertoire of nonlinear dynamics.
Sequences of neural activity arise in many brain areas, including cortex, hippocampus, and central pattern generator circuits that underlie rhythmic behaviors like locomotion. While network architectures supporting sequence generation vary considerably, a common feature is an abundance of inhibition. In this work, we focus on architectures that support sequential activity in recurrently connected networks with inhibition-dominated dynamics. Specifically, we study emergent sequences in a special family of threshold-linear networks, called combinatorial threshold-linear networks (CTLNs), whose connectivity matrices are defined from directed graphs. Such networks naturally give rise to an abundance of sequences whose dynamics are tightly connected to the underlying graph. We find that architectures based on generalizations of cycle graphs produce limit cycle attractors that can be activated to generate transient or persistent (repeating) sequences. Each architecture type gives rise to an infinite family of graphs that can be built from arbitrary component subgraphs. Moreover, we prove a number of graph rules for the corresponding CTLNs in each family. The graph rules allow us to strongly constrain, and in some cases fully determine, the fixed points of the network in terms of the fixed points of the component subnetworks. Finally, we also show how the structure of certain architectures gives insight into the sequential dynamics of the corresponding attractor.
Nonlinear network dynamics are notoriously difficult to understand. Here we study a class of recurrent neural networks called combinatorial threshold-linear networks (CTLNs) whose dynamics are determined by the structure of a directed graph. They are a special case of TLNs, a popular framework for modeling neural activity in computational neuroscience. In prior work, CTLNs were found to be surprisingly tractable mathematically. For small networks, the fixed points of the network dynamics can often be completely determined via a series of graph rules that can be applied directly to the underlying graph. For larger networks, it remains a challenge to understand how the global structure of the network interacts with local properties. In this work, we propose a method of covering graphs of CTLNs with a set of smaller directional graphs that reflect the local flow of activity. While directional graphs may or may not have a feedforward architecture, their fixed point structure is indicative of feedforward dynamics. The combinatorial structure of the graph cover is captured by the nerve of the cover. The nerve is a smaller, simpler graph that is more amenable to graphical analysis. We present three nerve theorems that provide strong constraints on the fixed points of the underlying network from the structure of the nerve. We then illustrate the power of these theorems with some examples. Remarkably, we find that the nerve not only constrains the fixed points of CTLNs, but also gives insight into the transient and asymptotic dynamics. This is because the flow of activity in the network tends to follow the edges of the nerve.
The architecture of a neural network constrains the potential dynamics that can emerge. Some architectures may only allow for a single dynamic regime, while others display a great deal of flexibility with qualitatively different dynamics that can be reached by modulating connection strengths. In this work, we develop novel mathematical techniques to study the dynamic constraints imposed by different network architectures in the context of competitive threshold-linear networks (TLNs). Any given TLN is naturally characterized by a hyperplane arrangement in $\mathbb{R}^n$, and the combinatorial properties of this arrangement determine the pattern of fixed points of the dynamics. This observation enables us to recast the question of network flexibility in the language of oriented matroids, allowing us to employ tools and results from this theory in order to characterize the different dynamic regimes a given architecture can support. In particular, fixed points of a TLN correspond to cocircuits of an associated oriented matroid; and mutations of the matroid correspond to bifurcations in the collection of fixed points. As an application, we provide a complete characterization of all possible sets of fixed points that can arise in networks through size $n=3$, together with descriptions of how to modulate synaptic strengths of the network in order to access the different dynamic regimes. These results provide a framework for studying the possible computational roles of various motifs observed in real neural networks.
Combinatorial threshold-linear networks (CTLNs) are a special class of recurrent neural networks whose dynamics are tightly controlled by an underlying directed graph. Recurrent networks have long been used as models for associative memory and pattern completion, with stable fixed points playing the role of stored memory patterns in the network. In prior work, we showed that target-free cliques of the graph correspond to stable fixed points of the dynamics, and we conjectured that these are the only stable fixed points possible. In this paper, we prove that the conjecture holds in a variety of special cases, including for networks with very strong inhibition and graphs of size $n \leq 4$. We also provide further evidence for the conjecture by showing that sparse graphs and graphs that are nearly cliques can never support stable fixed points. Finally, we translate some results from extremal combinatorics to obtain an upper bound on the number of stable fixed points of CTLNs in cases where the conjecture holds.
To any inhibition-dominated threshold-linear network (TLN) we can associate a directed graph that captures the pattern of strong and weak inhibition between neurons. Robust motifs are graphs for which the structure of fixed points in the network is independent of the choice of connectivity matrix $W$, and whose dynamics are thus greatly constrained. This makes them ideal building blocks for constructing larger networks whose behaviors are robust to changes in connection strengths. In contrast, flexible motifs correspond to networks with multiple dynamic regimes. In this work, we give a purely graphical characterization of both flexible and robust motifs of any size. We find that all but a few robust motifs fall into two infinite families of graphs with simple feedforward architectures.
Threshold-linear networks (TLNs) are models of neural networks that consist of simple, perceptron-like neurons and exhibit nonlinear dynamics that are determined by the network's connectivity. The fixed points of a TLN, including both stable and unstable equilibria, play a critical role in shaping its emergent dynamics. In this work, we provide two novel characterizations for the set of fixed points of a competitive TLN: the first is in terms of a simple sign condition, while the second relies on the concept of domination. We apply these results to a special family of TLNs, called combinatorial threshold-linear networks (CTLNs), whose connectivity matrices are defined from directed graphs. This leads us to prove a series of graph rules that enable one to determine fixed points of a CTLN by analyzing the underlying graph. Additionally, we study larger networks composed of smaller "building block" subnetworks, and prove several theorems relating the fixed points of the full network to those of its components. Our results provide the foundation for a kind of "graphical calculus" to infer features of the dynamics from a network's connectivity.
A convex code is a binary code generated by the pattern of intersections of a collection of open convex sets in some Euclidean space. Convex codes are relevant to neuroscience as they arise from the activity of neurons that have convex receptive fields. In this paper, we use algebraic methods to determine if a code is convex. Specifically, we use the neural ideal of a code, which is a generalization of the Stanley-Reisner ideal. Using the neural ideal together with its standard generating set, the canonical form, we provide algebraic signatures of certain families of codes that are non-convex. We connect these signatures to the precise conditions on the arrangement of sets that prevent the codes from being convex. Finally, we also provide algebraic signatures for some families of codes that are convex, including the class of intersection-complete codes. These results allow us to detect convexity and non-convexity in a variety of situations, and point to some interesting open questions.
Neural network models in neuroscience allow one to study how the connections between neurons shape the activity of neural circuits in the brain. In this chapter, we study Combinatorial Threshold-Linear Networks (CTLNs) in order to understand how the pattern of connectivity, as encoded by a directed graph, shapes the emergent nonlinear dynamics of the corresponding network. Important aspects of these dynamics are controlled by the stable and unstable fixed points of the network, and we show how these fixed points can be determined via graph-based rules. We also present an algorithm for predicting sequences of neural activation from the underlying directed graph, and examine the effect of graph symmetries on a network's set of attractors.
Neural codes allow the brain to represent, process, and store information about the world. Combinatorial codes, comprised of binary patterns of neural activity, encode information via the collective behavior of populations of neurons. A code is called convex if its codewords correspond to regions defined by an arrangement of convex open sets in Euclidean space. Convex codes have been observed experimentally in many brain areas, including sensory cortices and the hippocampus, where neurons exhibit convex receptive fields. What makes a neural code convex? That is, how can we tell from the intrinsic structure of a code if there exists a corresponding arrangement of convex open sets? In this work, we provide a complete characterization of local obstructions to convexity. This motivates us to define max intersection-complete codes, a family guaranteed to have no local obstructions. We then show how our characterization enables one to use free resolutions of Stanley-Reisner ideals in order to detect violations of convexity. Taken together, these results provide a significant advance in understanding the intrinsic combinatorial properties of convex codes.
Threshold-linear networks consist of simple units interacting in the presence of a threshold nonlinearity. Competitive threshold-linear networks have long been known to exhibit multistability, where the activity of the network settles into one of potentially many steady states. In this work, we find conditions that guarantee the absence of steady states, while maintaining bounded activity. These conditions lead us to define a combinatorial family of competitive threshold-linear networks, parametrized by a simple directed graph. By exploring this family, we discover that threshold-linear networks are capable of displaying a surprisingly rich variety of nonlinear dynamics, including limit cycles, quasiperiodic attractors, and chaos. In particular, several types of nonlinear behaviors can co-exist in the same network. Our mathematical results also enable us to engineer networks with multiple dynamic patterns. Taken together, these theoretical and computational findings suggest that threshold-linear networks may be a valuable tool for understanding the relationship between network connectivity and emergent dynamics. The new Matlab package CTLN Basic 2.0 can be used to reproduce the simulations in Figures 1-13. The package is available at https://github.com/nebneuron/CTLN-Basic-2.0
Threshold-linear networks are a common class of firing rate models that describe recurrent interactions among neurons. Unlike their linear counterparts, these networks generically possess multiple stable fixed points (steady states), making them viable candidates for memory encoding and retrieval. In this work, we characterize stable fixed points of general threshold-linear networks with constant external drive, and discover constraints on the co-existence of fixed points involving different subsets of active neurons. In the case of symmetric networks, we prove the following antichain property: if a set of neurons $τ$ is the support of a stable fixed point, then no proper subset or superset of $τ$ can support a stable fixed point. Symmetric threshold-linear networks thus appear to be well suited for pattern completion, since the dynamics are guaranteed not to get "stuck" in a subset or superset of a stored pattern. We also show that for any graph G, we can construct a network whose stable fixed points correspond precisely to the maximal cliques of G. As an application, we design network decoders for place field codes, and demonstrate their efficacy for error correction and pattern completion. The proofs of our main results build on the theory of permitted sets in threshold-linear networks, including recently-developed connections to classical distance geometry.
As a result of their applications in network coding, space-time coding, and coding for criss-cross errors, matrix codes have garnered significant attention; in various contexts, these codes have also been termed rank-metric codes, space-time codes over finite fields, and array codes. We focus on characterizing matrix codes that are both efficient (have high rate) and effective at error correction (have high minimum rank-distance). It is well known that the inherent trade-off between dimension and minimum distance for a matrix code is reversed for its dual code; specifically, if a matrix code has high dimension and low minimum distance, then its dual code will have low dimension and high minimum distance. With an aim towards finding codes with a perfectly balanced trade-off, we study self-dual matrix codes. In this work, we develop a framework based on double cosets of the matrix-equivalence maps to provide a complete classification of the equivalence classes of self-dual matrix codes, and we employ this method to enumerate the equivalence classes of these codes for small parameters.
Cyclic orbit codes are constant dimension subspace codes that arise as the orbit of a cyclic subgroup of the general linear group acting on subspaces in the given ambient space. With the aid of the largest subfield over which the given subspace is a vector space, the cardinality of the orbit code can be determined, and estimates for its distance can be found. This subfield is closely related to the stabilizer of the generating subspace. Finally, with a linkage construction larger, and longer, constant dimension codes can be derived from cyclic orbit codes without compromising the distance.
For a growing number of applications such as cellular, peer-to-peer, and sensor networks, efficient error-free transmission of data through a network is essential. Toward this end, Kötter and Kschischang propose the use of subspace codes to provide error correction in the network coding context. The primary construction for subspace codes is the lifting of rank-metric or matrix codes, a process that preserves the structural and distance properties of the underlying code. Thus, to characterize the structure and error-correcting capability of these subspace codes, it is valuable to perform such a characterization of the underlying rank-metric and matrix codes. This paper lays a foundation for this analysis through a framework for classifying rank-metric and matrix codes based on their structure and distance properties. To enable this classification, we extend work by Berger on equivalence for rank-metric codes to define a notion of equivalence for matrix codes, and we characterize the group structure of the collection of maps that preserve such equivalence. We then compare the notions of equivalence for these two related types of codes and show that matrix equivalence is strictly more general than rank-metric equivalence. Finally, we characterize the set of equivalence maps that fix the prominent class of rank-metric codes known as Gabidulin codes. In particular, we give a complete characterization of the rank-metric automorphism group of Gabidulin codes, correcting work by Berger, and give a partial characterization of the matrix-automorphism group of the expanded matrix codes that arise from Gabidulin codes.
Shannon's seminal 1948 work gave rise to two distinct areas of research: information theory and mathematical coding theory. While information theory has had a strong influence on theoretical neuroscience, ideas from mathematical coding theory have received considerably less attention. Here we take a new look at combinatorial neural codes from a mathematical coding theory perspective, examining the error correction capabilities of familiar receptive field codes (RF codes). We find, perhaps surprisingly, that the high levels of redundancy present in these codes does not support accurate error correction, although the error-correcting performance of RF codes "catches up" to that of random comparison codes when a small tolerance to error is introduced. On the other hand, RF codes are good at reflecting distances between represented stimuli, while the random comparison codes are not. We suggest that a compromise in error-correcting capability may be a necessary price to pay for a neural code whose structure serves not only error correction, but must also reflect relationships between stimuli.