arXiv subjects

Thomas Kinfe

Publications and source records attributed to Thomas Kinfe.

10 recordsLinked to original sources

arXiv2026

Convergent Evolution in Algorithmic Space

In evolutionary biology, unrelated organisms can independently evolve similar structures when exposed to similar functional demands. Here we ask whether an analogous form of convergent evolution occurs during neural network training: do networks with different random initializations develop similar internal weight structures when trained on the same task? This question is technically nontrivial because hidden neurons can be arbitrarily permuted without changing the represented function, making direct matrix comparisons misleading. We introduce a matching-based framework for comparing multilayer perceptrons in structural weight space. Hidden neurons are first coarsely aligned using permutation-invariant features and then refined by iterative Hungarian matching. After alignment, networks are compared with structural distance metrics designed to emphasize task-relevant weight patterns. Applying this approach to ensembles of small MLPs trained on MNIST, Fashion-MNIST, and KMNIST, we find that networks trained on the same task remain closer to one another than to networks trained on different tasks. Thus, task-specific training appears to guide initially random networks toward distinct regions, or attractors, in structural network space. The earliest phase of training reveals an additional and unexpected phenomenon. Classification accuracy rises rapidly before the matched structural distances show strong task-specific separation, and before the global weight distribution visibly changes. Nevertheless, individual weight entries already begin to drift in a coordinated manner. This suggests that early learning may first operate through subtle, distributed adjustments that strongly affect function while leaving coarse network morphology almost unchanged. We treat this early morphogenesis as a first glimpse of a richer dynamical process that will be investigated in future work.

q-bio.NC↗

arXiv2026

Convergent Evolution in Neural Representation Space: Emergent Order in Deep Belief Networks

Deep Belief Networks (DBNs) learn hierarchical generative models without class supervision. Here, we ask whether this purely unsupervised process nevertheless organizes internal representations according to the unknown data classes. We analyze successive layers of DBNs trained on MNIST, Fashion-MNIST, and KMNIST using the Generalized Discrimination Value (GDV), supervised probes applied only after training, a reconstruction-based measure of abstraction distance, effective dimensionality, and free sample generation. Remarkably, class-specific clustering generally increases with depth across datasets and network widths, although no label information is available during DBN training. Control experiments show that this effect depends on the learned feature structure and cannot be explained by random transformations, weight marginals, dimensionality reduction, or sigmoid saturation. The first hidden layers also frequently make class identity more accessible to linear and nonlinear probes. With greater depth, representations become increasingly compact and prototype-like as neurons acquire correlated feature directions. At the same time, GDV and probe accuracy reveal complementary aspects of class structure: improved average clustering can coexist with reduced accessibility for a few difficult class pairs. These findings demonstrate that layer-wise generative learning can spontaneously uncover and progressively amplify class-related structure in unlabeled data.

q-bio.NC↗

arXiv2026

Surviving by Serving: Functional Relevance Drives Self-Organization in Complex Adaptive Systems

Complex adaptive systems often develop organized structures without centralized control. Yet the local mechanisms by which functional organization emerges and persists remain incompletely understood. Here we propose Surviving by Serving (SBS) as a general principle of self-organization: components persist as long as their outputs are utilized by other components, whereas prolonged non-utilization promotes adaptation and exploration. To investigate this idea, we introduce a minimal multi-agent model in which agents transform shared resources and receive only local feedback when their outputs are subsequently utilized elsewhere in the system. Despite the absence of global objectives, the system spontaneously self-organizes into functional interaction networks. We observe the emergence of stable transformation chains, core-periphery organization, and the generation of novel states that enable previously inaccessible target conditions to be reached. Remarkably, self-sustaining interaction networks can arise even without external selection pressures, creating a pre-adaptive search phase from which later functional solutions emerge. These findings suggest that functional utilization may provide a simple, substrate-independent mechanism for the emergence and stabilization of organized structure in complex adaptive systems.

q-bio.NC↗

arXiv2026

Word Class Representations Spontaneously Emerge from Successor Representations Trained on Natural Language

Language models are typically trained to predict the next token in a sequence. Here, we explore an alternative predictive principle from reinforcement learning: Successor Representations (SRs), which model the expected discounted distribution of future states rather than the immediate next state. We transfer this framework to natural language and train neural networks to predict future word distributions across multiple temporal horizons, thereby learning representations of long-range transition structure. We train a deep residual neural network on WikiText-103 (103 million tokens; 20,000-word vocabulary) and optimize successor representations as probability distributions using KL divergence. Without explicit linguistic supervision, structured language representations emerge spontaneously. After training, the learned space develops a clear geometric organization with respect to part-of-speech (POS) categories: nouns, verbs, and adjectives become separable and recoverable through unsupervised clustering. This organization depends systematically on predictive horizon, with short horizons producing the strongest syntactic structure and longer horizons increasingly integrating broader contextual and semantic information. At finer resolutions, additional interpretable lexical substructure emerges, revealing coherent subclasses within major word categories. These findings suggest that syntactic categories need not be explicitly encoded but may arise as a consequence of predictive sequence learning. To our knowledge, this work provides the first systematic application of successor representations to natural language and establishes a conceptual bridge between reinforcement learning, linguistics, and cognitive neuroscience.

cs.CL↗

arXiv2026

Are cortical microcircuits optimized for information flux? -- A simulation-based reverse engineering study

A sufficiently large information flux in recurrent neural networks, quantified by the mutual information between successive network states, is considered a prerequisite for rich information processing capabilities. This raises the question of whether biological neural networks, such as cortical microcolumns, may be structurally organized to enhance information flux. To investigate this possibility, we study a simplified model of the cortical layer 5 architecture, in which a densely and strongly interconnected core population is embedded within a larger supporting network. Surprisingly, we find that the embedding network exerts a pronounced flux-enhancing effect on the core dynamics. Systematic reverse-engineering analyses reveal that the embedding network provides two key contributions: first, it generates effective biases that shift core neurons into a higher-entropy operating regime; second, it supplies stochastic fluctuations that prevent the network from becoming trapped in simple fixed-point or oscillatory attractors through the mechanism of Recurrence Resonance. We further show that the information flux can be increased even beyond the biologically embedded case by applying individually optimized biases to the core neurons, and that these biases can emerge from a simple self-organization principle. Our findings are relevant both for the functional interpretation of biological neural circuits and for the design of artificial recurrent systems such as reservoir computers.

q-bio.NC↗

arXiv2026

Neuroscience-Inspired Analyses of Visual Interestingness in Multimodal Transformers

Human attention is the gateway to conscious perception, memory and decision-making. However, its role in modern transformer models remains largely unexplored. As these systems increasingly influence what people see, prefer and buy, the question arises as to whether they encode principles of human interest or merely exploit large-scale correlations. Addressing this issue is crucial for understanding cognition and ensuring the responsible use of AI in communication and marketing. In order to address this issue, the concept of visual interest was examined within the multimodal vision-language-model Qwen3-VL-8B, using a pre-defined Common Interestingness (CI) score derived from large-scale human engagement data on the photo-sharing platform Flickr. Here, we analyzed internal representations across vision and language components using methods from the neurosciences. Our analyses revealed that CI information is linearly decodable from final-layer embeddings, indicating that it is aligned with human-derived measures of visual interestingness. Dimensionality reduction and Generalized Discrimination Value (GDV) analyses demonstrate that CI-related hidden representations emerge in intermediate vision transformer layers and becomes progressively more distinguishable across language model layers. Concept vectors derived using geometric, probe, and Sparse Auto-Encoder based methods converge in higher layers, as confirmed by representational similarity analysis. This indicates a robust and structured encoding of visual interestingness without explicit supervision. Future work will seek to identify shared computational principles linking human brain dynamics and transformer architectures, with the ultimate goal of uncovering the organizing mechanisms that give rise to attention and interest in both biological and artificial systems.

cs.CV↗

arXiv2026

Structural and dynamical strategies to prevent runaway excitation in reservoir computing

Reservoirs, typically implemented as recurrent neural networks with fixed random connection weights, can be combined with a simple trained readout layer to perform a wide range of computational tasks. However, increasing the magnitude of reservoir connection weights to exploit nonlinear dynamics can cause the network to develop strong spontaneous activity that drives neurons into saturation, dramatically degrading performance. In this work, we investigate two distinct countermeasures against such runaway excitation. The first approach introduces a subtle non-homogeneous structure into the matrix of connection weigths $w_{ij}$, without altering the overall probability distribution $p(w)$. We identify several favorable structuring principles, such as creating a small subset of neurons with weaker-than-average input connections. Even if the rest of the reservoir falls into runaway saturating behavior, this weakly coupled subset remains in a mildly nonlinear regime whose dynamics can still be exploited by the readout layer. The second approach implements a form of automatic gain control, in which a dedicated control unit dynamically regulates the reservoir's average global activation toward an optimal setpoint. Although the control unit modulates the excitability of the reservoir only via a global gain factor, this mechanism substantially enlarges the dynamical regime favorable for computation and renders performance largely independent of the underlying connection statistics.

q-bio.NC↗

arXiv2026

Convergent Representations of Linguistic Constructions in Human and Artificial Neural Systems

Understanding how the brain processes linguistic constructions is a central challenge in cognitive neuroscience and linguistics. Recent computational studies show that artificial neural language models spontaneously develop differentiated representations of Argument Structure Constructions (ASCs), generating predictions about when and how construction-level information emerges during processing. The present study tests these predictions in human neural activity using electroencephalography (EEG). Ten native English speakers listened to 200 synthetically generated sentences across four construction types (transitive, ditransitive, caused-motion, resultative) while neural responses were recorded. Analyses using time-frequency methods, feature extraction, and machine learning classification revealed construction-specific neural signatures emerging primarily at sentence-final positions, where argument structure becomes fully disambiguated, and most prominently in the alpha band. Pairwise classification showed reliable differentiation, especially between ditransitive and resultative constructions, while other pairs overlapped. Crucially, the temporal emergence and similarity structure of these effects mirror patterns in recurrent and transformer-based language models, where constructional representations arise during integrative processing stages. These findings support the view that linguistic constructions are neurally encoded as distinct form-meaning mappings, in line with Construction Grammar, and suggest convergence between biological and artificial systems on similar representational solutions. More broadly, this convergence is consistent with the idea that learning systems discover stable regions within an underlying representational landscape - recently termed a Platonic representational space - that constrains the emergence of efficient linguistic abstractions.

q-bio.NC↗

arXiv2025

Illuminating the Black Box of Reservoir Computing

Reservoir computers, based on large recurrent neural networks with fixed random connections, are known to perform a wide range of information processing tasks. However, the nature of data transformations within the reservoir, the interplay of input matrix, reservoir, and readout layer, as well as the effect of varying design parameters remain poorly understood. In this study, we shift the focus from performance maximization to systematic simplification, aiming to identify the minimal computational ingredients required for different model tasks. We examine how many neurons, how much nonlinearity, and which connective structure is necessary and sufficient to perform certain tasks, considering also neurons with non-sigmoidal activation functions and networks with non-random connectivity. Surprisingly, we find non-trivial cases where the readout layer performs the bulk of the computation, with the reservoir merely providing weak nonlinearity and memory. Furthermore, design aspects often considered secondary, such as the structure of the input matrix, the steepness of activation functions, or the precise input/output timing, emerge as critical determinants of system performance in certain tasks.

cs.NE↗

arXiv2025

The Predictive Brain: Neural Correlates of Word Expectancy Align with Large Language Model Prediction Probabilities

Predictive coding theory suggests that the brain continuously anticipates upcoming words to optimize language processing, but the neural mechanisms remain unclear, particularly in naturalistic speech. Here, we simultaneously recorded EEG and MEG data from 29 participants while they listened to an audio book and assigned predictability scores to nouns using the BERT language model. Our results show that higher predictability is associated with reduced neural responses during word recognition, as reflected in lower N400 amplitudes, and with increased anticipatory activity before word onset. EEG data revealed increased pre-activation in left fronto-temporal regions, while MEG showed a tendency for greater sensorimotor engagement in response to low-predictability words, suggesting a possible motor-related component to linguistic anticipation. These findings provide new evidence that the brain dynamically integrates top-down predictions with bottom-up sensory input to facilitate language comprehension. To our knowledge, this is the first study to demonstrate these effects using naturalistic speech stimuli, bridging computational language models with neurophysiological data. Our findings provide novel insights for cognitive computational neuroscience, advancing the understanding of predictive processing in language and inspiring the development of neuroscience-inspired AI. Future research should explore the role of prediction and sensory precision in shaping neural responses and further refine models of language processing.

q-bio.NC↗