arXiv subjects

Tobias Kühn

Publications and source records attributed to Tobias Kühn.

12 recordsLinked to original sources

arXiv2026

Diagrammatics of free energies with fixed variance for high-dimensional data

Systems with many interacting stochastic constituents are fully characterized by their free energy. Computing this quantity is therefore the objective of various approaches, notably perturbative expansions, which are applied in problems ranging from high-dimensional statistics to complex systems. However, a lot of these techniques are complicated to apply in practice because they lack a sufficient organization of the terms of the perturbative series. In this manuscript, we tackle this problem by using Feynman diagrams, extending a framework introduced earlier to the case of free energies at fixed variances. This diagrammatics do not require the theory to expand around to be Gaussian, which allows its application to the free energy of a spin system studied to derive message-passing algorithms by Maillard et al. 2019. We complete their perturbative derivation of the free energy in the thermodynamic limit. Furthermore, we derive resummations to estimate the entropies of poorly sampled systems requiring only limited statistics and we revisit earlier approaches to compute the free energy of the Ising model, revealing new insights due to the extension of our framework to the free energy at fixed variances. We expect our approach to be particularly useful for problems of high-dimensional statistics, like matrix factorization, and the study of complex networks.

cond-mat.stat-mech↗

arXiv2025

Phase Transitions of the Additive Uniform Noise Channel with Peak Amplitude and Cost Constraint

Under which condition is quantization optimal? We address this question in the context of the additive uniform noise channel under peak amplitude and cost constraints. We compute analytically the capacity-achieving input distribution as a function of the noise level, the average cost constraint, and the curvature of the cost function. We find that when the cost function is concave, the capacity-achieving input distribution is discrete, whereas when the cost function is convex and the cost constraint is active, the support of the capacity-achieving input distribution spans the entire interval. For the cases of a discrete capacity-achieving input distribution, we derive the analytical expressions for the capacity of the channel.

cs.IT↗

arXiv2025

Diagrammatic expansion for the mutual-information rate in the realm of limited statistics

Neurons in sensory systems encode stimulus information into their stochastic spiking response. The mutual information has been extensively applied to these systems to quantify the neurons' capacity of transmitting such information. Yet, while for discrete stimuli, like flashed images or single tones, its computation is straightforward, for dynamical stimuli it is necessary to compute a (mutual) information rate (MIR), therefore integrating over the multiple temporal correlations which characterize sensory systems. Previous methods are based on extensive sampling of the neuronal response, require large amounts of data and are therefore prone to biases and inaccuracy. Here, we develop Moba-MIRA (moment-based mutual-information-rate approximation), a computational method to estimate the mutual information rate. To derive Moba-MIRA, we use Feynman diagrams to expand the mutual information to arbitrary order in the correlations around the corresponding value for the empirical spike count distributions of single bins. As a result, only the empirical estimation of the pairwise correlations between time bins and the single-bin entropies are required, without the need for the whole joint probability distributions. We tested Moba-MIRA on synthetic data generated with generalized linear models, and showed that it requires only a few tens of stimulus repetitions to provide an accurate estimate of the information rate. Finally, we applied it to ex-vivo electrophysiological recordings of rats retina, obtaining rates ranging between 5 to 20 bits per second, consistent with earlier estimates.

q-bio.NC↗

arXiv2024

Information content in continuous attractor neural networks is preserved in the presence of moderate disordered background connectivity

Continuous attractor neural networks (CANN) form an appealing conceptual model for the storage of information in the brain. However a drawback of CANN is that they require finely tuned interactions. We here study the effect of quenched noise in the interactions on the coding of positional information within CANN. Using the replica method we compute the Fisher information for a network with position-dependent input and recurrent connections composed of a short-range (in space) and a disordered component. We find that the loss in positional information is small for not too large disorder strength, indicating that CANN have a regime in which the advantageous effects of local connectivity on information storage outweigh the detrimental ones. Furthermore, a substantial part of this information can be extracted with a simple linear readout.

cond-mat.dis-nn↗

arXiv2023

Diagrammatics for the Inverse Problem in Spin Systems and Simple Liquids

Modeling complex systems, like neural networks, simple liquids or flocks of birds, often works in reverse to textbook approaches: given data for which averages and correlations are known, we try to find the parameters of a given model consistent with it. In general, no exact calculation directly from the model is available and we are left with expensive numerical approaches. A particular situation is that of a perturbed Gaussian model with polynomial corrections for continuous degrees of freedom. Indeed perturbation expansions for this case have been implemented in the last 60 years. However, there are models for which the exactly solvable part is non-Gaussian, such as independent Ising spins in a field, or an ideal gas of particles. We implement a diagrammatic perturbative scheme in weak correlations around a non-Gaussian yet solvable probability weight. This applies in particular to spin models (Ising, Potts, Heisenberg) with weak couplings, or to a simple liquid with a weak interaction potential. Our method casts systems with discrete degrees of freedom and those with continuous ones within the same theoretical framework. When the core theory is Gaussian it reduces to the well-known Feynman diagrammatics.

cond-mat.stat-mech↗

arXiv2022

Self-consistent formulations for stochastic nonlinear neuronal dynamics

Neural dynamics is often investigated with tools from bifurcation theory. However, many neuron models are stochastic, mimicking fluctuations in the input from unknown parts of the brain or the spiking nature of signals. Noise changes the dynamics with respect to the deterministic model; in particular bifurcation theory cannot be applied. We formulate stochastic neuronal dynamics in the Martin-Siggia-Rose de Dominicis-Janssen (MSRDJ) formalism and present the fluctuation expansion of the effective action and the functional renormalization group (fRG) as two systematic ways to incorporate corrections to the mean dynamics and time-dependent statistics due to fluctuations in the presence of nonlinear neuronal gain. To formulate self-consistency equations, we derive a fundamental link between the effective action in the Onsager-Machlup(OM) formalism, which allows the study of phase transitions, and the MSRDJ effective action, which is computationally advantageous. These results in particular allow the derivation of an OM effective action for systems with non-Gaussian noise. This approach naturally leads to effective deterministic equations for the first moment of the stochastic system; they explain how nonlinearities and noise cooperate to produce memory effects. Moreover, the MSRDJ formulation yields an effective linear system that has identical power spectra and linear response. Starting from the better known loopwise approximation, we then discuss the use of the fRG as a method to obtain self-consistency beyond the mean. We present a new efficient truncation scheme for the hierarchy of flow equations for the vertex functions by adapting the Blaizot, Méndez and Wschebor approximation from the derivative expansion to the vertex expansion. The methods are presented by means of the simplest possible example of a stochastic differential equation that has generic features of neuronal dynamics.

cond-mat.stat-mech↗

arXiv2021

Gell-Mann-Low criticality in neural networks

Criticality is deeply related to optimal computational capacity. The lack of a renormalized theory of critical brain dynamics, however, so far limits insights into this form of biological information processing to mean-field results. These methods neglect a key feature of critical systems: the interaction between degrees of freedom across all length scales, which allows for complex nonlinear computation. We present a renormalized theory of a prototypical neural field theory, the stochastic Wilson-Cowan equation. We compute the flow of couplings, which parameterize interactions on increasing length scales. Despite similarities with the Kardar-Parisi-Zhang model, the theory is of a Gell-Mann-Low type, the archetypal form of a renormalizable quantum field theory. Here, nonlinear couplings vanish, flowing towards the Gaussian fixed point, but logarithmically slowly, thus remaining effective on most scales. We show this critical structure of interactions to implement a desirable trade-off between linearity, optimal for information storage, and nonlinearity, required for computation.

cond-mat.dis-nn↗

arXiv2021

Large Deviations Approach to Random Recurrent Neuronal Networks: Parameter Inference and Fluctuation-Induced Transitions

We here unify the field theoretical approach to neuronal networks with large deviations theory. For a prototypical random recurrent network model with continuous-valued units, we show that the effective action is identical to the rate function and derive the latter using field theory. This rate function takes the form of a Kullback-Leibler divergence which enables data-driven inference of model parameters and calculation of fluctuations beyond mean-field theory. Lastly, we expose a regime with fluctuation-induced transitions between mean-field solutions.

cond-mat.dis-nn↗

arXiv2021

Transient chaotic dimensionality expansion by recurrent networks

Neurons in the brain communicate with spikes, which are discrete events in time and value. Functional network models often employ rate units that are continuously coupled by analog signals. Is there a qualitative difference implied by these two forms of signaling? We develop a unified mean-field theory for large random networks to show that first- and second-order statistics in rate and binary networks are in fact identical if rate neurons receive the right amount of noise. Their response to presented stimuli, however, can be radically different. We quantify these differences by studying how nearby state trajectories evolve over time, asking to what extent the dynamics is chaotic. Chaos in the two models is found to be qualitatively different. In binary networks we find a network-size-dependent transition to chaos and a chaotic submanifold whose dimensionality expands stereotypically with time, while rate networks with matched statistics are nonchaotic. Dimensionality expansion in chaotic binary networks aids classification in reservoir computing and optimal performance is reached within about a single activation per neuron; a fast mechanism for computation that we demonstrate also in spiking networks. A generalization of this mechanism extends to rate networks in their respective chaotic regimes.

cond-mat.dis-nn↗

arXiv2018

Expansion of the effective action around non-Gaussian theories

This paper derives the Feynman rules for the diagrammatic perturbation expansion of the effective action around an arbitrary solvable problem. The perturbation expansion around a Gaussian theory is well known and composed of one-line irreducible diagrams only. For the expansions around an arbitrary, non-Gaussian problem, we show that a more general class of irreducible diagrams remains in addition to a second set of diagrams that has no analogue in the Gaussian case. The effective action is central to field theory, in particular to the study of phase transitions, symmetry breaking, effective equations of motion, and renormalization. We exemplify the method on the Ising model, where the effective action amounts to the Gibbs free energy, recovering the Thouless-Anderson-Palmer mean-field theory in a fully diagrammatic derivation. Higher order corrections follow with only minimal effort compared to existing techniques. Our results show further that the Plefka expansion and the high-temperature expansion are special cases of the general formalism presented here.

cond-mat.stat-mech↗

arXiv2017

Locking of correlated neural activity to ongoing oscillations

Population-wide oscillations are ubiquitously observed in mesoscopic signals of cortical activity. In these network states a global oscillatory cycle modulates the propensity of neurons to fire. Synchronous activation of neurons has been hypothesized to be a separate channel of signal processing information in the brain. A salient question is therefore if and how oscillations interact with spike synchrony and in how far these channels can be considered separate. Experiments indeed showed that correlated spiking co-modulates with the static firing rate and is also tightly locked to the phase of beta-oscillations. While the dependence of correlations on the mean rate is well understood in feed-forward networks, it remains unclear why and by which mechanisms correlations tightly lock to an oscillatory cycle. We here demonstrate that such correlated activation of pairs of neurons is qualitatively explained by periodically-driven random networks. We identify the mechanisms by which covariances depend on a driving periodic stimulus. Mean-field theory combined with linear response theory yields closed-form expressions for the cyclostationary mean activities and pairwise zero-time-lag covariances of binary recurrent random networks. Two distinct mechanisms cause time-dependent covariances: the modulation of the susceptibility of single neurons (via the external input and network feedback) and the time-varying variances of single unit activities. For some parameters, the effectively inhibitory recurrent feedback leads to resonant covariances even if mean activities show non-resonant behavior. Our analytical results open the question of time-modulated synchronous activity to a quantitative analysis.

q-bio.NC↗

arXiv2016

mlr Tutorial

This document provides and in-depth introduction to the mlr framework for machine learning experiments in R.

cs.LG↗