arXiv subjects

Klaus Obermayer

Publications and source records attributed to Klaus Obermayer.

At least 19 recordsLinked to original sources

arXiv2026

Forward and reverse delay-driven hippocampal replay without symmetric plasticity

Hippocampal replay is a phenomenon observed in mammals and songbirds where neural activation sequences experienced during wakeful periods are repeated during rest or sleep. This mechanism is believed to play a crucial role in episodic memory consolidation, retrieval, and planning. Interestingly, replay can occur in both forward and reverse temporal orders and across a wide range of increased speeds. The learning of activation sequences has traditionally been modeled by temporally asymmetric Hebbian plasticity rules, which explain replay as a chain of activity. The inability of such rules to strengthen backward connections has led to the widespread belief that they cannot account for bidirectional replay. Existing theoretical work therefore explains reverse replay through symmetric connections, which would be formed through symmetric plasticity rules. We propose a delay-coupled neural field model of CA3, where activity propagates as a wave in a ring, and replays occur both forwards and backwards at different speeds. Our model can learn bidirectional replay after a single noisy exposure to a stimulus with either symmetric or asymmetric plasticity. We derive a low-dimensional description of the dynamics and analyze the stability of possible replay speeds.

q-bio.NC↗

arXiv2026

Revisiting Scene Graph Generation from the Perspective of Detector-Conditioned Reachability

Scene graph generation (SGG) approaches can be broadly classified into detector-based and query-based methods according to their underlying reasoning mechanisms. However, the discrepancy in their predictive behaviors, induced by these distinct mechanisms, has not been systematically analyzed. In this work, we design a controlled experimental setup to examine prediction discrepancies from the perspective of detector-conditioned reachability. The results suggest clear complementary clues. Motivated by this observation, we introduce a Dual-SGG method that consolidates both reasoning mechanisms via a dual-query design, thereby leveraging the complementary predictive behaviors of both detector-based and query-based methods. Extensive experiments on the Visual Genome, Open Images v6, and GQA-200 datasets demonstrate the effectiveness of the proposed method.

cs.CV↗

arXiv2026

Spike-frequency and h-current based adaptation are dynamically equivalent in a Wilson-Cowan field model

During slow-wave sleep, the brain produces traveling waves of slow oscillations (SOs; $\leq 2$ Hz), characterized by the propagation of alternating high- and low-activity states. The question of internal mechanisms that modulate traveling waves of SOs is still unanswered although it is established that it is an adaptation mechanism that mediates them. One mechanism investigated is spike-frequency adaptation, a hyperpolarizing feedback current that is activated during periods of high-activity. An alternative mechanism is based on hyperpolarization-activated currents, which are positive feedback currents that are activated in low-activity states. Both adaptation mechanisms were shown to feature SO-like dynamics in neuronal populations, and the inclusion of a spatial domain seems to enhance observable differences in their effects. To investigate this in detail, we examine a spatially extended two-population Wilson-Cowan model with local spatial coupling and the excitatory populations equipped with either one of the two adaptation mechanisms. We describe them with the same dynamical equation and include the inverse mode of action by changing the signs of adaptation strength and gain. We show that the dynamical systems are mathematically equivalent under a compensatory external input, which depends on the adaptation strength, leading to a shift in state space of the otherwise equivalent bifurcation structure. Strong enough adaptation is required to induce traveling waves. Additionally, adaptation modulates the properties of the spatio-temporal activity patterns, such as temporal and spatial frequencies, and the speed of the traveling waves, all of which increase with increasing strength. Though being dynamically equivalent, our results also explain why location-dependent variations in feedback strength cause differences in the propagation of traveling waves between both adaptation mechanisms.

nlin.AO↗

arXiv2026

Salience-SGG: Enhancing Unbiased Scene Graph Generation with Iterative Salience Estimation

Scene Graph Generation (SGG) suffers from a long-tailed distribution, where a few predicate classes dominate while many others are underrepresented, leading to biased models that underperform on rare relations. Unbiased-SGG methods address this issue by implementing debiasing strategies, but often at the cost of spatial understanding, resulting in an over-reliance on semantic priors. We introduce Salience-SGG, a novel framework featuring an Iterative Salience Decoder (ISD) that emphasizes triplets with salient spatial structures. To support this, we propose semantic-agnostic salience labels guiding ISD. Evaluations on Visual Genome, Open Images V6, and GQA-200 show that Salience-SGG achieves state-of-the-art performance and improves existing Unbiased-SGG methods in their spatial understanding as demonstrated by the Pairwise Localization Average Precision

cs.CV↗

arXiv2025

Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations

Deep learning appears as an appealing solution for Automatic Synthesizer Programming (ASP), which aims to assist musicians and sound designers in programming sound synthesizers. However, integrating software synthesizers into training pipelines is challenging due to their potential non-differentiability. This work tackles this challenge by introducing a method to approximate arbitrary synthesizers. Specifically, we train a neural network to map synthesizer presets onto an audio embedding space derived from a pretrained model. This facilitates the definition of a neural proxy that produces compact yet effective representations, thereby enabling the integration of audio embedding loss into neural-based ASP systems for black-box synthesizers. We evaluate the representations derived by various pretrained audio models in the context of neural-based nASP and assess the effectiveness of several neural network architectures, including feedforward, recurrent, and transformer-based models, in defining neural proxies. We evaluate the proposed method using both synthetic and hand-crafted presets from three popular software synthesizers and assess its performance in a synthesizer sound matching downstream task. While the benefits of the learned representation are nuanced by resource requirements, encouraging results were obtained for all synthesizers, paving the way for future research into the application of synthesizer proxies for neural-based ASP systems.

cs.SD↗

arXiv2025

On the robustness of the emergent spatiotemporal dynamics in biophysically realistic and phenomenological whole-brain models at multiple network resolutions

The human brain is a complex dynamical system which displays a wide range of macroscopic and mesoscopic patterns of neural activity, whose mechanistic origin remains poorly understood. Whole-brain modelling allows us to explore candidate mechanisms causing the observed patterns. However, it is not fully established how the choice of model type and the networks' resolution influence the simulation results, hence, it remains unclear, to which extent conclusions drawn from these results are limited by modelling artefacts. Here, we compare the dynamics of a biophysically realistic, linear-nonlinear cascade model of whole-brain activity with a phenomenological Wilson-Cowan model using three structural connectomes based on the Schaefer parcellation scheme with 100, 200, and 500 nodes. Both neural mass models implement the same mechanistic hypotheses, which specifically address the interaction between excitation, inhibition, and a slow adaptation current, which affects the excitatory populations. We quantify the emerging dynamical states in detail and investigate how consistent results are across the different model variants. Then we apply both model types to the specific phenomenon of slow oscillations, which are a prevalent brain rhythm during deep sleep. We investigate the consistency of model predictions when exploring specific mechanistic hypotheses about the effects of both short- and long-range connections and of the antero-posterior structural connectivity gradient on key properties of these oscillations. Overall, our results demonstrate that the coarse-grained dynamics are robust to changes in both model type and network resolution. In some cases, however, model predictions do not generalize. Thus, some care must be taken when interpreting model results.

q-bio.NC↗

arXiv2025

A Robotics-Inspired Scanpath Model Reveals the Importance of Uncertainty and Semantic Object Cues for Gaze Guidance in Dynamic Scenes

The objects we perceive guide our eye movements when observing real-world dynamic scenes. Yet, gaze shifts and selective attention are critical for perceiving details and refining object boundaries. Object segmentation and gaze behavior are, however, typically treated as two independent processes. Here, we present a computational model that simulates these processes in an interconnected manner and allows for hypothesis-driven investigations of distinct attentional mechanisms. Drawing on an information processing pattern from robotics, we use a Bayesian filter to recursively segment the scene, which also provides an uncertainty estimate for the object boundaries that we use to guide active scene exploration. We demonstrate that this model closely resembles observers' free viewing behavior on a dataset of dynamic real-world scenes, measured by scanpath statistics, including foveation duration and saccade amplitude distributions used for parameter fitting and higher-level statistics not used for fitting. These include how object detections, inspections, and returns are balanced and a delay of returning saccades without an explicit implementation of such temporal inhibition of return. Extensive simulations and ablation studies show that uncertainty promotes balanced exploration and that semantic object cues are crucial to forming the perceptual units used in object-based attention. Moreover, we show how our model's modular design allows for extensions, such as incorporating saccadic momentum or pre-saccadic attention, to further align its output with human scanpaths.

cs.CV↗

arXiv2022

Transfer Learning for Segmentation Problems: Choose the Right Encoder and Skip the Decoder

It is common practice to reuse models initially trained on different data to increase downstream task performance. Especially in the computer vision domain, ImageNet-pretrained weights have been successfully used for various tasks. In this work, we investigate the impact of transfer learning for segmentation problems, being pixel-wise classification problems that can be tackled with encoder-decoder architectures. We find that transfer learning the decoder does not help downstream segmentation tasks, while transfer learning the encoder is truly beneficial. We demonstrate that pretrained weights for a decoder may yield faster convergence, but they do not improve the overall model performance as one can obtain equivalent results with randomly initialized decoders. However, we show that it is more effective to reuse encoder weights trained on a segmentation or reconstruction task than reusing encoder weights trained on classification tasks. This finding implicates that using ImageNet-pretrained encoders for downstream segmentation problems is suboptimal. We also propose a contrastive self-supervised approach with multiple self-reconstruction tasks, which provides encoders that are suitable for transfer learning in segmentation problems in the absence of segmentation labels.

cs.CV↗

arXiv2022

Similarity of Pre-trained and Fine-tuned Representations

In transfer learning, only the last part of the networks - the so-called head - is often fine-tuned. Representation similarity analysis shows that the most significant change still occurs in the head even if all weights are updatable. However, recent results from few-shot learning have shown that representation change in the early layers, which are mostly convolutional, is beneficial, especially in the case of cross-domain adaption. In our paper, we find out whether that also holds true for transfer learning. In addition, we analyze the change of representation in transfer learning, both during pre-training and fine-tuning, and find out that pre-trained structure is unlearned if not usable.

cs.LG↗

arXiv2022

Risk-Sensitive Partially Observable Markov Decision Processes as Fully Observable Multivariate Utility Optimization problems

We provide a new algorithm for solving Risk Sensitive Partially Observable Markov Decisions Processes, when the risk is modeled by a utility function, and both the state space and the space of observations is finite. This algorithm is based on an observation that the change of measure and the subsequent introduction of the information space that is used for exponential utility functions, can be actually extended for sums of exponentials if one introduces an extra vector parameter that tracks the "expected accumulated cost" that corresponds to each exponential. Since every increasing function can be approximated by sums of exponentials in finite intervals, the method can be essentially applied for any utility function, with its complexity depending on the number of exponentials.

math.OC↗

arXiv2021

Spatiotemporal patterns of adaptation-induced slow oscillations in a whole-brain model of slow-wave sleep

During slow-wave sleep, the brain is in a self-organized regime in which slow oscillations (SOs) between up- and down-states travel across the cortex. While an isolated piece of cortex can produce SOs, the brain-wide propagation of these oscillations are thought to be mediated by the long-range axonal connections. We address the mechanism of how SOs emerge and recruit large parts of the brain using a whole-brain model constructed from empirical connectivity data in which SOs are induced independently in each brain area by a local adaptation mechanism. Using an evolutionary optimization approach, good fits to human resting-state fMRI data and sleep EEG data are found at values of the adaptation strength close to a bifurcation where the model produces a balance between local and global SOs with realistic spatiotemporal statistics. Local oscillations are more frequent, last shorter, and have a lower amplitude. Global oscillations spread as waves of silence across the brain, traveling from anterior to posterior regions. These traveling waves are caused by heterogeneities in the brain network in which the connection strengths between brain areas determine which areas transition to a down-state first, and thus initiate traveling waves across the cortex. Our results demonstrate the utility of whole-brain models for explaining the origin of large-scale cortical oscillations and how they are shaped by the connectome.

q-bio.NC↗

arXiv2021

Applications of optimal nonlinear control to a whole-brain network of FitzHugh-Nagumo oscillators

We apply the framework of optimal nonlinear control to steer the dynamics of a whole-brain network of FitzHugh-Nagumo oscillators. Its nodes correspond to the cortical areas of an atlas-based segmentation of the human cerebral cortex, and the inter-node coupling strengths are derived from Diffusion Tensor Imaging data of the connectome of the human brain. Nodes are coupled using an additive scheme without delays and are driven by background inputs with fixed mean and additive Gaussian noise. Optimal control inputs to nodes are determined by minimizing a cost functional that penalizes the deviations from a desired network dynamic, the control energy, and spatially non-sparse control inputs. Using the strength of the background input and the overall coupling strength as order parameters, the network's state-space decomposes into regions of low and high activity fixed points separated by a high amplitude limit cycle all of which qualitatively correspond to the states of an isolated network node. Along the borders, however, additional limit cycles, asynchronous states and multistability can be observed. Optimal control is applied to several state-switching and network synchronization tasks, and the results are compared to controllability measures from linear control theory for the same connectome. We find that intuitions from the latter about the roles of nodes in steering the network dynamics, which are solely based on connectome features, do not generally carry over to nonlinear systems, as had been previously implied. Instead, the role of nodes under optimal nonlinear control critically depends on the specified task and the system's location in state space. Our results shed new light on the controllability of brain network states and may serve as an inspiration for the design of new paradigms for non-invasive brain stimulation.

q-bio.NC↗

arXiv2021

A modular framework for object-based saccadic decisions in dynamic scenes

Visually exploring the world around us is not a passive process. Instead, we actively explore the world and acquire visual information over time. Here, we present a new model for simulating human eye-movement behavior in dynamic real-world scenes. We model this active scene exploration as a sequential decision making process. We adapt the popular drift-diffusion model (DDM) for perceptual decision making and extend it towards multiple options, defined by objects present in the scene. For each possible choice, the model integrates evidence over time and a decision (saccadic eye movement) is triggered as soon as evidence crosses a decision threshold. Drawing this explicit connection between decision making and object-based scene perception is highly relevant in the context of active viewing, where decisions are made continuously while interacting with an external environment. We validate our model with a carefully designed ablation study and explore influences of our model parameters. A comparison on the VidCom dataset supports the plausibility of the proposed approach.

cs.CV↗

arXiv2021

Exploring the Similarity of Representations in Model-Agnostic Meta-Learning

In past years model-agnostic meta-learning (MAML) has been one of the most promising approaches in meta-learning. It can be applied to different kinds of problems, e.g., reinforcement learning, but also shows good results on few-shot learning tasks. Besides their tremendous success in these tasks, it has still not been fully revealed yet, why it works so well. Recent work proposes that MAML rather reuses features than rapidly learns. In this paper, we want to inspire a deeper understanding of this question by analyzing MAML's representation. We apply representation similarity analysis (RSA), a well-established method in neuroscience, to the few-shot learning instantiation of MAML. Although some part of our analysis supports their general results that feature reuse is predominant, we also reveal arguments against their conclusion. The similarity-increase of layers closer to the input layers arises from the learning task itself and not from the model. In addition, the representations after inner gradient steps make a broader change to the representation than the changes during meta-training.

cs.LG↗

arXiv2020

Biophysically grounded mean-field models of neural populations under electrical stimulation

Electrical stimulation of neural systems is a key tool for understanding neural dynamics and ultimately for developing clinical treatments. Many applications of electrical stimulation affect large populations of neurons. However, computational models of large networks of spiking neurons are inherently hard to simulate and analyze. We evaluate a reduced mean-field model of excitatory and inhibitory adaptive exponential integrate-and-fire (AdEx) neurons which can be used to efficiently study the effects of electrical stimulation on large neural populations. The rich dynamical properties of this basic cortical model are described in detail and validated using large network simulations. Bifurcation diagrams reflecting the network's state reveal asynchronous up- and down-states, bistable regimes, and oscillatory regions corresponding to fast excitation-inhibition and slow excitation-adaptation feedback loops. The biophysical parameters of the AdEx neuron can be coupled to an electric field with realistic field strengths which then can be propagated up to the population description.We show how on the edge of bifurcation, direct electrical inputs cause network state transitions, such as turning on and off oscillations of the population rate. Oscillatory input can frequency-entrain and phase-lock endogenous oscillations. Relatively weak electric field strengths on the order of 1 V/m are able to produce these effects, indicating that field effects are strongly amplified in the network. The effects of time-varying external stimulation are well-predicted by the mean-field model, further underpinning the utility of low-dimensional neural mass models.

q-bio.NC↗

arXiv2020

Training Generative Networks with general Optimal Transport distances

We propose a new algorithm that uses an auxiliary neural network to express the potential of the optimal transport map between two data distributions. In the sequel, we use the aforementioned map to train generative networks. Unlike WGANs, where the Euclidean distance is ${\it implicitly}$ used, this new method allows to ${\it explicitly}$ use ${\it any}$ transportation cost function that can be chosen to match the problem at hand. For example, it allows to use the squared distance as a transportation cost function, giving rise to the Wasserstein-2 metric for probability distributions, which results in fast and stable gradient descends. It also allows to use image centered distances, like the structure similarity index, with notable differences in the results.

cs.LG↗

arXiv2020

The NIGENS General Sound Events Database

Computational auditory scene analysis is gaining interest in the last years. Trailing behind the more mature field of speech recognition, it is particularly general sound event detection that is attracting increasing attention. Crucial for training and testing reasonable models is having available enough suitable data -- until recently, general sound event databases were hardly found. We release and present a database with 714 wav files containing isolated high quality sound events of 14 different types, plus 303 `general' wav files of anything else but these 14 types. All sound events are strongly labeled with perceptual on- and offset times, paying attention to omitting in-between silences. The amount of isolated sound events, the quality of annotations, and the particular general sound class distinguish NIGENS from other databases.

cs.SD↗

arXiv2019

Joining Sound Event Detection and Localization Through Spatial Segregation

Identification and localization of sounds are both integral parts of computational auditory scene analysis. Although each can be solved separately, the goal of forming coherent auditory objects and achieving a comprehensive spatial scene understanding suggests pursuing a joint solution of the two problems. This work presents an approach that robustly binds localization with the detection of sound events in a binaural robotic system. Both tasks are joined through the use of spatial stream segregation which produces probabilistic time-frequency masks for individual sources attributable to separate locations, enabling segregated sound event detection operating on these streams. We use simulations of a comprehensive suite of test scenes with multiple co-occurring sound sources, and propose performance measures for systematic investigation of the impact of scene complexity on this segregated detection of sound types. Analyzing the effect of spatial scene arrangement, we show how a robot could facilitate high performance through optimal head rotation. Furthermore, we investigate the performance of segregated detection given possible localization error as well as error in the estimation of number of active sources. Our analysis demonstrates that the proposed approach is an effective method to obtain joint sound event location and type information under a wide range of conditions.

cs.SD↗