Searcharxiv⌕ Search

arXiv subjects

Klaus Obermayer

Publications and source records attributed to Klaus Obermayer.

At least 37 records · Page 2Linked to original sources
arXiv2019

A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with Applications in Partially Observable Markov Decision Processes

By using the fact that the space of all probability measures with finite support can be somehow completed in two different fashions, one generating the Arens-Eells space and another generating the Kantorovich-Wasserstein (Wasserstein-1) space, and by exploiting the duality relationship between the Arens-Eells space with the space of Lipschitz functions, we provide a dual representation of Fenchel-Moreau-Rockafellar type for proper convex functionals on Wasserstein-1. We retrieve dual transportation inequalities as a Corollary and we provide examples where the theorem can be used to easily prove dual expressions like the celebrated Donsker-Varadhan variational formula. Finally our result allows to write convex functions as the supremum over all linear functions that are generated by roots of its conjugate dual, something that we apply to the field of Partially observable Markov decision processes (POMDPs) to approximate the value function of a given POMDP by iterating level sets. This extends the method used in Smallwood 1973 for finite state spaces to the case were the state space is a Polish metric space.

math.PR↗
arXiv2017

Low-dimensional spike rate models derived from networks of adaptive integrate-and-fire neurons: Comparison and implementation

The spiking activity of single neurons can be well described by a nonlinear integrate-and-fire model that includes somatic adaptation. When exposed to fluctuating inputs sparsely coupled populations of these model neurons exhibit stochastic collective dynamics that can be effectively characterized using the Fokker-Planck equation. [...] Here we derive from that description four simple models for the spike rate dynamics in terms of low-dimensional ordinary differential equations using two different reduction techniques: one uses the spectral decomposition of the Fokker-Planck operator, the other is based on a cascade of two linear filters and a nonlinearity, which are determined from the Fokker-Planck equation and semi-analytically approximated. We evaluate the reduced models for a wide range of biologically plausible input statistics and find that both approximation approaches lead to spike rate models that accurately reproduce the spiking behavior of the underlying adaptive integrate-and-fire population. [...] The low-dimensional models also well reproduce stable oscillatory spike rate dynamics that is generated by recurrent synaptic excitation and neuronal adaptation. [...] We have made available implementations that allow to numerically integrate the low-dimensional spike rate models as well as the Fokker-Planck partial differential equation in efficient ways for arbitrary model parametrizations as open source software. The derived spike rate descriptions retain a direct link to the properties of single neurons, allow for convenient mathematical analyses of network states, and are well suited for application in neural mass/mean-field based brain network models.

q-bio.NC↗
arXiv2017

Approximate Inference for Time-varying Interactions and Macroscopic Dynamics of Neural Populations

The models in statistical physics such as an Ising model offer a convenient way to characterize stationary activity of neural populations. Such stationary activity of neurons may be expected for recordings from in vitro slices or anesthetized animals. However, modeling activity of cortical circuitries of awake animals has been more challenging because both spike-rates and interactions can change according to sensory stimulation, behavior, or an internal state of the brain. Previous approaches modeling the dynamics of neural interactions suffer from computational cost; therefore, its application was limited to only a dozen neurons. Here by introducing multiple analytic approximation methods to a state-space model of neural population activity, we make it possible to estimate dynamic pairwise interactions of up to 60 neurons. More specifically, we applied the pseudolikelihood approximation to the state-space model, and combined it with the Bethe or TAP mean-field approximation to make the sequential Bayesian estimation of the model parameters possible. The large-scale analysis allows us to investigate dynamics of macroscopic properties of neural circuitries underlying stimulus processing and behavior. We show that the model accurately estimates dynamics of network properties such as sparseness, entropy, and heat capacity by simulated data, and demonstrate utilities of these measures by analyzing activity of monkey V4 neurons as well as a simulated balanced network of spiking neurons.

q-bio.NC↗
arXiv2017

A computational study on synaptic and extrasynaptic effects of astrocyte glutamate uptake on orientation tuning in V1

Astrocytes affect neural transmission by a tight control via glutamate transporters on glutamate concentrations in direct vicinity to the synaptic cleft and by extracellular glutamate. Their relevance for information representation has been supported by in-vivo studies in ferret and mouse primary visual cortex. In ferret blocking glutamate transport pharmacologically broadened tuning curves and enhanced the response at preferred orientation. In knock-out mice with reduced expression of glutamate transporters sharpened tuning was observed. It is however unclear how focal and ambient changes in glutamate concentration affect stimulus representation. Here we develop a computational framework, which allows the investigation of synaptic and extrasynaptic effects of glutamate uptake on orientation tuning in recurrently connected network models with pinwheel-domain (ferret) or salt-and-pepper (mouse) organization. This model proposed that glutamate uptake shapes information representation when it affects the contribution of excitatory and inhibitory neurons to the network activity. Namely, strengthening the contribution of excitatory neurons generally broadens tuning and elevates the response. In contrast, strengthening the contribution of inhibitory neurons can have a sharpening effect on tuning. In addition local representational topology also plays a role: In the pinwheel-domain model effects were strongest within domains - regions where neighboring neurons share preferred orientations. Around pinwheels but also within salt-and-pepper networks the effects were less strong. Our model proposes that the pharmacological intervention in ferret increases the contribution of excitatory cells, while the reduced expression in mouse increases the contribution of inhibitory cells to network activity.

q-bio.NC↗
arXiv2016

Non-Deterministic Policy Improvement Stabilizes Approximated Reinforcement Learning

This paper investigates a type of instability that is linked to the greedy policy improvement in approximated reinforcement learning. We show empirically that non-deterministic policy improvement can stabilize methods like LSPI by controlling the improvements' stochasticity. Additionally we show that a suitable representation of the value function also stabilizes the solution to some degree. The presented approach is simple and should also be easily transferable to more sophisticated algorithms like deep reinforcement learning.

cs.AI↗
arXiv2016

Extending integrate-and-fire model neurons to account for the effects of weak electric fields and input filtering mediated by the dendrite

How extracellular electric fields, as generated endogenously or through transcranial brain stimulation, affect the dynamics of large neuronal populations is of great interest but not well understood. To study the collective dynamics of large populations single-compartment (point) model neurons have been proven very successful. These models, however, lack the dendritic morphology to biophysically account for the effects of electric fields, and for changes in synaptic integration due to morphology alone. Here we (i) characterize the response of a canonical spatial (ball-and-stick) model neuron to fluctuating synaptic input as well as an oscillatory, weak electric field, and (ii) analytically derive an extension for popular integrate-and-fire point neuron models to accurately reproduce these responses. We obtain distinct filters mediated by the dendrite for inputs at the soma (high-pass filter) or at the distal dendritic site (low-pass filter), and find that the electric field induces spike rate resonance in the beta and gamma frequency bands or even higher frequencies, depending on the location of synaptic background input. Due to their computational efficiency the extended point models are well suited for application in large populations of coupled neurons with different morphology, exposed to extracellular electric fields.

q-bio.NC↗
arXiv2016

Controlling Statistical Moments of Stochastic Dynamical Networks

We consider a general class of stochastic networks and ask which network nodes need to be controlled, and how, to stabilize and switch between desired metastable (target) states in terms of the first and second statistical moments of the system. We first show that it is sufficient to directly interfere with a subset of nodes which can be identified using information about the graph of the network only. Then, we develop a suitable method for feedback control which acts on that subset of nodes and preserves the covariance structure of the desired target state. Finally, we demonstrate our theoretical results using a stochastic Hopfield network and a global brain model. Our results are applicable to a variety of (model) networks, and further our understanding of the relationship between network structure and collective dynamics for the benefit of effective control.

nlin.AO↗
arXiv2015

On Average Risk-sensitive Markov Control Processes

We introduce the Lyapunov approach to optimal control problems of average risk-sensitive Markov control processes with general risk maps. Motivated by applications in particular to behavioral economics, we consider possibly non-convex risk maps, modeling behavior with mixed risk preference. We introduce classical objective functions to the risk-sensitive setting and we are in particular interested in optimizing the average risk in the infinite-time horizon for Markov Control Processes on general, possibly non-compact, state spaces allowing also unbounded cost. Existence and uniqueness of an optimal control is obtained with a fixed point theorem applied to the nonlinear map modeling the risk-sensitive expected total cost. The necessary contraction is obtained in a suitable chosen seminorm under a new set of conditions: 1) Lyapunov-type conditions on both risk maps and cost functions that control the growth of iterations, and 2) Doeblin-type conditions, known for Markov chains, generalized to nonlinear mappings. In the particular case of the entropic risk map, the above conditions can be replaced by the existence of a Lyapunov function, a local Doeblin-type condition for the underlying Markov chain, and a growth condition on the cost functions.

math.OC↗
arXiv2015

Regression with Linear Factored Functions

Many applications that use empirically estimated functions face a curse of dimensionality, because the integrals over most function classes must be approximated by sampling. This paper introduces a novel regression-algorithm that learns linear factored functions (LFF). This class of functions has structural properties that allow to analytically solve certain integrals and to calculate point-wise products. Applications like belief propagation and reinforcement learning can exploit these properties to break the curse and speed up computation. We derive a regularized greedy optimization scheme, that learns factored basis functions during training. The novel regression algorithm performs competitively to Gaussian processes on benchmark tasks, and the learned LFF functions are with 4-9 factored basis functions on average very compact.

cs.LG↗
arXiv2014

Analyzing critical propagation in a reaction-diffusion-advection model using unstable slow waves

The effect of advection on the critical minimal speed of traveling waves is studied. Previous theoretical studies estimated the effect on the velocity of stable fast waves and predicted the existence of a critical advection strength below which propagating waves are not supported anymore. In this paper, the critical advection strength is calculated taking into account the unstable slow wave solution. Thereby, theoretical results predict, that advection can induce stable wave propagation in the non-excitable parameter regime, if the advection strength exceeds a critical value. In addition, an analytical expression for the advection-velocity relation of the unstable slow wave is derived. Predictions are confirmed numerically in a two-variable reaction-diffusion model.

nlin.PS↗
arXiv2014

Risk-sensitive Markov control processes

We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate also unbounded costs, we study two types of infinite-horizon risk-sensitive criteria, discounted total risk and average risk, and solve the associated optimization problems by dynamic programming. For the discounted case, we propose a new discount scheme, which is different from the conventional form but consistent with the existing literature, while for the average risk criterion, we state Lyapunov-like stability conditions that generalize known conditions for Markov chains to ensure the existence of solutions to the optimality equation.

math.OC↗
arXiv2014

Risk-sensitive Reinforcement Learning

We derive a family of risk-sensitive reinforcement learning methods for agents, who face sequential decision-making tasks in uncertain environments. By applying a utility function to the temporal difference (TD) error, nonlinear transformations are effectively applied not only to the received rewards but also to the true transition probabilities of the underlying Markov decision process. When appropriate utility functions are chosen, the agents' behaviors express key features of human behavior as predicted by prospect theory (Kahneman and Tversky, 1979), for example different risk-preferences for gains and losses as well as the shape of subjective probability curves. We derive a risk-sensitive Q-learning algorithm, which is necessary for modeling human behavior when transition probabilities are unknown, and prove its convergence. As a proof of principle for the applicability of the new framework we apply it to quantify human behavior in a sequential investment task. We find, that the risk-sensitive variant provides a significantly better fit to the behavioral data and that it leads to an interpretation of the subject's responses which is indeed consistent with prospect theory. The analysis of simultaneously measured fMRI signals show a significant correlation of the risk-sensitive TD error with BOLD signal change in the ventral striatum. In addition we find a significant correlation of the risk-sensitive Q-values with neural activity in the striatum, cingulate cortex and insula, which is not present if standard Q-values are used.

cs.LG↗
arXiv2013

How adaptation currents change threshold, gain and variability of neuronal spiking

Many types of neurons exhibit spike rate adaptation, mediated by intrinsic slow $\mathrm{K}^+$-currents, which effectively inhibit neuronal responses. How these adaptation currents change the relationship between in-vivo like fluctuating synaptic input, spike rate output and the spike train statistics, however, is not well understood. In this computational study we show that an adaptation current which primarily depends on the subthreshold membrane voltage changes the neuronal input-output relationship (I-O curve) subtractively, thereby increasing the response threshold. A spike-dependent adaptation current alters the I-O curve divisively, thus reducing the response gain. Both types of adaptation currents naturally increase the mean inter-spike interval (ISI), but they can affect ISI variability in opposite ways. A subthreshold current always causes an increase of variability while a spike-triggered current decreases high variability caused by fluctuation-dominated inputs and increases low variability when the average input is large. The effects on I-O curves match those caused by synaptic inhibition in networks with asynchronous irregular activity, for which we find subtractive and divisive changes caused by external and recurrent inhibition, respectively. Synaptic inhibition, however, always increases the ISI variability. We analytically derive expressions for the I-O curve and ISI variability, which demonstrate the robustness of our results. Furthermore, we show how the biophysical parameters of slow $\mathrm{K}^+$-conductances contribute to the two different types of adaptation currents and find that $\mathrm{Ca}^{2+}$-activated $\mathrm{K}^+$-currents are effectively captured by a simple spike-dependent description, while muscarine-sensitive or $\mathrm{Na}^+$-activated $\mathrm{K}^+$-currents show a dominant subthreshold component.

q-bio.NC↗
arXiv2013

Adaptation controls synchrony and cluster states of coupled threshold-model neurons

We analyze zero-lag and cluster synchrony of delay-coupled non-smooth dynamical systems by extending the master stability approach, and apply this to networks of adaptive threshold-model neurons. For a homogeneous population of excitatory and inhibitory neurons we find (i) that subthreshold adaptation stabilizes or destabilizes synchrony depending on whether the recurrent synaptic excitatory or inhibitory couplings dominate, and (ii) that synchrony is always unstable for networks with balanced recurrent synaptic inputs. If couplings are not too strong, synchronization properties are similar for very different coupling topologies, i.e., random connections or spatial networks with localized connectivity. We generalize our approach for two subpopulations of neurons with non-identical local dynamics, including bursting, for which activity-based adaptation controls the stability of cluster states, independent of a specific coupling topology.

nlin.AO↗
arXiv2013

Impact of adaptation currents on synchronization of coupled exponential integrate-and-fire neurons

Author summary: Synchronization of neuronal spiking in the brain is related to cognitive functions, such as perception, attention, and memory. It is therefore important to determine which properties of neurons influence their collective behavior in a network and to understand how. A prominent feature of many cortical neurons is spike frequency adaptation, which is caused by slow transmembrane currents. We investigated how these adaptation currents affect the synchronization tendency of coupled model neurons. Using the efficient adaptive exponential integrate-and-fire (aEIF) model and a biophysically detailed neuron model for validation, we found that increased adaptation currents promote synchronization of coupled excitatory neurons at lower spike frequencies, as long as the conduction delays between the neurons are negligible. Inhibitory neurons on the other hand synchronize in presence of conduction delays, with or without adaptation currents. Our results emphasize the utility of the aEIF model for computational studies of neuronal network dynamics. We conclude that adaptation currents provide a mechanism to generate low frequency oscillations in local populations of excitatory neurons, while faster rhythms seem to be caused by inhibition rather than excitation.

q-bio.NC↗
arXiv2013

Afferent specificity, feature specific connectivity influence orientation selectivity: A computational study in mouse primary visual cortex

Primary visual cortex (V1) provides crucial insights into the selectivity and emergence of specific output features such as orientation tuning. Tuning and selectivity of cortical neurons in mouse visual cortex is not equivocally resolved so far. While many in-vivo experimental studies found inhibitory neurons of all subtypes to be broadly tuned for orientation other studies report inhibitory neurons that are as sharply tuned as excitatory neurons. These diverging findings about the selectivity of excitatory and inhibitory cortical neurons prompted us to ask the following questions: (1) How different or similar is the cortical computation with that in previously described species that relies on map? (2) What is the network mechanism underlying the sharpening of orientation selectivity in the mouse primary visual cortex? Here, we investigate the above questions in a computational framework with a recurrent network composed of Hodgkin-Huxley (HH) point neurons. Our cortical network with random connectivity alone could not account for all the experimental observations, which led us to hypothesize, (a) Orientation dependent connectivity (b) Feedforward afferent specificity to understand orientation selectivity of V1 neurons in mouse. Using population (orientation selectivity index) OSI as a measure of neuronal selectivity to stimulus orientation we test each hypothesis separately and in combination against experimental data. Based on our analysis of orientation selectivity (OS) data we find a good fit of network parameters in a model based on afferent specificity and connectivity that scales with feature similarity. We conclude that this particular model class best supports data sets of orientation selectivity of excitatory and inhibitory neurons in layer 2/3 of primary visual cortex of mouse.

q-bio.NC↗
arXiv2012

Learning in Riemannian Orbifolds

Learning in Riemannian orbifolds is motivated by existing machine learning algorithms that directly operate on finite combinatorial structures such as point patterns, trees, and graphs. These methods, however, lack statistical justification. This contribution derives consistency results for learning problems in structured domains and thereby generalizes learning in vector spaces and manifolds.

cs.LG↗