arXiv subjects

Patrick Krauss

Publications and source records attributed to Patrick Krauss.

52 records · Page 3Linked to original sources

arXiv2021

Control of noise-induced coherent oscillations in three-neuron motifs

The phenomenon of self-induced stochastic resonance (SISR) requires a nontrivial scaling limit between the deterministic and the stochastic timescales of an excitable system, leading to the emergence of coherent oscillations which are absent without noise. In this paper, we numerically investigate SISR and its control in single neurons and three-neuron motifs made up of the Morris-Lecar model. In single neurons, we compare the effects of electrical and chemical autapses on the degree of coherence of the oscillations due to SISR. In the motifs, we compare the effects of altering the synaptic time-delayed couplings and the topologies on the degree of SISR. Finally, we provide two enhancement strategies for a particularly poor degree of SISR in motifs with chemical synapses: (i) we show that a poor SISR can be significantly enhanced by attaching an electrical or an excitatory chemical autapse on one of the neurons, and (ii) we show that by multiplexing the motif with a poor SISR to another motif (with a high SISR in isolation), the degree of SISR in the former motif can be significantly enhanced. We show that the efficiency of these enhancement strategies depends on the topology of the motifs and the nature of synaptic time-delayed couplings mediating the multiplexing connections.

nlin.AO↗

arXiv2021

Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future

In this article, we perform a review of the state-of-the-art of hybrid machine learning in medical imaging. We start with a short summary of the general developments of the past in machine learning and how general and specialized approaches have been in competition in the past decades. A particular focus will be the theoretical and experimental evidence pro and contra hybrid modelling. Next, we inspect several new developments regarding hybrid machine learning with a particular focus on so-called known operator learning and how hybrid approaches gain more and more momentum across essentially all applications in medical imaging and medical image analysis. As we will point out by numerous examples, hybrid models are taking over in image reconstruction and analysis. Even domains such as physical simulation and scanner and acquisition design are being addressed using machine learning grey box modelling approaches. Towards the end of the article, we will investigate a few future directions and point out relevant areas in which hybrid modelling, meta learning, and other domains will likely be able to drive the state-of-the-art ahead.

cs.LG↗

arXiv2021

Dynamical Phases and Resonance Phenomena in Information-Processing Recurrent Neural Networks

Recurrent neural networks (RNNs) are complex dynamical systems, capable of ongoing activity without any driving input. The long-term behavior of free-running RNNs, described by periodic, chaotic and fixed point attractors, is controlled by the statistics of the neural connection weights, such as the density $d$ of non-zero connections, or the balance $b$ between excitatory and inhibitory connections. However, for information processing purposes, RNNs need to receive external input signals, and it is not clear which of the dynamical regimes is optimal for this information import. We use both the average correlations $C$ and the mutual information $I$ between the momentary input vector and the next system state vector as quantitative measures of information import and analyze their dependence on the balance and density of the network. Remarkably, both resulting phase diagrams $C(b,d)$ and $I(b,d)$ are highly consistent, pointing to a link between the dynamical systems and the information-processing approach to complex systems. Information import is maximal not at the 'edge of chaos', which is optimally suited for computation, but surprisingly in the low-density chaotic regime and at the border between the chaotic and fixed point regime. Moreover, we find a completely new type of resonance phenomenon, called 'Import Resonance' (IR), where the information import shows a maximum, i.e. a peak-like dependence on the coupling strength between the RNN and its input. IR complements Recurrence Resonance (RR), where correlation and mutual information of successive system states peak for a certain amplitude of noise added to the system. Both IR and RR can be exploited to optimize information processing in artificial neural networks and might also play a crucial role in biological neural systems.

q-bio.NC↗

arXiv2020

Towards a Cognitive Computational Neuroscience of Auditory Phantom Perceptions

In order to gain a mechanistic understanding of how tinnitus emerges in the brain, we must build biologically plausible computational models that mimic both tinnitus development and perception, and test the tentative models with brain and behavioral experiments. With a special focus on tinnitus research, we review recent work at the intersection of artificial intelligence, psychology and neuroscience, indicating a new research agenda that follows the idea that experiments will yield theoretical insight only when employed to test brain-computational models. This view challenges the popular belief, that tinnitus research is primarily data limited, and that producing large, multi-modal, and complex datasets, analyzed with advanced data analysis algorithms, will finally lead to fundamental insights into how tinnitus emerges. However, there is converging evidence that although modern technologies allow assessing neural activity in unprecedentedly rich ways in both, animals and humans, empirical testing one verbally defined hypothesis about tinnitus after another, will never lead to a mechanistic understanding. Instead, hypothesis testing needs to be complemented with the construction of computational models that generate verifiable predictions. We argue, that even though, contemporary artificial intelligence and machine learning approaches largely lack biological plausibility, the models to be constructed will have to draw on concepts from these fields, since they have already proven to do well in modeling brain function. Nevertheless, biological fidelity will have to be increased successively, leading to ever better and fine-grained models, allowing at the end for even testing possible treatment strategies in silico, before application in animal or patient studies.

q-bio.NC↗

arXiv2020

Will we ever have Conscious Machines?

The question of whether artificial beings or machines could become self-aware or consciousness has been a philosophical question for centuries. The main problem is that self-awareness cannot be observed from an outside perspective and the distinction of whether something is really self-aware or merely a clever program that pretends to do so cannot be answered without access to accurate knowledge about the mechanism's inner workings. We review the current state-of-the-art regarding these developments and investigate common machine learning approaches with respect to their potential ability to become self-aware. We realise that many important algorithmic steps towards machines with a core consciousness have already been devised. For human-level intelligence, however, many additional techniques have to be discovered.

cs.AI↗

arXiv2020

Sparsity through evolutionary pruning prevents neuronal networks from overfitting

Modern Machine learning techniques take advantage of the exponentially rising calculation power in new generation processor units. Thus, the number of parameters which are trained to resolve complex tasks was highly increased over the last decades. However, still the networks fail - in contrast to our brain - to develop general intelligence in the sense of being able to solve several complex tasks with only one network architecture. This could be the case because the brain is not a randomly initialized neural network, which has to be trained by simply investing a lot of calculation power, but has from birth some fixed hierarchical structure. To make progress in decoding the structural basis of biological neural networks we here chose a bottom-up approach, where we evolutionarily trained small neural networks in performing a maze task. This simple maze task requires dynamical decision making with delayed rewards. We were able to show that during the evolutionary optimization random severance of connections lead to better generalization performance of the networks compared to fully connected networks. We conclude that sparsity is a central property of neural networks and should be considered for modern Machine learning approaches.

cs.NE↗

arXiv2019

How deep is deep enough? -- Quantifying class separability in the hidden layers of deep neural networks

Deep neural networks typically outperform more traditional machine learning models in their ability to classify complex data, and yet is not clear how the individual hidden layers of a deep network contribute to the overall classification performance. We thus introduce a Generalized Discrimination Value (GDV) that measures, in a non-invasive manner, how well different data classes separate in each given network layer. The GDV can be used for the automatic tuning of hyper-parameters, such as the width profile and the total depth of a network. Moreover, the layer-dependent GDV(L) provides new insights into the data transformations that self-organize during training: In the case of multi-layer perceptrons trained with error backpropagation, we find that classification of highly complex data sets requires a temporal {\em reduction} of class separability, marked by a characteristic 'energy barrier' in the initial part of the GDV(L) curve. Even more surprisingly, for a given data set, the GDV(L) is running through a fixed 'master curve', independently from the total number of network layers. Furthermore, applying the GDV to Deep Belief Networks reveals that also unsupervised training with the Contrastive Divergence method can systematically increase class separability over tens of layers, even though the system does not 'know' the desired class labels. These results indicate that the GDV may become a useful tool to open the black box of deep learning.

cs.LG↗

arXiv2018

Stochastic resonance in three-neuron motifs

Stochastic resonance is a non-linear phenomenon, in which the sensitivity of signal detectors can be enhanced by adding random noise to the detector input. Here, we demonstrate that noise can also improve the information flux in recurrent neural networks. In particular, we show for the case of three-neuron motifs that the mutual information between successive network states can be maximized by adding a suitable amount of noise to the neuron inputs. This striking result suggests that noise in the brain may not be a problem that needs to be suppressed, but indeed a resource that is dynamically regulated in order to optimize information processing.

q-bio.NC↗

arXiv2018

Analysis of structure and dynamics in three-neuron motifs

In neural networks with identical neurons, the matrix of connection weights completely describes the network structure and thereby determines how it is processing information. However, due to the non-linearity of these systems, it is not clear if similar microscopic connection structures also imply similar functional properties, or if a network is impacted more by macroscopic structural quantities, such as the ratio of excitatory and inhibitory connections (balance), or the ratio of non-zero connections (density). To clarify these questions, we focus on motifs of three binary neurons with discrete ternary connection strengths, an important class of network building blocks that can be analyzed exhaustively. We develop new, permutation-invariant metrics to quantify the structural and functional distance between two given network motifs. We then use multidimensional scaling to identify and visualize clusters of motifs with similar structural and functional properties. Our comprehensive analysis reveals that the function of a neural network is only weakly correlated with its microscopic structure, but depends strongly on the balance of the connections.

q-bio.NC↗

arXiv2018

Objective estimation of sensory thresholds based on neurophysiological parameters

Reliable determination of sensory thresholds is the holy grail of signal detection theory. However, there exists no gold standard for the estimation of thresholds based on neurophysiological parameters, although a reliable estimation method is crucial for both scientific investigations and clinical diagnosis. Whenever it is impossible to communicate with the subjects, as in studies with animals or neonatales, thresholds have to be derived from neural recordings. In such cases when the threshold is estimated based on neuronal measures, the standard approach is still the subjective setting of the threshold to the value where at least a "clear" neuronal signal is detectable. These measures are highly subjective, strongly depend on the noise, and fluctuate due to the low signal-to-noise ratio near the threshold. Here we show a novel method to reliably estimate physiological thresholds based on neurophysiological parameters. Using surrogate data, we demonstrate that fitting the responses to different stimulus intensities with a hard sigmoid function, in combination with subsampling, provides a robust threshold value as well as an accurate uncertainty estimate. This method has no systematic dependence on the noise and does not even require samples in the full dynamic range of the sensory system. It is universally applicable to all types of sensory systems, ranging from somatosensory stimulus processing in the cortex to auditory processing in the brain stem.

q-bio.QM↗

arXiv2018

Open(G)PIAS: An open source solution for the construction of a high-precision Acoustic-Startle-Response (ASR) setup for tinnitus screening and threshold estimation in rodents

The acoustic startle reflex (ASR) that can be induced by a loud sound stimulus can be used as a versatile tool to, e.g., estimate hearing thresholds or identify subjective tinnitus percepts in rodents. These techniques are based on the fact that the ASR amplitude can be suppressed by a pre-stimulus of lower, non-startling intensity, an effect named pre-pulse inhibition (PPI). For hearing threshold estimation, pure tone pre-stimuli of varying amplitudes are presented before an intense noise burst serving as startle stimulus. The amount of suppression of the ASR amplitude as a function of the pre-stimulus intensity can be used as a behavioral correlate to determine the hearing ability. For tinnitus assessment, the pure-tone pre-stimulus is replaced by a gap of silence in a narrowband noise background, a paradigm termed GPIAS (gap-pre-pulse inhibition of the acoustic startle response). A proper application of these paradigms depend on a reliable measurement of the ASR amplitudes, an exact stimulus presentation in terms of frequency and intensity. Here we introduce a novel open source solution for the construction of a low-cost ASR setup for the above mentioned purpose. The complete software for data acquisition and stimulus presentation is written in Python 3.6 and is provided as an anaconda package. Furthermore, we provide a construction plan for the sensory system based on low-cost hardware components. Exemplary data show that the ratios (1-PPI) of the pre and post trauma ASR amplitudes can be well described by a lognormal distribution being in good accordance to previous studies with already established setups. Hence, the open access solution described here will help to further establish the ASR method in many laboratories and thus facilitate and standardize research in animal models of tinnitus or hearing loss.

q-bio.NC↗

arXiv2016

A statistical method for analyzing and comparing spatiotemporal cortical activation patterns

We present a new statistical method to analyze multichannel steady-state local field potentials (LFP) recorded within different sensory cortices of different rodent species. Our spatiotemporal multi-dimensional cluster statistics (MCS) method enables statistical analyzing and comparing clusters of data points in n-dimensional space. We demonstrate that using this approach stimulus-specific attractor-like spatiotemporal activity patterns can be detected and be significantly different from each other during stimulation with long-lasting stimuli. Our method may be applied to other types of multichannel neuronal data, like EEG, MEG or spiking responses and used for the development of new read-out algorithms of brain activity and by that opens new perspectives for the development of brain-computer interfaces.

q-bio.QM↗

arXiv2016

Stochastic resonance controlled upregulation of internal noise after hearing loss as a putative correlate of tinnitus-related neuronal hyperactivity

Subjective tinnitus (ST) is generally assumed to be a consequence of hearing loss (HL). In animal studies acoustic trauma can lead to behavioral signs of ST, in human studies ST patients without increased hearing thresholds were found to suffer from so called hidden HL. Additionally, ST is correlated with pathologically increased spontaneous firing rates and neuronal hyperactivity (NH) along the auditory pathway. Homeostatic plasticity (HP) has been proposed as a compensation mechanism leading to the development of NH, arguing that after HL initially decreased mean firing rates of neurons are subsequently restored by increased spontaneous rates. However all HP models fundamentally lack explanatory power since the function of keeping mean firing rate constant remains elusive as does the benefit this might have in terms of information processing. Furthermore the neural circuitry being able to perform the comparison of preferred with actual mean firing rate remains unclear. Here we propose an entirely new interpretation of ST related development of NH in terms of information theory. We suggest that stochastic resonance (SR) plays a key role in short- and long-term plasticity within the auditory system and is the ultimate cause of NH and ST. SR has been found ubiquitous in neuroscience and refers to the phenomenon that sub-threshold, unperceivable signals can be transmitted by adding noise to sensor input. We argue that after HL, SR serves to lift signals above the increased hearing threshold, hence subsequently decreasing thresholds again. The increased amount of internal noise is the correlate of the NH, which finally leads to the development of ST, due to neuronal plasticity along the auditory pathway. We demonstrate the plausibility of our hypothesis by using a computational model and provide exemplarily findings of human and animal studies that are consistent with our model.

q-bio.QM↗

arXiv2015

Adaptive stochastic resonance based on output autocorrelations

Successful detection of weak signals is a universal challenge for numerous technical and biological systems and crucially limits signal transduction and transmission. Stochastic resonance (SR) has been identified to have the potential to tackle this problem, namely to enable non-linear systems to detect small, otherwise sub-threshold signals by means of added non-zero noise. This has been demonstrated within a wide range of systems in physical, technological and biological contexts. Based on its ubiquitous importance, numerous theoretical and technical approaches aim at an optimization of signal transduction based on SR. Several quantities like mutual information, signal-to-noise-ratio, or the cross-correlation between input stimulus and resulting detector response have been used to determine optimal noise intensities for SR. The fundamental shortcoming with all these measures is that knowledge of the signal to be detected is required to compute them. This dilemma prevents the use of adaptive SR procedures in any application where the signal to be detected is unknown. We here show that the autocorrelation function (AC) of the detector response fundamentally overcomes this drawback. For a simplified model system, the equivalence of the output AC with the measures mentioned above is proven analytically. In addition, we test our approach numerically for a variety of systems comprising different input signals and different types of detectors. The results indicate a strong similarity between mutual information and output AC in terms of the optimal noise intensity for SR. Hence, using the output AC to adaptively vary the amount of added noise in order to maximize information transmission via SR might be a fundamental processing principle in nature, in particular within neural systems which could be implemented in future technical applications.

cs.IT↗

arXiv2011

Reconstructing fiber networks from confocal image stacks

We present a numerically efficient method to reconstruct a disordered network of thin biopolymers, such as collagen gels, from three-dimensional (3D) image stacks recorded with a confocal microscope. Our method is based on a template matching algorithm that simultaneously performs a binarization and skeletonization of the network. The size and intensity pattern of the template is automatically adapted to the input data so that the method is scale invariant and generic. Furthermore, the template matching threshold is iteratively optimized to ensure that the final skeletonized network obeys a universal property of voxelized random line networks, namely, solid-phase voxels have most likely three solid-phase neighbors in a $3\times3$ neighborhood. This optimization criterion makes our method free of user-defined parameters and the output exceptionally robust against imaging noise.

q-bio.QM↗

arXiv2011

Poresizes in random line networks

Many natural fibrous networks with fiber diameters much smaller than the average poresize can be described as three-dimensional (3D) random line networks. We consider here a `Mikado' model for such systems, consisting of straight line segments of equal length, distributed homogeneously and isotropically in space. First, we derive analytically the probability density distribution $p(r_{no})$ for the `nearest obstacle distance' $r_{no}$ between a randomly chosen test point within the network pores and its closest neighboring point on a line segment. Second, we show that in the limit where the line segments are much longer than the typical pore size, $p(r_{no})$ becomes a Rayleigh distribution. The single parameter $σ$ of this Rayleigh distribution represents the most probable nearest obstacle distance and can be expressed in terms of the total line length per unit volume. Finally, we show by numerical simulations that $σ$ differs only by a constant factor from the intuitive notion of average `pore size', defined by finding the maximum sphere that fits into each pore and then averaging over the radii of these spheres.

q-bio.QM↗