arXiv subjects

Sarah Hallerberg

Publications and source records attributed to Sarah Hallerberg.

13 recordsLinked to original sources

arXiv2024

On the weight dynamics of learning networks

Neural networks have become a widely adopted tool for tackling a variety of problems in machine learning and artificial intelligence. In this contribution we use the mathematical framework of local stability analysis to gain a deeper understanding of the learning dynamics of feed forward neural networks. Therefore, we derive equations for the tangent operator of the learning dynamics of three-layer networks learning regression tasks. The results are valid for an arbitrary numbers of nodes and arbitrary choices of activation functions. Applying the results to a network learning a regression task, we investigate numerically, how stability indicators relate to the final training-loss. Although the specific results vary with different choices of initial conditions and activation functions, we demonstrate that it is possible to predict the final training loss, by monitoring finite-time Lyapunov exponents or covariant Lyapunov vectors during the training process.

cs.LG↗

arXiv2021

Estimating covariant Lyapunov vectors from data

Covariant Lyapunov vectors characterize the directions along which perturbations in dynamical systems grow. They have also been studied as predictors of critical transitions and extreme events. For many applications like, for example, prediction, it is necessary to estimate the vectors from data since model equations are unknown for many interesting phenomena. We propose a novel method for estimating covariant Lyapunov vectors based on data records without knowing the underlying equations of the system. In contrast to previous approaches, our approach can be applied to high-dimensional data-sets. We demonstrate that this purely data-driven approach can accurately estimate covariant Lyapunpov vectors from data records generated by low and high-dimensional dynamical systems. The highest dimension of a time-series from which covariant Lyapunov vectors were estimated in this contribution is 128. Being able to infer covariant Lyapunov vectors from data-records could encourage numerous future applications in data-analysis and data-based predictions.

physics.data-an↗

arXiv2018

Fluctuation-induced Distributed Resonances in Oscillatory Networks

Self-organized network dynamics prevails for systems across physics, biology and engineering. How external signals generate distributed responses in networked systems fundamentally underlies their function, yet is far from fully understood. Here we analyze the dynamic response patterns of oscillatory networks to fluctuating input signals. We disentangle the impact of the signal distribution across the network, the signals' frequency contents and the network topology. We analytically derive qualitatively different dynamic response patterns and find three frequency regimes: homogeneous responses at low frequencies, topology-dependent resonances at intermediate frequencies, and localized responses at high frequencies. The theory faithfully predicts the network-wide collective responses to regular and irregular, localized and distributed simulated signals, as well as to real input signals to power grids recorded from renewable-energy supplies. These results not only provide general insights into the formation of dynamic response patterns in networked systems but also suggest regime- and topology-specific design principles underlying network function.

nlin.AO↗

arXiv2018

Model-free inference of direct network interactions from nonlinear collective dynamics

The topology of interactions in network dynamical systems fundamentally underlies their function. Accelerating technological progress creates massively available data about collective nonlinear dynamics in physical, biological, and technological systems. Detecting direct interaction patterns from those dynamics still constitutes a major open problem. In particular, current nonlinear dynamics approaches mostly require to know a priori a model of the (often high dimensional) system dynamics. Here we develop a model-independent framework for inferring direct interactions solely from recording the nonlinear collective dynamics generated. Introducing an explicit dependency matrix in combination with a block-orthogonal regression algorithm, the approach works reliably across many dynamical regimes, including transient dynamics toward steady states, periodic and non-periodic dynamics, and chaos. Together with its capabilities to reveal network (two point) as well as hypernetwork (e.g., three point) interactions, this framework may thus open up nonlinear dynamics options of inferring direct interaction patterns across systems where no model is known.

physics.soc-ph↗

arXiv2017

Critical Transitions and Perturbation Growth Directions

Critical transitions occur in a variety of dynamical systems. Here, we employ quantifiers of chaos to identify changes in the dynamical structure of complex systems preceding critical transitions. As suitable indicator variables for critical transitions, we consider changes in growth rates and directions of covariant Lyapunov vectors. Studying critical transitions in several models of fast-slow systems, i.e., a network of coupled FitzHugh-Nagumo oscillators, models for Josephson junctions and the Hindmarsh-Rose model, we find that tangencies between covariant Lyapunov vectors are a common and maybe generic feature during critical transitions. We further demonstrate that this deviation from hyperbolic dynamics is linked to the occurrence of critical transitions by using it as an indicator variable and evaluating the prediction success through receiver operating characteristic curves. In the presence of noise, we find the alignment of covariant Lyapunov vectors and changes in finite-time Lyapunov exponents to be more successful in announcing critical transitions than common indicator variables as, e.g., finite-time estimates of the variance. Additionally, we propose a new method for estimating approximations of covariant Lyapunov vectors without knowledge of the future trajectory of the system. We find that these approximated covariant Lyapunov vectors can also be applied to predict critical transitions.

nlin.CD↗

arXiv2017

High-Intensity Discharge Lamp and Duffing Oscillator - Similarities and Differences

The processes inside the arc tube of high-intensity discharge lamps are investigated by finite element simulations. The behavior of the gas mixture inside the arc tube is governed by differential equations describing mass, energy and charge conservation as well as the Helmholtz equation for the acoustic pressure and the Navier-Stokes equation for the flow driven by the buoyancy and the acoustic streaming force. The model is highly nonlinear and requires a recursion procedure to account for the impact of acoustic streaming on the temperature and other fields. The investigations reveal the presence of a hysteresis and the corresponding jump phenomenon, quite similar to a Duffing oscillator. The similarities and, in particular, the differences of the nonlinear behavior of the high-intensity discharge lamp to that of a Duffing oscillator are discussed. For large amplitudes the high-intensity discharge lamp exhibits a stiffening effect in contrast to the Duffing oscillator.

physics.app-ph↗

arXiv2016

Network susceptibilities: theory and applications

We introduce the concept of network susceptibilities quantifying the response of the collective dy- namics of a network to small parameter changes. We distinguish two types of susceptibilities: vertex susceptibilities and edge susceptibilities, measuring the responses due to changes in the properties of units and their interactions, respectively. We derive explicit forms of network susceptibilities for oscillator networks close to steady states and offer example applications for Kuramoto-type phase- oscillator models, power grid models and generic flow models. Focusing on the role of the network topology implies that these ideas can be easily generalized to other types of networks, in particular those characterizing flow, transport, or spreading phenomena. The concept of network susceptibil- ities is broadly applicable and may straightforwardly be transferred to all settings where networks responses of the collective dynamics to topological changes are essential.

nlin.AO↗

arXiv2015

Predictability of Critical Transitions

Critical transitions in multistable systems have been discussed as models for a variety of phenomena ranging from the extinctions of species to socio-economic changes and climate transitions between ice-ages and warm-ages. From bifurcation theory we can expect certain critical transitions to be preceded by a decreased recovery from external perturbations. The consequences of this critical slowing down have been observed as an increase in variance and autocorrelation prior to the transition. However especially in the presence of noise it is not clear, whether these changes in observation variables are statistically relevant such that they could be used as indicators for critical transitions. In this contribution we investigate the predictability of critical transitions in conceptual models. We study the quadratic integrate-and-fire model and the van der Pol model, under the influence of external noise. We focus especially on the statistical analysis of the success of predictions and the overall predictability of the system. The performance of different indicator variables turns out to be dependent on the specific model under study and the conditions of accessing it. Furthermore, we study the influence of the magnitude of transitions on the predictive performance.

physics.data-an↗

arXiv2015

Critical links and nonlocal rerouting in complex supply networks

Link failures repeatedly induce large-scale outages in power grids and other supply networks. Yet, it is still not well understood, which links are particularly prone to inducing such outages. Here we analyze how the nature and location of each link impact the network's capability to maintain stable supply. We propose two criteria to identify critical links on the basis of the topology and the load distribution of the network prior to link failure. They are determined via a link's redundant capacity and a renormalized linear response theory we derive. These criteria outperform critical link prediction based on local measures such as loads. The results not only further our understanding of the physics of supply networks in general. As both criteria are available before any outage from the state of normal operation, they may also help real-time monitoring of grid operation, employing counter-measures and support network planning and design.

physics.soc-ph↗

arXiv2015

Bag-of-calls analysis reveals group-specific vocal repertoire in long-finned pilot whales

Besides humans, several marine mammal species exhibit prerequisites to evolve language: high cognitive abilities, flexibility in vocal production and advanced social interactions. Here, we describe and analyse the vocal repertoire of long-finned pilot whales (Globicephalus melas) recorded in northern Norway. Observer based analysis reveals a complex vocal repertoire with 140 different call types, call sequences, call repetitions and group-specific differences in the usage of call types. Developing and applying a new automated analysis method, the bag-of-calls approach, we find that groups of pilot whales can be distinguished purely by statistical properties of their vocalisations. Comparing inter-and intra-group differences of ensembles of calls allows to identify and quantify group-specificity. Consequently, the bag-of-calls approach is a valid method to specify difference and concordance in acoustic communication in the absence of exact knowledge about signalers, which is common observing marine mammals under natural conditions.

q-bio.QM↗

arXiv2010

Logarithmic bred vectors in spatiotemporal chaos: structure and growth

Bred vectors are a type of finite perturbation used in prediction studies of atmospheric models that exhibit spatially extended chaos. We study the structure, spatial correlations, and the growth- rates of logarithmic bred vectors (which are constructed by using a given norm). We find that, after a suitable transformation, logarithmic bred vectors are roughly piecewise copies of the leading Lyapunov vector. This fact allows us to deduce a scaling law for the bred vector growth rate as a function of their amplitude. In addition, we relate growth rates with the spectrum of Lyapunov exponents corresponding to the most expanding directions. We illustrate our results with simulations of the Lorenz '96 model.

nlin.CD↗

arXiv2006

Precursors of extreme increments

We investigate precursors and predictability of extreme increments in a time series. The events we are focusing on consist in large increments within successive time steps. We are especially interested in understanding how the quality of the predictions depends on the strategy to choose precursors, on the size of the event and on the correlation strength. We study the prediction of extreme increments analytically in an AR(1) process, and numerically in wind speed recordings and long-range correlated ARMA data. We evaluate the success of predictions via receiver operator characteristics (ROC-curves). Furthermore, we observe an increase of the quality of predictions with increasing event size and with decreasing correlation in all examples. Both effects can be understood by using the likelihood ratio as a summary index for smooth ROC-curves.

physics.data-an↗

arXiv2005

Reactions to extreme events: moving threshold model

In spite of precautions to avoid the harmful effects of extreme events, we experience recurrently phenomena that overcome the preventive barriers. These barriers usually increase drastically right after the occurrence of such extreme events, but steadily decay in their absence. In this paper we consider a simple model that mimics the evolution of the protection barriers to study the efficiency of the system's reaction to extreme events and how it changes our perception of the sequence of extreme events itself. We obtain that the usual method of fighting extreme events introduces a periodicity in their occurrence and is generally less efficient than the use of a constant barrier. On the other hand, it shows a good adaptation to the presence of slow non-stationarities.

physics.data-an↗