arXiv subjects

Daniel Henrik Nevermann

Publications and source records attributed to Daniel Henrik Nevermann.

4 recordsLinked to original sources

arXiv2026

Distance generalization in transformers: why bother with positional encoding?

Out-of-distribution length generalization, namely to extrapolate a task from short to longer context, has been studied intensively for transformers. Here we focus on distance generalization, which probes performance when inter-token distances are changed between training and inference, while keeping a fixed context length. We construct two synthetic delay copy tasks, both involving finite distances between source and recall, where tokens are copied either fully or selectively, and test models on delays unseen during training. We address three questions: (A) Do positional encoding schemes such as RoPE and ALiBi improve distance resolution relative to no positional encoding (NoPE)? (B) How does data diversity, the number of inter-token distances seen in training, affect performance? (C) When is distance transfer learning positive or negative? We present a thorough investigation, finding that it is paramount to improve our understanding of the underlying mechanisms.

cs.CL↗

arXiv2024

How oscillations in SIRS epidemic models are affected by the distribution of immunity times

Models for resident infectious diseases, like the SIRS model, may settle into an endemic state with constant numbers of susceptible ($S$), infected ($I$) and recovered ($R$) individuals, where recovered individuals attain a temporary immunity to reinfection. For many infectious pathogens, infection dynamics may also show periodic outbreaks corresponding to a limit cycle in phase space. One way to reproduce oscillations in SIRS models is to include a non-exponential dwell-time distribution in the recovered state. Here, we study a SIRS model with a step-function-like kernel for the immunity time, mapping out the model's full phase diagram. Using the kernel series framework, we are able to identify the onset of periodic outbreaks when successively broadening the step-width. We further investigate the shape of the outbreaks, finding that broader steps cause more sinusoidal oscillations while more uniform immunity time distributions are related to sharper outbreaks occurring after extended periods of low infection activity. Our main results concern recovery distributions characterized by a single dominant timescale. We also consider recovery distributions with two timescales, which may be observed when two or more distinct recovery processes co-exist. Surprisingly, two qualitatively different limit cycles are found to be stable in this case, with only one of the two limit cycles emerging via a standard supercritical Hopf bifurcation.

q-bio.PE↗

arXiv2024

A game of life with dormancy

The factors contributing to the persistence and stability of life are fundamental for understanding complex living systems. Organisms are commonly challenged by harsh and fluctuating environments that are suboptimal for growth and reproduction, which can lead to extinction. Species often contend with unfavorable and noisy conditions by entering a reversible state of reduced metabolic activity, a phenomenon known as dormancy. Here, we develop Spore Life, a model to investigate the effects of dormancy on population dynamics. It is based on Conway's Game of Life, a deterministic cellular automaton where simple rules govern the metabolic state of an individual based on the metabolic state of its neighbors. For individuals that would otherwise die, Spore Life provides a refuge in the form of an inactive state. These dormant individuals (spores) can resuscitate when local conditions improve. The model includes a parameter alpha that controls the survival probability of spores, interpolating between Game of Life (alpha = 0) and Spore Life (alpha = 1), while capturing stochastic dynamics in the intermediate regime (0 < alpha < 1). In addition to identifying the emergence of unique periodic configurations, we find that spore survival increases the average number of active individuals and buffers populations from extinction. Contrary to expectations, the stabilization of the population is not the result of a large and long-lived seed bank. Instead, the demographic patterns in Spore Life only require a small number of resuscitation events. Our approach yields novel insight into what is minimally required for the emergence of complex behaviors associated with dormancy and the seed banks that they generate.

q-bio.PE↗

arXiv2023

Mapping dynamical systems with distributed time delays to sets of ordinary differential equations

Real-world dynamical systems with retardation effects are described in general not by a single, precisely defined time delay, but by a range of delay times. An exact mapping onto a set of $N+1$ ordinary differential equations exists when the respective delay distribution is given in terms of a gamma distribution with discrete exponents. The number of auxiliary variables one needs to introduce, $N$, is inversely proportional to the variance of the delay distribution. The case of a single delay is therefore recovered when $N\to\infty$. Using this approach, denoted here the `kernel series framework', we examine systematically how the bifurcation phase diagram of the Mackey-Glass system changes under the influence of distributed delays. We find that local properties, f.i.\ the locus of a Hopf bifurcation, are robust against the introduction of broadened memory kernels. Period-doubling transitions and the onset of chaos, which involve non-local properties of the flow, are found in contrast to be more sensitive to distributed delays. In general, the observed effects are found to scale as $1/N$. Furthermore, we consider time-delayed systems exhibiting chaotic diffusion, which is present in particular for sinusoidal flows. We find that chaotic diffusion is substantially more pronounced for distributed delays. Our results indicate in consequence that modeling approaches of real-world processes should take the effects of distributed delay times into account.

math.DS↗