arXiv subjects

Erik Bollt

Publications and source records attributed to Erik Bollt.

At least 19 recordsLinked to original sources

arXiv2026

Deep-Koopman-KANDy: Dictionary Discovery for Deep-Koopman Operators with Kolmogorov-Arnold Networks for Dynamics

Symbolic library -- or Koopman dictionary -- selection is a fundamental challenge in data-driven dynamical systems. Extended Dynamic Mode Decomposition (EDMD), Sparse Identification of Nonlinear Dynamics (SINDy), and Kolmogorov--Arnold Networks for Dynamics (KANDy) all require the practitioner to commit to a function library at training time; Deep-Koopman Operators avoid this commitment but produce uninterpretable latent observables. We propose Deep-Koopman-KANDy, a structured approach to post-hoc symbolic dictionary readout that combines Deep-Koopman modeling with Kolmogorov-Arnold Networks for Dynamics (KANDy). The encoder and decoder of a Deep-Koopman Operator are replaced with two-layer Kolmogorov--Arnold Networks (KANs), and a level-set construction together with a chain-rule gradient identity exposes the compositional structure of the learned observables in a basis chosen \emph{after} training. We evaluate the method on the Lorenz system, the Chirikov standard map, the Ikeda map, and the Arnold cat map. On Lorenz it recovers the target dictionary $\{x,y,z,xy,xz\}$ with perfect recall and Jaccard score $0.79\pm0.06$; on the standard map it recovers a low-order Fourier basis matching the analytical structure; on Ikeda -- which has no sparse polynomial representation -- a misspecified polynomial readout still recovers the correct foliation coordinate $g\approx x^2+y^2$ together with a nontrivial outer function; and on the Arnold cat map -- used as a negative control because finite-dimensional Koopman closure is provably impossible -- the method fails to find a sparse closure, as expected.

math.DS↗

arXiv2026

KANDy: Kolmogorov-Arnold Networks and Dynamical System Discovery

We introduce the Kolmogorov-Arnold Network for Dynamics (KANDy) as a zero-depth, wide neural architecture capable of discovering governing equations in chaotic and complex dynamical systems. Building on the foundation of Kolmogorov-Arnold Networks (KANs), KANDy explicitly learns governing equations by replacing sparse regression with a KAN. The synthesis of KANs and sparse regression addresses the limitations of equation discovery for KANs applied to dynamical systems and overcomes cases where sparse regression is hindered by sparsity constraints. Additionally, we show that our model, applied to the Hopf Fibration, recovers topological structure, thereby improving coherence with attractor properties. We apply our model to discrete and continuous dynamical systems, as well as to chaotic partial differential equations (PDEs). These results position KANDy as an interpretable and effective alternative for data-driven modeling of nonlinear dynamical systems.

math.DS↗

arXiv2026

Assimilative Causal Inference

Causal inference is fundamental across scientific disciplines, yet existing methods struggle to capture instantaneous, time-evolving causal relationships in complex, high-dimensional systems. In this paper, assimilative causal inference (ACI) is developed, which is a methodological framework that leverages Bayesian data assimilation to trace causes backward from observed effects. ACI solves the inverse problem rather than quantifying forward influence. It uniquely identifies dynamic causal interactions without requiring observations of candidate causes, accommodates short datasets, and, in principle, can be implemented in high-dimensional settings by employing efficient data assimilation algorithms. Crucially, it provides online tracking of causal roles that may reverse intermittently and facilitates a mathematically rigorous criterion for the causal influence range, revealing how far effects propagate. The effectiveness of ACI is demonstrated by complex dynamical systems showcasing intermittency and extreme events. ACI opens valuable pathways for studying complex systems, where transient causal structures are critical.

cs.LG↗

arXiv2026

CausationEntropy: Pythonic Optimal Causation Entropy

Optimal Causation Entropy (oCSE) is a robust causal network modeling technique that reveals causal networks from dynamical systems and coupled oscillators, distinguishing direct from indirect paths. CausationEntropy is a Python package that implements oCSE and several of its significant optimizations and methodological extensions. In this paper, we introduce the version 1.1 release of CausationEntropy, which includes new synthetic data generators, plotting tools, and several advanced information-theoretical causal network discovery algorithms with criteria for estimating Gaussian, k-nearest neighbors (kNN), geometric k-nearest neighbors (geometric-kNN), kernel density (KDE) and Poisson entropic estimators. The package is easy to install from the PyPi software repository, is thoroughly documented, supplemented with extensive code examples, and is modularly structured to support future additions. The entire codebase is released under the MIT license and is available on GitHub and through PyPi Repository. We expect this package to serve as a benchmark tool for causal discovery in complex dynamical systems.

cs.LG↗

arXiv2025

How fragile is your network? More than you think

Graphs are pervasive in our everyday lives, with relevance to biology, the internet, and infrastructure, as well as numerous other applications. It is thus necessary to have an understanding as to how quickly a graph disintegrates, whether by random failure or by targeted attack. While much of the interest in this subject has been focused on targeted removal of nodes, there has been some recent interest in targeted edge removal. Here, we focus on how robust a graph is against edge removal. We define a measure of network fragility that relates the fraction of edges removed to the largest connected component. We construct a class of graphs that is robust to edge removal. Furthermore, it is demonstrated that graphs generally disintegrate faster than would be anticipated by greedy targeted attack. Finally it is shown that our fragility measure as demonstrated real and natural networks.

cs.SI↗

arXiv2025

On the emergence of numerical instabilities in Next Generation Reservoir Computing

Next Generation Reservoir Computing (NGRC) is a low-cost machine learning method for forecasting chaotic time series from data. Computational efficiency is crucial for scalable reservoir computing, requiring better strategies to reduce training cost. In this work, we uncover a connection between the numerical conditioning of the NGRC feature matrix -- formed by polynomial evaluations on time-delay coordinates -- and the long-term NGRC dynamics. We show that NGRC can be trained without regularization, reducing computational time. Our contributions are twofold. First, merging tools from numerical linear algebra and ergodic theory of dynamical systems, we systematically study how the feature matrix conditioning varies across hyperparameters. We demonstrate that the NGRC feature matrix tends to be ill-conditioned for short time lags, high-degree polynomials, and short length of training data. Second, we evaluate the impact of different numerical algorithms (Cholesky, singular value decomposition (SVD), and lower-upper (LU) decomposition) for solving the regularized least-squares problem. Our results reveal that SVD-based training achieves accurate forecasts without regularization, being preferable when compared against the other algorithms.

stat.ML↗

arXiv2025

Linear Stability Analysis of Physics-Informed Random Projection Neural Networks for ODEs

We present a linear stability analysis of physics-informed random projection neural networks (PI-RPNNs), for the numerical solution of {the initial value problem (IVP)} of (stiff) ODEs. We begin by proving that PI-RPNNs are uniform approximators of the solution to ODEs. We then provide a constructive proof demonstrating that PI-RPNNs offer consistent and asymptotically stable numerical schemes, thus convergent schemes. In particular, we prove that multi-collocation PI-RPNNs guarantee asymptotic stability. Our theoretical results are illustrated via numerical solutions of benchmark examples including indicative comparisons with the backward Euler method, the midpoint method, the trapezoidal rule, the 2-stage Gauss scheme, and the 2- and 3-stage Radau schemes.

math.NA↗

arXiv2025

Locality Blended Next Generation Reservoir Computing For Attention Accuracy

We extend an advanced variation of a machine learning algorithm, next-generation reservoir Computing (NGRC), to forecast the dynamics of the Ikeda map of a chaotic laser. The machine learning model is created by observing time-series data generated by the Ikeda map, and the trained model is used to forecast the behavior without any input from the map. The Ikeda map is a particularly challenging problem to learn because of the complicated map functions. We overcome the challenge by a novel improvement of the NGRC concept by emphasizing simpler polynomial models localized to well-designed regions of phase space and then blending these models between regions, a method that we call locality blended next-generation reservoir computing (LB-NGRC). This approach allows for better performance with relatively smaller data sets, and gives a new level of interpretability. We achieve forecasting horizons exceeding five Lyapunov times, and we demonstrate that the `climate' of the model is learned over long times.

nlin.CD↗

arXiv2024

Kolmogorov-Arnold Network Autoencoders

Deep learning models have revolutionized various domains, with Multi-Layer Perceptrons (MLPs) being a cornerstone for tasks like data regression and image classification. However, a recent study has introduced Kolmogorov-Arnold Networks (KANs) as promising alternatives to MLPs, leveraging activation functions placed on edges rather than nodes. This structural shift aligns KANs closely with the Kolmogorov-Arnold representation theorem, potentially enhancing both model accuracy and interpretability. In this study, we explore the efficacy of KANs in the context of data representation via autoencoders, comparing their performance with traditional Convolutional Neural Networks (CNNs) on the MNIST, SVHN, and CIFAR-10 datasets. Our results demonstrate that KAN-based autoencoders achieve competitive performance in terms of reconstruction accuracy, thereby suggesting their viability as effective tools in data analysis tasks.

cs.LG↗

arXiv2024

Entropic Regression DMD (ERDMD) Discovers Informative Sparse and Nonuniformly Time Delayed Models

In this work, we present a method which determines optimal multi-step dynamic mode decomposition (DMD) models via entropic regression, which is a nonlinear information flow detection algorithm. Motivated by the higher-order DMD (HODMD) method of \cite{clainche}, and the entropic regression (ER) technique for network detection and model construction found in \cite{bollt, bollt2}, we develop a method that we call ERDMD that produces high fidelity time-delay DMD models that allow for nonuniform time space, and the time spacing is discovered by consider most informativity based on ER. These models are shown to be highly efficient and robust. We test our method over several data sets generated by chaotic attractors and show that we are able to build excellent reconstructions using relatively minimal models. We likewise are able to better identify multiscale features via our models which enhances the utility of dynamic mode decomposition.

stat.ML↗

arXiv2024

On Learning what to Learn: heterogeneous observations of dynamics and establishing (possibly causal) relations among them

Before we attempt to learn a function between two (sets of) observables of a physical process, we must first decide what the inputs and what the outputs of the desired function are going to be. Here we demonstrate two distinct, data-driven ways of initially deciding ``the right quantities'' to relate through such a function, and then proceed to learn it. This is accomplished by processing multiple simultaneous heterogeneous data streams (ensembles of time series) from observations of a physical system: multiple observation processes of the system. We thus determine (a) what subsets of observables are common between the observation processes (and therefore observable from each other, relatable through a function); and (b) what information is unrelated to these common observables, and therefore particular to each observation process, and not contributing to the desired function. Any data-driven function approximation technique can subsequently be used to learn the input-output relation, from k-nearest neighbors and Geometric Harmonics to Gaussian Processes and Neural Networks. Two particular ``twists'' of the approach are discussed. The first has to do with the identifiability of particular quantities of interest from the measurements. We now construct mappings from a single set of observations of one process to entire level sets of measurements of the process, consistent with this single set. The second attempts to relate our framework to a form of causality: if one of the observation processes measures ``now'', while the second observation process measures ``in the future'', the function to be learned among what is common across observation processes constitutes a dynamical model for the system evolution.

cs.LG↗

arXiv2024

Symmetry breaker governs synchrony patterns in neuronal inspired networks

Experiments in the human brain reveal switching between different activity patterns and functional network organization over time. Recently, multilayer modeling has been employed across multiple neurobiological levels (from spiking networks to brain regions) to unveil novel insights into the emergence and time evolution of synchrony patterns. We consider two layers with the top layer directly coupled to the bottom layer. When isolated, the bottom layer would remain in a specific stable pattern. However, in the presence of the top layer, the network exhibits spatiotemporal switching. The top layer in combination with the inter-layer coupling acts as a symmetry breaker, governing the bottom layer and restricting the number of allowed symmetry-induced patterns. This structure allows us to demonstrate the existence and stability of pattern states on the bottom layer, but most remarkably, it enables a simple mechanism for switching between patterns based on the unique symmetry-breaking role of the governing layer. We demonstrate that the symmetry breaker prevents complete synchronization in the bottom layer, a situation that would not be desirable in a normal functioning brain. We illustrate our findings using two layers of Hindmarsh-Rose (HR) oscillators, employing the Master Stability function approach in small networks to investigate the switching between patterns.

math.DS↗

arXiv2024

A Causation-Based Computationally Efficient Strategy for Deploying Lagrangian Drifters to Improve Real-Time State Estimation

Deploying Lagrangian drifters that facilitate the state estimation of the underlying flow field within a future time interval is practically important. However, the uncertainty in estimating the flow field prevents using standard deterministic approaches for designing strategies and applying trajectory-wise skill scores to evaluate performance. In this paper an information measurement is developed to quantitatively assess the information gain in the estimated flow field by deploying an additional set of drifters. This information measurement is derived by exploiting causal inference. It is characterized by the inferred probability density function of the flow field, which naturally considers the uncertainty. Although the information measurement is an ideal theoretical metric, using it as the direct cost makes the optimization problem computationally expensive. To this end, an effective surrogate cost function is developed. It is highly efficient to compute while capturing the essential features of the information measurement when solving the optimization problem. Based upon these properties, a practical strategy for deploying drifter observations to improve future state estimation is designed. Due to the forecast uncertainty, the approach exploits the expected value of spatial maps of the surrogate cost associated with different forecast realizations to seek the optimal solution. Numerical experiments justify the effectiveness of the surrogate cost. The proposed strategy significantly outperforms the method by randomly deploying the drifters. It is also shown that, under certain conditions, the drifters determined by the expected surrogate cost remain skillful for the state estimation of a single forecast realization of the flow field as in reality.

math.DS↗

arXiv2023

Fractal Basins as a Mechanism for the Nimble Brain

An interesting feature of the brain is its ability to respond to disparate sensory signals from the environment in unique ways depending on the environmental context or current brain state. In dynamical systems, this is an example of multi-stability, the ability to switch between multiple stable states corresponding to specific patterns of brain activity/connectivity. In this article, we describe chimera states, which are patterns consisting of mixed synchrony and incoherence, in a brain-inspired dynamical systems model composed of a network with weak individual interactions and chaotic/periodic local dynamics. We illustrate the mechanism using synthetic time series interacting on a realistic anatomical brain network derived from human diffusion tensor imaging (DTI). We introduce the so-called Vector Pattern State (VPS) as an efficient way of identifying chimera states and mapping basin structures. Clustering similar VPSs for different initial conditions, we show that coexisting attractors of such states reveal intricately "mingled" fractal basin boundaries that are immediately reachable. This could explain the nimble brain's ability to rapidly switch patterns between coexisting attractors.

math.DS↗

arXiv2023

Machine Learning Enhanced Hankel Dynamic-Mode Decomposition

While the acquisition of time series has become more straightforward, developing dynamical models from time series is still a challenging and evolving problem domain. Within the last several years, to address this problem, there has been a merging of machine learning tools with what is called the dynamic mode decomposition (DMD). This general approach has been shown to be an especially promising avenue for accurate model development. Building on this prior body of work, we develop a deep learning DMD based method which makes use of the fundamental insight of Takens' Embedding Theorem to build an adaptive learning scheme that better approximates higher dimensional and chaotic dynamics. We call this method the Deep Learning Hankel DMD (DLHDMD). We likewise explore how our method learns mappings which tend, after successful training, to significantly change the mutual information between dimensions in the dynamics. This appears to be a key feature in enhancing the DMD overall, and it should help provide further insight for developing other deep learning methods for time series analysis and model generation.

cs.LG↗

arXiv2021

Melnikov theory for two-dimensional manifolds in three-dimensional flows

We present a Melnikov method to analyze two-dimensional stable or unstable manifolds associated with a saddle point in three-dimensional non-volume preserving autonomous systems. The time-varying perturbed locations of such manifolds is obtained under very general, non-volume preserving and with arbitrary time-dependence, perturbations. In unperturbed situations with a two-dimensional heteroclinic manifold, we adapt our theory to quantify the splitting into a stable and unstable manifold, and thereby obtain a Melnikov function characterizing the time-varying locations of transverse intersections of these manifolds. Formulas for lobe volumes arising from such intersections, as well as the instantaneous flux across the broken heteroclinic manifold, are obtained in terms of the Melnikov function. Our theory has specific application to transport in fluid mechanics, where the flow is in three dimensions and flow separators are two-dimensional stable/unstable manifolds. We demonstrate our theory using both the classical and the swirling versions of Hill's spherical vortex.

math.DS↗

arXiv2021

Interaction Networks from Discrete Event Data by Poisson Multivariate Mutual Information Estimation and Information Flow with Applications from Gene Expression Data

In this work, we introduce a new methodology for inferring the interaction structure of discrete valued time series which are Poisson distributed. While most related methods are premised on continuous state stochastic processes, in fact, discrete and counting event oriented stochastic process are natural and common, so called time-point processes (TPP). An important application that we focus on here is gene expression. Nonparameteric methods such as the popular k-nearest neighbors (KNN) are slow converging for discrete processes, and thus data hungry. Now, with the new multi-variate Poisson estimator developed here as the core computational engine, the causation entropy (CSE) principle, together with the associated greedy search algorithm optimal CSE (oCSE) allows us to efficiently infer the true network structure for this class of stochastic processes that were previously not practical. We illustrate the power of our method, first in benchmarking with synthetic datum, and then by inferring the genetic factors network from a breast cancer micro-RNA (miRNA) sequence count data set. We show the Poisson oCSE gives the best performance among the tested methods anfmatlabd discovers previously known interactions on the breast cancer data set.

stat.ME↗

arXiv2021

Next Generation Reservoir Computing

Reservoir computing is a best-in-class machine learning algorithm for processing information generated by dynamical systems using observed time-series data. Importantly, it requires very small training data sets, uses linear optimization, and thus requires minimal computing resources. However, the algorithm uses randomly sampled matrices to define the underlying recurrent neural network and has a multitude of metaparameters that must be optimized. Recent results demonstrate the equivalence of reservoir computing to nonlinear vector autoregression, which requires no random matrices, fewer metaparameters, and provides interpretable results. Here, we demonstrate that nonlinear vector autoregression excels at reservoir computing benchmark tasks and requires even shorter training data sets and training time, heralding the next generation of reservoir computing.

cs.LG↗