arXiv subjects

Erik Bollt

Publications and source records attributed to Erik Bollt.

At least 37 records · Page 2Linked to original sources

arXiv2021

Geometric Considerations of a Good Dictionary for Koopman Analysis of Dynamical Systems: Cardinality, 'Primary Eigenfunction,' and Efficient Representation

Representation of a dynamical system in terms of simplifying modes is a central premise of reduced order modelling and a primary concern of the increasingly popular DMD (dynamic mode decomposition) empirical interpretation of Koopman operator analysis of complex systems. In the spirit of optimal approximation and reduced order modelling the goal of DMD methods and variants are to describe the dynamical evolution as a linear evolution in an appropriately transformed lower rank space, as best as possible. That Koopman eigenfunctions follow a linear PDE that is solvable by the method of characteristics yields several interesting relationships between geometric and algebraic properties. Corresponding to freedom to arbitrarily define functions on a data surface, for each eigenvalue, there are infinitely many eigenfunctions emanating along characteristics. We focus on contrasting cardinality and equivalence. In particular, we introduce an equivalence class, "primary eigenfunctions," consisting of those eigenfunctions with identical sets of level sets, that helps contrast algebraic multiplicity from other geometric aspects. Popularly, Koopman methods and notably dynamic mode decomposition (DMD) and variants, allow data-driven study of how measurable functions evolve along orbits. As far as we know, there has not been an in depth study regarding the underlying geometry as related to an efficient representation. We present a construction that leads to functions on the data surface whose corresponding eigenfunctions are efficient in a least squares sense. We call this construction optimal Koopman eigenfunction DMD, (oKEEDMD), and we highlight with examples.

math.DS↗

arXiv2021

On Explaining the Surprising Success of Reservoir Computing Forecaster of Chaos? The Universal Machine Learning Dynamical System with Contrasts to VAR and DMD

Machine learning has become a widely popular and successful paradigm, including in data-driven science and engineering. A major application problem is data-driven forecasting of future states from a complex dynamical. Artificial neural networks (ANN) have evolved as a clear leader amongst many machine learning approaches, and recurrent neural networks (RNN) are considered to be especially well suited for forecasting dynamical systems. In this setting, the echo state networks (ESN) or reservoir computer (RC) have emerged for their simplicity and computational complexity advantages. Instead of a fully trained network, an RC trains only read-out weights by a simple, efficient least squares method. What is perhaps quite surprising is that nonetheless an RC succeeds to make high quality forecasts, competitively with more intensively trained methods, even if not the leader. There remains an unanswered question as to why and how an RC works at all, despite randomly selected weights. We explicitly connect the RC with linear activation and linear read-out to well developed time-series literature on vector autoregressive averages (VAR) that includes theorems on representability through the WOLD theorem, which already perform reasonably for short term forecasts. In the case of a linear activation and now popular quadratic read-out RC, we explicitly connect to a nonlinear VAR (NVAR), which performs quite well. Further, we associate this paradigm to the now widely popular dynamic mode decomposition (DMD), and thus these three are in a sense different faces of the same thing. We illustrate our observations in terms of popular benchmark examples including Mackey-Glass differential delay equations and the Lorenz63 system.

physics.data-an↗

arXiv2021

Emergent hierarchy through conductance-based node constraints

The presence of hierarchy in many real-world networks is not yet fully explained. Complex interaction networks are often coarse-grain models of vast modular networks, where tightly connected subgraphs are agglomerated into nodes for simplicity of representation and feasibility of analysis. The emergence of hierarchy in growing complex networks may stem from one particular property of these ignored subgraphs: their graph conductance. Being a quantification of the main bottleneck of flow on the subgraph, all such subgraphs will then have a specific structural limitation associated with this scalar value. This supports the consideration of heterogeneous degree restrictions on a randomly growing network for which a hidden variable model is proposed based on the basic \textit{rich-get-richer} scheme. Such node degree restrictions are drawn from various probability distributions, and it is shown that restriction generally leads to increased measures of hierarchy, while altering the tail of the degree distribution. Thus, a general mechanism is provided whereby inherent limitations lead to hierarchical self-organization.

physics.soc-ph↗

arXiv2021

Entropic Causal Inference for Neurological Applications

The ultimate goal of cognitive neuroscience is to understand the mechanistic neural processes underlying the functional organization of the brain. Key to this study is understanding structure of both the structural and functional connectivity between anatomical regions. In this paper we follow previous work in developing a simple dynamical model of the brain by simulating its various regions as Kuramoto oscillators whose coupling structure is described by a complex network. However in our simulations rather than generating synthetic networks, we simulate our synthetic model but coupled by a real network of the anatomical brain regions which has been reconstructed from diffusion tensor imaging (DTI) data. By using an information theoretic approach that defines direct information flow in terms of causation entropy (CSE), we show that we can more accurately recover the true structural network than either of the popular correlation or LASSO regression techniques. We demonstrate the effectiveness of our method when applied to data simulated on the realistic DTI network, as well as on randomly generated small-world and Erdös-Rényi (ER) networks.

q-bio.NC↗

arXiv2021

Globally optimal stretching foliations of dynamical systems reveal the organizing skeleton of intensive instabilities

Understanding instabilities in dynamical systems drives to the heart of modern chaos theory, whether forecasting or attempting to control future outcomes. Instabilities in the sense of locally maximal stretching in maps is well understood, and is connected to the concepts of Lyapunov exponents/vectors, Oseledec spaces and the Cauchy--Green tensor. In this paper, we extend the concept to global optimization of stretching, as this forms a skeleton organizing the general instabilities. The `map' is general but incorporates the inevitability of finite-time as in any realistic application: it can be defined via a finite sequence of discrete maps, or a finite-time flow associated with a continuous dynamical system. Limiting attention to two-dimensions, we formulate the global optimization problem as one over a restricted class of foliations, and establish the foliations which both maximize and minimize global stretching. A classification of nondegenerate singularities of the foliations is obtained. Numerical issues in computing optimal foliations are examined, in particular insights into special curves along which foliations appear to veer and/or do not cross, and foliation behavior near singularities. Illustrations and validations of the results to the Hénon map, the double-gyre flow and the standard map are provided.

nlin.CD↗

arXiv2020

An Early Warning Sign of Critical Transition in The Antarctic Ice Sheet -- A Data Driven Tool for Spatiotemporal Tipping Point

Our recently developed tool, called Directed Affinity Segmentation was originally designed for data-driven discovery of coherent sets in fluidic systems. Here we interpret that it can also be used to indicate early warning signs of critical transitions in ice shelves as seen from remote sensing data. We apply a directed spectral clustering methodology, including an asymmetric affinity matrix and the associated directed graph Laplacian, to reprocess the ice velocity data and remote sensing satellite images of the Larsen C ice shelf. Our tool has enabled the simulated prediction of historical events from historical data, fault lines responsible for the critical transitions leading to the break up of the Larsen C ice shelf crack, which resulted in the A68 iceberg. Such benchmarking of methods using data from the past to forecast events that are now also in the past is sometimes called post-casting, analogous to forecasting into the future. Our method indicated the coming crisis months before the actual occurrence.

physics.ao-ph↗

arXiv2020

ERFit: Entropic Regression Fit Matlab Package, for Data-Driven System Identification of Underlying Dynamic Equations

Data-driven sparse system identification becomes the general framework for a wide range of problems in science and engineering. It is a problem of growing importance in applied machine learning and artificial intelligence algorithms. In this work, we developed the Entropic Regression Software Package (ERFit), a MATLAB package for sparse system identification using the entropic regression method. The code requires minimal supervision, with a wide range of options that make it adapt easily to different problems in science and engineering. The ERFit is available at https://github.com/almomaa/ERFit-Package

math.DS↗

arXiv2020

Data-Driven Learning of Boolean Networks and Functions by Optimal Causation Entropy Principle (BoCSE)

Boolean functions and networks are commonly used in the modeling and analysis of complex biological systems, and this paradigm is highly relevant in other important areas in data science and decision making, such as in the medical field and in the finance industry. Automated learning of a Boolean network and Boolean functions, from data, is a challenging task due in part to the large number of unknowns (including both the structure of the network and the functions) to be estimated, for which a brute force approach would be exponentially complex. In this paper we develop a new information theoretic methodology that we show to be significantly more efficient than previous approaches. Building on the recently developed optimal causation entropy principle (oCSE), that we proved can correctly infer networks distinguishing between direct versus indirect connections, we develop here an efficient algorithm that furthermore infers a Boolean network (including both its structure and function) based on data observed from the evolving states at nodes. We call this new inference method, Boolean optimal causation entropy (BoCSE), which we will show that our method is both computationally efficient and also resilient to noise. Furthermore, it allows for selection of a set of features that best explains the process, a statement that can be described as a networked Boolean function reduced order model. We highlight our method to the feature selection in several real-world examples: (1) diagnosis of urinary diseases, (2) Cardiac SPECT diagnosis, (3) informative positions in the game Tic-Tac-Toe, and (4) risk causality analysis of loans in default status. Our proposed method is effective and efficient in all examples.

cs.AI↗

arXiv2020

Informative Ranking of Stand Out Collections of Symptoms: A New Data-Driven Approach to Identify the Strong Warning Signs of COVID 19

We develop here a data-driven approach for disease recognition based on given symptoms, to be efficient tool for anomaly detection. In a clinical setting and when presented with a patient with a combination of traits, a doctor may wonder if a certain combination of symptoms may be especially predictive, such as the question, "Are fevers more informative in women than men?" The answer to this question is, yes. We develop here a methodology to enumerate such questions, to learn what are the stronger warning signs when attempting to diagnose a disease, called Conditional Predictive Informativity, (CPI), whose ranking we call CPIR. This simple to use process allows us to identify particularly informative combinations of symptoms and traits that may help medical field analysis in general, and possibly to become a new data-driven advised approach for individual medical diagnosis, as well as for broader public policy discussion. In particular we have been motivated to develop this tool in the current enviroment of the pressing world crisis due to the COVID 19 pandemic. We apply the methods here to data collected from national, provincial, and municipal health reports, as well as additional information from online, and then curated to an online publically available Github repository.

q-bio.PE↗

arXiv2019

How Entropic Regression Beats the Outliers Problem in Nonlinear System Identification

In this work, we developed a nonlinear System Identification (SID) method that we called Entropic Regression. Our method adopts an information-theoretic measure for the data-driven discovery of the underlying dynamics. Our method shows robustness toward noise and outliers and it outperforms many of the current state-of-the-art methods. Moreover, the method of Entropic Regression overcomes many of the major limitations of the current methods such as sloppy parameters, diverse scale, and SID in high dimensional systems such as complex networks. The use of information-theoretic measures in entropic regression poses unique advantages, due to the Asymptotic Equipartition Property (AEP) of probability distributions, that outliers and other low-occurrence events are conveniently and intrinsically de-emphasized as not-typical, by definition. We provide a numerical comparison with the current state-of-the-art methods in sparse regression, and we apply the methods to different chaotic systems such as the Lorenz System, the Kuramoto-Sivashinsky equations, and the Double Well Potential.

eess.SP↗

arXiv2018

Bayesian Optical Flow with Uncertainty Quantification

Optical flow refers to the visual motion observed between two consecutive images. Since the degree of freedom is typically much larger than the constraints imposed by the image observations, the straightforward formulation of optical flow as an inverse problem is ill-posed. Standard approaches to determine optical flow rely on formulating and solving an optimization problem that contains both a data fidelity term and a regularization term, the latter effectively resolves the otherwise ill-posedness of the inverse problem. In this work, we depart from the deterministic formalism, and instead treat optical flow as a statistical inverse problem. We discuss how a classical optical flow solution can be interpreted as a point estimate in this more general framework. The statistical approach, whose "solution" is a distribution of flow fields, which we refer to as Bayesian optical flow, allows not only "point" estimates (e.g., the computation of average flow field), but also statistical estimates (e.g., quantification of uncertainty) that are beyond any standard method for optical flow. As application, we benchmark Bayesian optical flow together with uncertainty quantification using several types of prescribed ground-truth flow fields and images.

stat.ML↗

arXiv2018

Anatomy of Leadership in Collective Behaviour

Understanding the mechanics behind the coordinated movement of mobile animal groups (collective motion) provides key insights into their biology and ecology, while also yielding algorithms for bio-inspired technologies and autonomous systems. It is becoming increasingly clear that many mobile animal groups are composed of heterogeneous individuals with differential levels and types of influence over group behaviors. The ability to infer this differential influence, or leadership, is critical to understanding group functioning in these collective animal systems. Due to the broad interpretation of leadership, many different measures and mathematical tools are used to describe and infer "leadership", e.g., position, causality, influence, information flow. But a key question remains: which, if any, of these concepts actually describes leadership? We argue that instead of asserting a single definition or notion of leadership, the complex interaction rules and dynamics typical of a group implies that leadership itself is not merely a binary classification (leader or follower), but rather, a complex combination of many different components. In this paper we develop an anatomy of leadership, identify several principle components and provide a general mathematical framework for discussing leadership. With the intricacies of this taxonomy in mind we present a set of leadership-oriented toy models that should be used as a proving ground for leadership inference methods going forward. We believe this multifaceted approach to leadership will enable a broader understanding of leadership and its inference from data in mobile animal groups and beyond.

cs.MA↗

arXiv2017

Data Fusion Reconstruction of Spatially Embedded Complex Networks

We introduce a kernel Lasso (kLasso) optimization that simultaneously accounts for spatial regularity and network sparsity to reconstruct spatial complex networks from data. Through a kernel function, the proposed approach exploits spatial embedding distances to penalize overabundance of spatially long-distance connections. Examples of both synthetic and real-world spatial networks show that the proposed method improves significantly upon existing network reconstruction techniques that mainly concerns sparsity but not spatial regularity. Our results highlight the promise of data fusion in the reconstruction of complex networks, by utilizing both microscopic node-level dynamics (e.g., time series data) and macroscopic network-level information (metadata).

physics.data-an↗

arXiv2017

Quantifying the role of folding in nonautonomous flows: the unsteady Double-Gyre

We analyze chaos in the well-known nonautonomous Double-Gyre system. A key focus is on folding, which is possibly the less-studied aspect of the "stretching + folding = chaos" mantra of chaotic dynamics. Despite the Double-Gyre not having the classical homoclinic structure for the usage of the Smale-Birkhoff theorem to establish chaos, we use the concept of folding to prove the existence of an embedded horseshoe-map. We also show how curvature of manifolds can be used to identify fold points in the Double-Gyre. This method is applicable to general nonautonomous flows in two dimensions, defined for either finite or infinite times.

math.DS↗

arXiv2016

Information theoretical noninvasive damage detection in bridge structures

Damage detection of mechanical structures such as bridges is an important research problem in civil engineering. Using spatially distributed sensor time series data collected from a recent experiment on a local bridge in upper state New York, we study noninvasive damage detection using information-theoretical methods. Several findings are in order. First, the time series data, which represent accelerations measured at the sensors, more closely follow Laplace distribution than normal distribution, allowing us to develop parameter estimators for various information-theoretic measures such as entropy and mutual information. Secondly, as damage is introduced by the removal of bolts of the first diaphragm connection, the interaction between spatially nearby sensors as measured by mutual information become weaker, suggesting that the bridge is "loosened". Finally, using a proposed oMII procedure to prune away indirect interactions, we found that the primary direction of interaction or influence aligns with the traffic direction on the bridge even after damaging the bridge.

stat.AP↗

arXiv2014

Shape Coherence and Finite-Time Curvature Evolution

We introduce a definition of finite-time curvature evolution along with our recent study on shape coherence in nonautonomous dynamical systems. Comparing to slow evolving curvature preserving the shape, large curvature growth points reveal the dramatic change on shape such as the folding behaviors in a system. The level curves of the finite-time curvature evolution field indicate the existence of shape coherent sets.

math.DS↗

arXiv2014

Differential Geometry Perspective of Shape Coherence and Curvature Evolution by Finite-Time Non-hyperbolic Splitting

Mixing, and coherence are fundamental issues at the heart of understanding transport in fluid dynamics and other non-autonomous dynamical systems. Recently, the notion of coherence has come to a more rigorous footing, and particularly within the recent advances of finite-time studies of non-autonomous dynamical systems. Here we define shape coherent sets as a means to emphasize the intuitive notion of ensembles which "hold together" for some period of time, and we contrast this notion to other recent perspectives of coherence, notably "coherent pairs", and likewise also to the geodesic theory of material lines. We will relate shape coherence to the differential geometry concept of curve congruence through matching curvatures. We show that points in phase space where there is a zero-splitting between stable and unstable manifolds locally correspond to points where curvature will evolve only slowly in time. Then we develop curves of points with zero-angle, meaning non-hyperbolic splitting, by continuation methods in terms of the implicit function theorem. From this follows a simple ODE description of the boundaries of shape coherent sets. We will illustrate our methods with popular benchmark examples, and further investigate the intricate structure of foliations geometry.

math.DS↗

arXiv2006

Network Structure Revealed by Short Cycles

This article explores the relationship between communities and short cycles in complex networks, based on the fact that nodes more densely connected amongst one another are more likely to be linked through short cycles. By identifying combinations of 3-, 4- and 5-edge-cycles, a subnetwork is obtained which contains only those nodes and links belonging to such cycles, which can then be used to highlight community structure. Examples are shown using a theoretical model (Sznajd networks) and a real-world network (NCAA football).

cond-mat.dis-nn↗