arXiv subjects

Ioannis G. Kevrekidis

Publications and source records attributed to Ioannis G. Kevrekidis.

At least 73 records · Page 4Linked to original sources

arXiv2022

Quantifying the Structure of Disordered Materials

Durable interest in developing a framework for the detailed structure of glassy materials has produced numerous structural descriptors that trade off between general applicability and interpretability. However, none approach the combination of simplicity and wide-ranging predictive power of the lattice-grain-defect framework for crystalline materials. Working from the hypothesis that the local atomic environments of a glassy material are constrained by enthalpy minimization to a low-dimensional manifold in atomic coordinate space, we develop a novel generalized distance function, the Gaussian Integral Inner Product (GIIP) distance, in connection with agglomerative clustering and diffusion maps, to parameterize that manifold. Applying this approach to a two-dimensional model crystal and a three-dimensional binary model metallic glass results in parameters interpretable as coordination number, composition, volumetric strain, and local symmetry. In particular, we show that a more slowly quenched glass has a higher degree of local tetrahedral symmetry at the expense of cyclic symmetry. While these descriptors require post-hoc interpretation, they minimize bias rooted in crystalline materials science and illuminate a range of structural trends that might otherwise be missed.

cond-mat.dis-nn↗

arXiv2022

Staying the course: Locating equilibria of dynamical systems on Riemannian manifolds defined by point-clouds

We introduce a method to successively locate equilibria (steady states) of dynamical systems on Riemannian manifolds. The manifolds need not be characterized by an a priori known atlas or by the zeros of a smooth map. Instead, they can be defined by point-clouds and sampled as needed through an iterative process. If the manifold is an Euclidean space, our method follows isoclines, curves along which the direction of the vector field $X$ is constant. For a generic vector field $X$, isoclines are smooth curves and every equilibrium lies on isoclines. We generalize the definition of isoclines to Riemannian manifolds through the use of parallel transport: generalized isoclines are curves along which the directions of $X$ are parallel transports of each other. As in the Euclidean case, generalized isoclines of generic vector fields $X$ are smooth curves that connect equilibria of $X$. Our algorithm can be regarded as an extension of the method of Newton trajectories to the manifold setting when the manifold is unknown. This work is motivated by computational statistical mechanics, specifically high dimensional (stochastic) differential equations that model the dynamics of molecular systems. Often, these dynamics concentrate near low-dimensional manifolds and have transitions (saddle points with a single unstable direction) between metastable equilibria. We employ iteratively sampled data and isoclines to locate these saddle points. Coupling a black-box sampling scheme (e.g., Markov chain Monte Carlo) with manifold learning techniques (diffusion maps in the case presented here), we show that our method reliably locates equilibria of $X$.

cs.LG↗

arXiv2022

Two novel families of multiscale staggered patch schemes efficiently simulate large-scale, weakly damped, linear waves

Many multiscale wave systems exhibit macroscale emergent behaviour, for example, the fluid dynamics of floods and tsunamis. Resolving a large range of spatial scales typically requires a prohibitively high computational cost. The small dissipation in wave systems poses a significant challenge to further developing multiscale modelling methods in multiple dimensions. This article develops and evaluates two families of equation-free multiscale methods on novel 2D staggered patch schemes, and demonstrates the power and utility of these multiscale schemes for weakly damped linear waves. A detailed study of sensitivity to numerical roundoff errors establishes the robustness of developed staggered patch schemes. Comprehensive eigenvalue analysis over a wide range of parameters establishes the stability, accuracy, and consistency of the multiscale schemes. Analysis of the computational complexity shows that the measured compute times of the multiscale schemes may be 10^5 times smaller than the compute time for the corresponding full-domain computation. This work provides the essential foundation for efficient large-scale simulation of challenging nonlinear multiscale waves.

math.NA↗

arXiv2022

A note on the control of processes exhibiting input multiplicity

Steady state multiplicity can occur in nonlinear systems, and this presents challenges to feedback control. Input multiplicity arises when the same steady state output values can be reached with system inputs at different values. Dynamic systems with input multiplicities equipped with controllers with integral action have multiple stationary points, which may be locally stable or not. This is undesirable for operation. For a 2x2 example system with three stationary points we demonstrate how to design a set of two single loop controllers such that only one of the stationary points is locally stable, thus effectively eliminating the "input multiplicity problem" for control. We also show that when MPC is used for the example system, all three closed-loop stationary points are stable. Depending on the initial value of the input variables, the closed loop system under MPC may converge to different steady state input instances (but the same output steady state). Therefore we computationally explore the basin boundaries of this closed loop system. It is not clear how MPC or other modern nonlinear controllers could be designed so that only specific equilibrium points are stable.

math.OC↗

arXiv2022

Algorithmic (Semi-)Conjugacy via Koopman Operator Theory

Iterative algorithms are of utmost importance in decision and control. With an ever growing number of algorithms being developed, distributed, and proprietarized, there is a similarly growing need for methods that can provide classification and comparison. By viewing iterative algorithms as discrete-time dynamical systems, we leverage Koopman operator theory to identify (semi-)conjugacies between algorithms using their spectral properties. This provides a general framework with which to classify and compare algorithms.

cs.DS↗

arXiv2022

Data-driven Discovery of Chemotactic Migration of Bacteria via Machine Learning

E. coli chemotactic motion in the presence of a chemoattractant field has been extensively studied using wet laboratory experiments, stochastic computational models as well as partial differential equation-based models (PDEs). The most challenging step in bridging these approaches, is establishing a closed form of the so-called chemotactic term, which describes how bacteria bias their motion up chemonutrient concentration gradients, as a result of a cascade of biochemical processes. Data-driven models can be used to learn the entire evolution operator of the chemotactic PDEs (black box models), or, in a more targeted fashion, to learn just the chemotactic term (gray box models). In this work, data-driven Machine Learning approaches for learning the underlying model PDEs are (a) validated through the use of simulation data from established continuum models and (b) used to infer chemotactic PDEs from experimental data. Even when the data at hand are sparse (coarse in space and/or time), noisy (due to inherent stochasticity in measurements) or partial (e.g. lack of measurements of the associated chemoattractant field), we can attempt to learn the right-hand-side of a closed PDE for an evolving bacterial density. In fact we show that data-driven PDEs including a short history of the bacterial density field (e.g. in the form of higher-order in time PDEs in terms of the measurable bacterial density) can be successful in predicting further bacterial density evolution, and even possibly recovering estimates of the unmeasured chemonutrient field. The main tool in this effort is the effective low-dimensionality of the dynamics (in the spirit of the Whitney and Takens embedding theorems). The resulting data-driven PDE can then be simulated to reproduce/predict computational or experimental bacterial density profile data, and estimate the underlying (unmeasured) chemonutrient field evolution.

q-bio.QM↗

arXiv2022

Limits of Entrainment of Circadian Neuronal Networks

Circadian rhythmicity lies at the center of various important physiological and behavioral processes in mammals, such as sleep, metabolism, homeostasis, mood changes and more. It has been shown that this rhythm arises from self-sustained biomolecular oscillations of a neuronal network located in the Suprachiasmatic Nucleus (SCN). Under normal circumstances, this network remains synchronized to the day-night cycle due to signaling from the retina. Misalignment of these neuronal oscillations with the external light signal can disrupt numerous physiological functions and take a long-lasting toll on health and well-being. In this work, we study a modern computational neuroscience model to determine the limits of circadian synchronization to external light signals of different frequency and duty cycle. We employ a matrix-free approach to locate periodic steady states of the high-dimensional model for various driving conditions. Our algorithmic pipeline enables numerical continuation and construction of bifurcation diagrams w.r.t. forcing parameters. We computationally explore the effect of heterogeneity in the circadian neuronal network, as well as the effect of corrective therapeutic interventions, such as that of the drug molecule Longdaysin. Lastly, we employ unsupervised learning to construct a data-driven embedding space for representing neuronal heterogeneity.

q-bio.NC↗

arXiv2022

Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach

We present an Equation/Variable free machine learning (EVFML) framework for the control of the collective dynamics of complex/multiscale systems modelled via microscopic/agent-based simulators. The approach obviates the need for construction of surrogate, reduced-order models.~The proposed implementation consists of three steps: (A) from high-dimensional agent-based simulations, machine learning (in particular, non-linear manifold learning (Diffusion Maps (DMs)) helps identify a set of coarse-grained variables that parametrize the low-dimensional manifold on which the emergent/collective dynamics evolve. The out-of-sample extension and pre-image problems, i.e. the construction of non-linear mappings from the high-dimensional input space to the low-dimensional manifold and back, are solved by coupling DMs with the Nystrom extension and Geometric Harmonics, respectively; (B) having identified the manifold and its coordinates, we exploit the Equation-free approach to perform numerical bifurcation analysis of the emergent dynamics; then (C) based on the previous steps, we design data-driven embedded wash-out controllers that drive the agent-based simulators to their intrinsic, imprecisely known, emergent open-loop unstable steady-states, thus demonstrating that the scheme is robust against numerical approximation errors and modelling uncertainty.~The efficiency of the framework is illustrated by controlling emergent unstable (i) traveling waves of a deterministic agent-based model of traffic dynamics, and (ii) equilibria of a stochastic financial market agent model with mimesis.

math.DS↗

arXiv2022

Staggered grids for multidimensional multiscale modelling

Numerical schemes for wave-like systems with small dissipation are often inaccurate and unstable due to truncation errors and numerical roundoff errors. Hence, numerical simulations of wave-like systems lacking proper handling of these numerical issues often fail to represent the physical characteristics of wave phenomena. This challenge gets even more intricate for multiscale modelling, especially in multiple dimensions. When using the usual collocated grid, about two-thirds of the resolved wave modes are incorrect with significant dispersion. But, numerical schemes on staggered grids (with alternating variable arrangement) are significantly less dispersive and preserve much of the wave characteristics. Also, the group velocity of the energy propagation in the numerical waves on a staggered grid is in the correct direction, in contrast to the collocated grid. For high accuracy and to preserve much of the wave characteristics, this article extends the concept of staggered grids in full-domain modelling to multidimensional multiscale modelling. Specifically, this article develops 120 multiscale staggered grids and demonstrates their stability, accuracy, and wave-preserving characteristic for equation-free multiscale modelling of weakly damped linear waves. But most characteristics of the developed multiscale staggered grids must also hold in general for multiscale modelling of many complex spatio-temporal physical phenomena such as the general computational fluid dynamics.

math.NA↗

arXiv2022

Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning

We identify effective stochastic differential equations (SDE) for coarse observables of fine-grained particle- or agent-based simulations; these SDE then provide useful coarse surrogate models of the fine scale dynamics. We approximate the drift and diffusivity functions in these effective SDE through neural networks, which can be thought of as effective stochastic ResNets. The loss function is inspired by, and embodies, the structure of established stochastic numerical integrators (here, Euler-Maruyama and Milstein); our approximations can thus benefit from backward error analysis of these underlying numerical schemes. They also lend themselves naturally to "physics-informed" gray-box identification when approximate coarse models, such as mean field equations, are available. Existing numerical integration schemes for Langevin-type equations and for stochastic partial differential equations (SPDE) can also be used for training; we demonstrate this on a stochastically forced oscillator and the stochastic wave equation. Our approach does not require long trajectories, works on scattered snapshot data, and is designed to naturally handle different time steps per snapshot. We consider both the case where the coarse collective observables are known in advance, as well as the case where they must be found in a data-driven manner.

physics.comp-ph↗

arXiv2022

Personalized Algorithm Generation: A Case Study in Learning ODE Integrators

We study the learning of numerical algorithms for scientific computing, which combines mathematically driven, handcrafted design of general algorithm structure with a data-driven adaptation to specific classes of tasks. This represents a departure from the classical approaches in numerical analysis, which typically do not feature such learning-based adaptations. As a case study, we develop a machine learning approach that automatically learns effective solvers for initial value problems in the form of ordinary differential equations (ODEs), based on the Runge-Kutta (RK) integrator architecture. We show that we can learn high-order integrators for targeted families of differential equations without the need for computing integrator coefficients by hand. Moreover, we demonstrate that in certain cases we can obtain superior performance to classical RK methods. This can be attributed to certain properties of the ODE families being identified and exploited by the approach. Overall, this work demonstrates an effective learning-based approach to the design of algorithms for the numerical solution of differential equations. This can be readily extended to other numerical tasks.

math.NA↗

arXiv2022

Black and Gray Box Learning of Amplitude Equations: Application to Phase Field Systems

We present a data-driven approach to learning surrogate models for amplitude equations, and illustrate its application to interfacial dynamics of phase field systems. In particular, we demonstrate learning effective partial differential equations describing the evolution of phase field interfaces from full phase field data. We illustrate this on a model phase field system, where analytical approximate equations for the dynamics of the phase field interface (a higher order eikonal equation and its approximation, the Kardar-Parisi-Zhang (KPZ) equation) are known. For this system, we discuss data-driven approaches for the identification of equations that accurately describe the front interface dynamics. When the analytical approximate models mentioned above become inaccurate, as we move beyond the region of validity of the underlying assumptions, the data-driven equations outperform them. In these regimes, going beyond black-box identification, we explore different approaches to learn data-driven corrections to the analytically approximate models, leading to effective gray box partial differential equations.

stat.ML↗

arXiv2022

On the Parameter Combinations That Matter and on Those That do Not

We present a data-driven approach to characterizing nonidentifiability of a model's parameters and illustrate it through dynamic as well as steady kinetic models. By employing Diffusion Maps and their extensions, we discover the minimal combinations of parameters required to characterize the output behavior of a chemical system: a set of effective parameters for the model. Furthermore, we introduce and use a Conformal Autoencoder Neural Network technique, as well as a kernel-based Jointly Smooth Function technique, to disentangle the redundant parameter combinations that do not affect the output behavior from the ones that do. We discuss the interpretability of our data-driven effective parameters, and demonstrate the utility of the approach both for behavior prediction and parameter estimation. In the latter task, it becomes important to describe level sets in parameter space that are consistent with a particular output behavior. We validate our approach on a model of multisite phosphorylation, where a reduced set of effective parameters (nonlinear combinations of the physical ones) has previously been established analytically.

cs.LG↗

arXiv2022

Learning black- and gray-box chemotactic PDEs/closures from agent based Monte Carlo simulation data

We propose a machine learning framework for the data-driven discovery of macroscopic chemotactic Partial Differential Equations (PDEs) -- and the closures that lead to them -- from high-fidelity, individual-based stochastic simulations of E.coli bacterial motility. The fine scale, detailed, hybrid (continuum - Monte Carlo) simulation model embodies the underlying biophysics, and its parameters are informed from experimental observations of individual cells. We exploit Automatic Relevance Determination (ARD) within a Gaussian Process framework for the identification of a parsimonious set of collective observables that parametrize the law of the effective PDEs. Using these observables, in a second step we learn effective, coarse-grained "Keller-Segel class" chemotactic PDEs using machine learning regressors: (a) (shallow) feedforward neural networks and (b) Gaussian Processes. The learned laws can be black-box (when no prior knowledge about the PDE law structure is assumed) or gray-box when parts of the equation (e.g. the pure diffusion part) is known and "hardwired" in the regression process. We also discuss data-driven corrections (both additive and functional) of analytically known, approximate closures.

q-bio.QM↗

arXiv2022

Learning the temporal evolution of multivariate densities via normalizing flows

In this work, we propose a method to learn multivariate probability distributions using sample path data from stochastic differential equations. Specifically, we consider temporally evolving probability distributions (e.g., those produced by integrating local or nonlocal Fokker-Planck equations). We analyze this evolution through machine learning assisted construction of a time-dependent mapping that takes a reference distribution (say, a Gaussian) to each and every instance of our evolving distribution. If the reference distribution is the initial condition of a Fokker-Planck equation, what we learn is the time-T map of the corresponding solution. Specifically, the learned map is a multivariate normalizing flow that deforms the support of the reference density to the support of each and every density snapshot in time. We demonstrate that this approach can approximate probability density function evolutions in time from observed sampled data for systems driven by both Brownian and Lévy noise. We present examples with two- and three-dimensional, uni- and multimodal distributions to validate the method.

stat.ML↗

arXiv2022

Double Diffusion Maps and their Latent Harmonics for Scientific Computations in Latent Space

We introduce a data-driven approach to building reduced dynamical models through manifold learning; the reduced latent space is discovered using Diffusion Maps (a manifold learning technique) on time series data. A second round of Diffusion Maps on those latent coordinates allows the approximation of the reduced dynamical models. This second round enables mapping the latent space coordinates back to the full ambient space (what is called lifting); it also enables the approximation of full state functions of interest in terms of the reduced coordinates. In our work, we develop and test three different reduced numerical simulation methodologies, either through pre-tabulation in the latent space and integration on the fly or by going back and forth between the ambient space and the latent space. The data-driven latent space simulation results, based on the three different approaches, are validated through (a) the latent space observation of the full simulation through the Nyström Extension formula, or through (b) lifting the reduced trajectory back to the full ambient space, via Latent Harmonics. Latent space modeling often involves additional regularization to favor certain properties of the space over others, and the mapping back to the ambient space is then constructed mostly independently from these properties; here, we use the same data-driven approach to construct the latent space and then map back to the ambient space.

stat.ML↗

arXiv2022

Questionnaires to PDEs: From Disorganized Data to Emergent Generative Dynamic Models

Starting with sets of disorganized observations of spatially varying and temporally evolving systems, obtained at different (also disorganized) sets of parameters, we demonstrate the data-driven derivation of parameter dependent, evolutionary partial differential equation (PDE) models capable of generating the data. This tensor type of data is reminiscent of shuffled (multi-dimensional) puzzle tiles. The independent variables for the evolution equations (their "space" and "time") as well as their effective parameters are all "emergent", i.e., determined in a data-driven way from our disorganized observations of behavior in them. We use a diffusion map based "questionnaire" approach to build a parametrization of our emergent space/time/parameter space for the data. This approach iteratively processes the data by successively observing them on the "space", the "time", and the "parameter" axes of a tensor. Once the data are organized, we use machine learning (here, neural networks) to approximate the operators governing the evolution equations in this emergent space. Our illustrative example is based on a previously developed vertex-plus-signaling model of Drosophila embryonic development. This allows us to discuss features of the process like symmetry breaking, translational invariance, and autonomousness of the emergent PDE model, as well as its interpretability.

math.DS↗

arXiv2022

An Operator Theoretic View on Pruning Deep Neural Networks

The discovery of sparse subnetworks that are able to perform as well as full models has found broad applied and theoretical interest. While many pruning methods have been developed to this end, the naïve approach of removing parameters based on their magnitude has been found to be as robust as more complex, state-of-the-art algorithms. The lack of theory behind magnitude pruning's success, especially pre-convergence, and its relation to other pruning methods, such as gradient based pruning, are outstanding open questions in the field that are in need of being addressed. We make use of recent advances in dynamical systems theory, namely Koopman operator theory, to define a new class of theoretically motivated pruning algorithms. We show that these algorithms can be equivalent to magnitude and gradient based pruning, unifying these seemingly disparate methods, and find that they can be used to shed light on magnitude pruning's performance during the early part of training.

cs.LG↗