arXiv subjects

Steven L. Brunton

Publications and source records attributed to Steven L. Brunton.

At least 91 records · Page 5Linked to original sources

arXiv2022

Multiscale model reduction for incompressible flows

Many unsteady flows exhibiting complex dynamics are nevertheless characterized by emergent large-scale coherence in space and time. Reduced-order models based on Galerkin projection of the governing equations onto an orthogonal modal basis approximate the flow as a low-dimensional dynamical system with linear and quadratic terms. However, these Galerkin models often fail to reproduce the true dynamics, in part because they ignore important nonlinear interactions with unresolved flow scales. Here, we use a separation of time scales between the resolved and subscale variables to derive a reduced-order model with cubic closure terms for the truncated modes, generalizing the classic Stuart-Landau equation. The leading order cubic terms are determined by averaging out fast variables through a perturbation series approximation of the action of a stochastic Koopman operator. We show analytically that this multiscale closure model can capture both the effects of mean-flow deformation and the energy cascade before demonstrating improved stability and accuracy in models of chaotic lid-driven cavity flow and vortex pairing in a mixing layer. This approach to closure modeling establishes a general theory for the origin and role of cubic nonlinearities in low-dimensional models of incompressible flows

physics.flu-dyn↗

arXiv2022

The Experimental Multi-Arm Pendulum on a Cart: A Benchmark System for Chaos, Learning, and Control

The single, double, and triple pendulum has served as an illustrative experimental benchmark system for scientists to study dynamical behavior for more than four centuries. The pendulum system exhibits a wide range of interesting behaviors, from simple harmonic motion in the single pendulum to chaotic dynamics in multi-arm pendulums. Under forcing, even the single pendulum may exhibit chaos, providing a simple example of a damped-driven system. All multi-armed pendulums are characterized by the existence of index-one saddle points, which mediate the transport of trajectories in the system, providing a simple mechanical analog of various complex transport phenomena, from biolocomotion to transport within the solar system. Further, pendulum systems have long been used to design and test both linear and nonlinear control strategies, with the addition of more arms making the problem more challenging. In this work, we provide extensive designs for the construction and operation of a high-performance, multi-link pendulum on a cart system. Although many experimental setups have been built to study the behavior of pendulum systems, such an extensive documentation on the design, construction, and operation is missing from the literature. The resulting experimental system is highly flexible, enabling a wide range of benchmark problems in dynamical systems modeling, system identification and learning, and control. To promote reproducible research, we have made our entire system open-source, including 3D CAD drawings, basic tutorial code, and data. Moreover, we discuss the possibility of extending our system capability to be operated remotely to enable researchers all around the world to use it, thus increasing access.

nlin.CD↗

arXiv2022

Swarm Modelling with Dynamic Mode Decomposition

Modelling biological or engineering swarms is challenging due to the inherently high dimension of the system, despite the often low-dimensional emergent dynamics. Most existing swarm modelling approaches are based on first principles and often result in swarm-specific parameterizations that do not generalize to a broad range of applications. In this work, we apply a purely data-driven method to (1) learn local interactions of homogeneous swarms through observation data and to (2) generate similar swarming behaviour using the learned model. In particular, a modified version of dynamic mode decomposition with control, called swarmDMD, is developed and tested on the canonical Vicsek swarm model. The goal is to use swarmDMD to learn inter-agent interactions that give rise to the observed swarm behaviour. We show that swarmDMD can faithfully reconstruct the swarm dynamics, and the model learned by swarmDMD provides a short prediction window for data extrapolation with a trade-off between prediction accuracy and prediction horizon. We also provide a comprehensive analysis on the efficacy of different observation data types on the modelling, where we find that inter-agent distance yields the most accurate models. We believe the proposed swarmDMD approach will be useful for studying multi-agent systems found in biology, physics, and engineering.

cs.NE↗

arXiv2022

Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning

The complexity of real-world geophysical systems is often compounded by the fact that the observed measurements depend on hidden variables. These latent variables include unresolved small scales and/or rapidly evolving processes, partially observed couplings, or forcings in coupled systems. This is the case in ocean-atmosphere dynamics, for which unknown interior dynamics can affect surface observations. The identification of computationally-relevant representations of such partially-observed and highly nonlinear systems is thus challenging and often limited to short-term forecast applications. Here, we investigate the physics-constrained learning of implicit dynamical embeddings, leveraging neural ordinary differential equation (NODE) representations. A key objective is to constrain their boundedness, which promotes the generalization of the learned dynamics to arbitrary initial condition. The proposed architecture is implemented within a deep learning framework, and its relevance is demonstrated with respect to state-of-the-art schemes for different case-studies representative of geophysical dynamics.

stat.ML↗

arXiv2022

Dimensionally Consistent Learning with Buckingham Pi

In the absence of governing equations, dimensional analysis is a robust technique for extracting insights and finding symmetries in physical systems. Given measurement variables and parameters, the Buckingham Pi theorem provides a procedure for finding a set of dimensionless groups that spans the solution space, although this set is not unique. We propose an automated approach using the symmetric and self-similar structure of available measurement data to discover the dimensionless groups that best collapse this data to a lower dimensional space according to an optimal fit. We develop three data-driven techniques that use the Buckingham Pi theorem as a constraint: (i) a constrained optimization problem with a non-parametric input-output fitting function, (ii) a deep learning algorithm (BuckiNet) that projects the input parameter space to a lower dimension in the first layer, and (iii) a technique based on sparse identification of nonlinear dynamics (SINDy) to discover dimensionless equations whose coefficients parameterize the dynamics. We explore the accuracy, robustness and computational complexity of these methods as applied to three example problems: a bead on a rotating hoop, a laminar boundary layer, and Rayleigh-Bénard convection.

cs.LG↗

arXiv2022

PySINDy: A comprehensive Python package for robust sparse system identification

Automated data-driven modeling, the process of directly discovering the governing equations of a system from data, is increasingly being used across the scientific community. PySINDy is a Python package that provides tools for applying the sparse identification of nonlinear dynamics (SINDy) approach to data-driven model discovery. In this major update to PySINDy, we implement several advanced features that enable the discovery of more general differential equations from noisy and limited data. The library of candidate terms is extended for the identification of actuated systems, partial differential equations (PDEs), and implicit differential equations. Robust formulations, including the integral form of SINDy and ensembling techniques, are also implemented to improve performance for real-world data. Finally, we provide a range of new optimization algorithms, including several sparse regression techniques and algorithms to enforce and promote inequality constraints and stability. Together, these updates enable entirely new SINDy model discovery capabilities that have not been reported in the literature, such as constrained PDE identification and ensembling with different sparse regression optimizers.

eess.SY↗

arXiv2022

Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoders

A central challenge in data-driven model discovery is the presence of hidden, or latent, variables that are not directly measured but are dynamically important. Takens' theorem provides conditions for when it is possible to augment these partial measurements with time delayed information, resulting in an attractor that is diffeomorphic to that of the original full-state system. However, the coordinate transformation back to the original attractor is typically unknown, and learning the dynamics in the embedding space has remained an open challenge for decades. Here, we design a custom deep autoencoder network to learn a coordinate transformation from the delay embedded space into a new space where it is possible to represent the dynamics in a sparse, closed form. We demonstrate this approach on the Lorenz, Rössler, and Lotka-Volterra systems, learning dynamics from a single measurement variable. As a challenging example, we learn a Lorenz analogue from a single scalar variable extracted from a video of a chaotic waterwheel experiment. The resulting modeling framework combines deep learning to uncover effective coordinates and the sparse identification of nonlinear dynamics (SINDy) for interpretable modeling. Thus, we show that it is possible to simultaneously learn a closed-form model and the associated coordinate system for partially observed dynamics.

cs.LG↗

arXiv2022

Dynamic Mode Decomposition for Aero-Optic Wavefront Characterization

Aero-optical beam control relies on the development of low-latency forecasting techniques to quickly predict wavefronts aberrated by the Turbulent Boundary Layer (TBL) around an airborne optical system, and its study applies to a multi-domain need from astronomy to microscopy for high-fidelity laser propagation. We leverage the forecasting capabilities of the Dynamic Mode Decomposition (DMD) -- an equation-free, data-driven method for identifying coherent flow structures and their associated spatiotemporal dynamics -- in order to estimate future state wavefront phase aberrations to feed into an adaptive optic (AO) control loop. We specifically leverage the optimized DMD (opt-DMD) algorithm on a subset of the Airborne Aero-Optics Laboratory Transonic (AAOL-T) experimental dataset, characterizing aberrated wavefront dynamics for 23 beam propagation directions via the spatiotemporal decomposition underlying DMD. Critically, we show that opt-DMD produces an optimally de-biased eigenvalue spectrum with imaginary eigenvalues, allowing for arbitrarily long forecasting to produce a robust future-state prediction, while exact DMD loses structural information due to modal decay rates.

physics.flu-dyn↗

arXiv2022

Principal component trajectories for modeling spectrally-continuous dynamics as forced linear systems

Delay embeddings of time series data have emerged as a promising coordinate basis for data-driven estimation of the Koopman operator, which seeks a linear representation for observed nonlinear dynamics. Recent work has demonstrated the efficacy of Dynamic Mode Decomposition (DMD) for obtaining finite-dimensional Koopman approximations in delay coordinates. In this paper we demonstrate how nonlinear dynamics with sparse Fourier spectra can be (i) represented by a superposition of principal component trajectories (PCT) and (ii) modeled by DMD in this coordinate space. For continuous or mixed (discrete and continuous) spectra, DMD can be augmented with an external forcing term. We present a method for learning linear control models in delay coordinates while simultaneously discovering the corresponding exogeneous forcing signal in a fully unsupervised manner. This extends the existing DMD with control (DMDc) algorithm to cases where a control signal is not known a priori. We provide examples to validate the learned forcing against a known ground truth and illustrate their statistical similarity. Finally we offer a demonstration of this method applied to real-world power grid load data to show its utility for diagnostics and interpretation on systems in which somewhat periodic behavior is strongly forced by unknown and unmeasurable environmental variables.

physics.comp-ph↗

arXiv2021

Deeptime: a Python library for machine learning dynamical models from time series data

Generation and analysis of time-series data is relevant to many quantitative fields ranging from economics to fluid mechanics. In the physical sciences, structures such as metastable and coherent sets, slow relaxation processes, collective variables dominant transition pathways or manifolds and channels of probability flow can be of great importance for understanding and characterizing the kinetic, thermodynamic and mechanistic properties of the system. Deeptime is a general purpose Python library offering various tools to estimate dynamical models based on time-series data including conventional linear learning methods, such as Markov state models (MSMs), Hidden Markov Models and Koopman models, as well as kernel and deep learning approaches such as VAMPnets and deep MSMs. The library is largely compatible with scikit-learn, having a range of Estimator classes for these different models, but in contrast to scikit-learn also provides deep Model classes, e.g. in the case of an MSM, which provide a multitude of analysis methods to compute interesting thermodynamic, kinetic and dynamical quantities, such as free energies, relaxation times and transition paths. The library is designed for ease of use but also easily maintainable and extensible code. In this paper we introduce the main features and structure of the deeptime software.

math.DS↗

arXiv2021

Physics-informed dynamic mode decomposition (piDMD)

In this work, we demonstrate how physical principles -- such as symmetries, invariances, and conservation laws -- can be integrated into the dynamic mode decomposition (DMD). DMD is a widely-used data analysis technique that extracts low-rank modal structures and dynamics from high-dimensional measurements. However, DMD frequently produces models that are sensitive to noise, fail to generalize outside the training data, and violate basic physical laws. Our physics-informed DMD (piDMD) optimization, which may be formulated as a Procrustes problem, restricts the family of admissible models to a matrix manifold that respects the physical structure of the system. We focus on five fundamental physical principles -- conservation, self-adjointness, localization, causality, and shift-invariance -- and derive several closed-form solutions and efficient algorithms for the corresponding piDMD optimizations. With fewer degrees of freedom, piDMD models are less prone to overfitting, require less training data, and are often less computationally expensive to build than standard DMD models. We demonstrate piDMD on a range of challenging problems in the physical sciences, including energy-preserving fluid flow, travelling-wave systems, the Schrödinger equation, solute advection-diffusion, a system with causal dynamics, and three-dimensional transitional channel flow. In each case, piDMD significantly outperforms standard DMD in metrics such as spectral identification, state prediction, and estimation of optimal forcings and responses.

math.DS↗

arXiv2021

Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control

Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from subsets of limited and noisy data. The aggregate model statistics are then used to produce inclusion probabilities of the candidate functions, which enables uncertainty quantification and probabilistic forecasts. We apply this ensemble-SINDy (E-SINDy) algorithm to several synthetic and real-world data sets and demonstrate substantial improvements to the accuracy and robustness of model discovery from extremely noisy and limited data. For example, E-SINDy uncovers partial differential equations models from data with more than twice as much measurement noise as has been previously reported. Similarly, E-SINDy learns the Lotka Volterra dynamics from remarkably limited data of yearly lynx and hare pelts collected from 1900-1920. E-SINDy is computationally efficient, with similar scaling as standard SINDy. Finally, we show that ensemble statistics from E-SINDy can be exploited for active learning and improved model predictive control.

math.NA↗

arXiv2021

Modern Koopman Theory for Dynamical Systems

The field of dynamical systems is being transformed by the mathematical tools and algorithms emerging from modern computing and data science. First-principles derivations and asymptotic reductions are giving way to data-driven approaches that formulate models in operator theoretic or probabilistic frameworks. Koopman spectral theory has emerged as a dominant perspective over the past decade, in which nonlinear dynamics are represented in terms of an infinite-dimensional linear operator acting on the space of all possible measurement functions of the system. This linear representation of nonlinear dynamics has tremendous potential to enable the prediction, estimation, and control of nonlinear systems with standard textbook methods developed for linear systems. However, obtaining finite-dimensional coordinate systems and embeddings in which the dynamics appear approximately linear remains a central open challenge. The success of Koopman analysis is due primarily to three key factors: 1) there exists rigorous theory connecting it to classical geometric approaches for dynamical systems, 2) the approach is formulated in terms of measurements, making it ideal for leveraging big-data and machine learning techniques, and 3) simple, yet powerful numerical algorithms, such as the dynamic mode decomposition (DMD), have been developed and extended to reduce Koopman theory to practice in real-world applications. In this review, we provide an overview of modern Koopman operator theory, describing recent theoretical and algorithmic developments and highlighting these methods with a diverse range of applications. We also discuss key advances and challenges in the rapidly growing field of machine learning that are likely to drive future developments and significantly transform the theoretical landscape of dynamical systems.

math.DS↗

arXiv2021

Data-driven stochastic modeling of coarse-grained dynamics with finite-size effects using Langevin regression

Obtaining coarse-grained models that accurately incorporate finite-size effects is an important open challenge in the study of complex, multi-scale systems. We apply Langevin regression, a recently developed method for finding stochastic differential equation (SDE) descriptions of realistically-sampled time series data, to understand finite-size effects in the Kuramoto model of coupled oscillators. We find that across the entire bifurcation diagram, the dynamics of the Kuramoto order parameter are statistically consistent with an SDE whose drift term has the form predicted by the Ott-Antonsen ansatz in the $N\to \infty$ limit. We find that the diffusion term is nearly independent of the bifurcation parameter, and has a magnitude decaying as $N^{-1/2}$, consistent with the central limit theorem. This shows that the diverging fluctuations of the order parameter near the critical point are driven by a bifurcation in the underlying drift term, rather than increased stochastic forcing.

nlin.AO↗

arXiv2021

Applying Machine Learning to Study Fluid Mechanics

This paper provides a short overview of how to use machine learning to build data-driven models in fluid mechanics. The process of machine learning is broken down into five stages: (1) formulating a problem to model, (2) collecting and curating training data to inform the model, (3) choosing an architecture with which to represent the model, (4) designing a loss function to assess the performance of the model, and (5) selecting and implementing an optimization algorithm to train the model. At each stage, we discuss how prior physical knowledge may be embedding into the process, with specific examples from the field of fluid mechanics.

physics.flu-dyn↗

arXiv2021

Improved approximations to the Wagner function using sparse identification of nonlinear dynamics

The Wagner function in classical unsteady aerodynamic theory represents the response in lift on an airfoil that is subject to a sudden change in conditions. While it plays a fundamental role in the development and application of unsteady aerodynamic methods, explicit expressions for this function are difficult to obtain. The Wagner function requires computation of an inverse Laplace transform, or similar inversion, of a non-rational function in the Laplace domain, which is closely related to the Theodorsen function. This has led to numerous proposed approximations to the Wagner function, which facilitate convenient and rapid computations. While these approximations can be sufficient for many purposes, their behavior is often noticeably different from the true Wagner function, especially for long-time asymptotic behavior. In particular, while many approximations have small maximum absolute error across all times, the relative error of the asymptotic behavior can be substantial. As well as documenting this error, we propose an alternative approximation methodology that is accurate for all times, for a variety of accuracy measures. This methodology casts the Wagner function as the solution of a nonlinear scalar ordinary differential equation, which is identified using a variant of the sparse identification of nonlinear dynamics (SINDy) algorithm. We show that this approach can give accurate approximations using either first- or second-order differential equations. We additionally show that this method can be applied to model the analogous lift response for a more realistic aerodynamic system, featuring a finite thickness airfoil and a nonplanar wake.

physics.flu-dyn↗

arXiv2021

Challenges in Dynamic Mode Decomposition

Dynamic Mode Decomposition (DMD) is a powerful tool for extracting spatial and temporal patterns from multi-dimensional time series, and it has been used successfully in a wide range of fields, including fluid mechanics, robotics, and neuroscience. Two of the main challenges remaining in DMD research are noise sensitivity and issues related to Krylov space closure when modeling nonlinear systems. Here, we investigate the combination of noise and nonlinearity in a controlled setting, by studying a class of systems with linear latent dynamics which are observed via multinomial observables. Our numerical models include system and measurement noise. We explore the influences of dataset metrics, the spectrum of the latent dynamics, the normality of the system matrix, and the geometry of the dynamics. Our results show that even for these very mildly nonlinear conditions, DMD methods often fail to recover the spectrum and can have poor predictive ability. Our work is motivated by our experience modeling multilegged robot data, where we have encountered great difficulty in reconstructing time series for oscillatory systems with slow transients, which decay only slightly faster than a period.

math.DS↗

arXiv2021

SINDy with Control: A Tutorial

Many dynamical systems of interest are nonlinear, with examples in turbulence, epidemiology, neuroscience, and finance, making them difficult to control using linear approaches. Model predictive control (MPC) is a powerful model-based optimization technique that enables the control of such nonlinear systems with constraints. However, modern systems often lack computationally tractable models, motivating the use of system identification techniques to learn accurate and efficient models for real-time control. In this tutorial article, we review emerging data-driven methods for model discovery and how they are used for nonlinear MPC. In particular, we focus on the sparse identification of nonlinear dynamics (SINDy) algorithm and show how it may be used with MPC on an infectious disease control example. We compare the performance against MPC based on a linear dynamic mode decomposition (DMD) model. Code is provided to run the tutorial examples and may be modified to extend this data-driven control framework to arbitrary nonlinear systems.

math.OC↗