arXiv subjects

Brian R. Hunt

Publications and source records attributed to Brian R. Hunt.

At least 19 recordsLinked to original sources

arXiv2025

Model Training, Data Assimilation, and Forecast Experiments with a Hybrid Atmospheric Model that Incorporates Machine Learning

The hybrid model combines the physics-based primitive-equations model SPEEDY with a machine learning-based (ML-based) model component, while ERA5 reanalyses provide the presumed true states of the atmosphere. Six-hourly simulated noisy observations are generated for a 30-year ML training period and a one-year testing period. These observations are assimilated with a Local Ensemble Transform Kalman Filter (LETKF), and a 10-day deterministic forecast is also started from each ensemble mean analysis of the testing period. In the first experiment, the physics-based model provides the background ensemble members and the 10-day deterministic forecasts. In the other three experiments, the hybrid model plays the same role as the physics-based model in the first experiment, but it is trained on a different data set in each experiment. These training data sets are analyses obtained by using the physics-based model (second experiment), the hybrid model of the previous experiment (third experiment), and for comparison, ERA5 reanalyses (fourth experiment). The results of the experiments show that hybridizing the model can substantially improve the accuracy of the analyses and forecasts. When the model is trained on ERA5 reanalyses, the biases of the analyses are negligible and the magnitude of the flow-dependent part of the analysis errors is greatly reduced. While the gains in analysis accuracy are distinctly more modest in the other two hybrid model experiments, the gains in forecast accuracy tend to be larger in those experiments after 1-3 forecast days. However, these extra gains of forecast accuracy are achieved, in part, by a modest gradual reduction of the spatial variability of the forecasts.

nlin.CD↗

arXiv2022

Stabilizing Machine Learning Prediction of Dynamics: Noise and Noise-inspired Regularization

Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this technique can result in artificially rapid error growth. In this article, we systematically examine the technique of adding noise to the ML model input during training to promote stability and improve prediction accuracy. Furthermore, we introduce Linearized Multi-Noise Training (LMNT), a regularization technique that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. Our case study uses reservoir computing, a machine-learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while reservoir computers trained without regularization are unstable. Compared with other regularization techniques that yield stability in some cases, we find that both short-term and climate predictions from reservoir computers trained with noise or with LMNT are substantially more accurate. Finally, we show that the deterministic aspect of our LMNT regularization facilitates fast hyperparameter tuning when compared to training with noise.

cs.LG↗

arXiv2021

Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components

We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is in the form of noisy partial measurements of the past and present state of the dynamical system. Recently there have been several promising data-driven approaches to forecasting of chaotic dynamical systems using machine learning. Particularly promising among these are hybrid approaches that combine machine learning with a knowledge-based model, where a machine-learning technique is used to correct the imperfections in the knowledge-based model. Such imperfections may be due to incomplete understanding and/or limited resolution of the physical processes in the underlying dynamical system, e.g., the atmosphere or the ocean. Previously proposed data-driven forecasting approaches tend to require, for training, measurements of all the variables that are intended to be forecast. We describe a way to relax this assumption by combining data assimilation with machine learning. We demonstrate this technique using the Ensemble Transform Kalman Filter (ETKF) to assimilate synthetic data for the 3-variable Lorenz system and for the Kuramoto-Sivashinsky system, simulating model error in each case by a misspecified parameter value. We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.

cs.LG↗

arXiv2020

Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics

We examine the efficiency of Recurrent Neural Networks in forecasting the spatiotemporal dynamics of high dimensional and reduced order complex systems using Reservoir Computing (RC) and Backpropagation through time (BPTT) for gated network architectures. We highlight advantages and limitations of each method and discuss their implementation for parallel computing architectures. We quantify the relative prediction accuracy of these algorithms for the longterm forecasting of chaotic systems using as benchmarks the Lorenz-96 and the Kuramoto-Sivashinsky (KS) equations. We find that, when the full state dynamics are available for training, RC outperforms BPTT approaches in terms of predictive performance and in capturing of the long-term statistics, while at the same time requiring much less training time. However, in the case of reduced order data, large scale RC models can be unstable and more likely than the BPTT algorithms to diverge. In contrast, RNNs trained via BPTT show superior forecasting abilities and capture well the dynamics of reduced order systems. Furthermore, the present study quantifies for the first time the Lyapunov Spectrum of the KS equation with BPTT, achieving similar accuracy as RC. This study establishes that RNNs are a potent computational framework for the learning and forecasting of complex spatiotemporal systems.

eess.SP↗

arXiv2018

Attractor Reconstruction by Machine Learning

A machine-learning approach called "reservoir computing" has been used successfully for short-term prediction and attractor reconstruction of chaotic dynamical systems from time series data. We present a theoretical framework that describes conditions under which reservoir computing can create an empirical model capable of skillful short-term forecasts and accurate long-term ergodic behavior. We illustrate this theory through numerical experiments. We also argue that the theory applies to certain other machine learning methods for time series prediction.

nlin.CD↗

arXiv2017

Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data

We use recent advances in the machine learning area known as 'reservoir computing' to formulate a method for model-free estimation from data of the Lyapunov exponents of a chaotic process. The technique uses a limited time series of measurements as input to a high-dimensional dynamical system called a 'reservoir'. After the reservoir's response to the data is recorded, linear regression is used to learn a large set of parameters, called the 'output weights'. The learned output weights are then used to form a modified autonomous reservoir designed to be capable of producing arbitrarily long time series whose ergodic properties approximate those of the input signal. When successful, we say that the autonomous reservoir reproduces the attractor's 'climate'. Since the reservoir equations and output weights are known, we can compute derivatives needed to determine the Lyapunov exponents of the autonomous reservoir, which we then use as estimates of the Lyapunov exponents for the original input generating system. We illustrate the effectiveness of our technique with two examples, the Lorenz system, and the Kuramoto-Sivashinsky (KS) equation. In particular, we use the Lorenz system to show that achieving climate reproduction may require tuning of the reservoir parameters. For the case of the KS equation, we note that as the system's spatial size is increased, the number of Lyapunov exponents increases, thus yielding a challenging test of our method, which we find the method successfully passes.

nlin.CD↗

arXiv2015

Defining Chaos

In this paper we propose, discuss and illustrate a computationally feasible definition of chaos which can be applied very generally to situations that are commonly encountered, including attractors, repellers and non-periodically forced systems. This definition is based on an entropy-like quantity, which we call "expansion entropy", and we define chaos as occurring when this quantity is positive. We relate and compare expansion entropy to the well-known concept of topological entropy, to which it is equivalent under appropriate conditions. We also present example illustrations, discuss computational implementations, and point out issues arising from attempts at giving definitions of chaos that are not entropy-based.

nlin.CD↗

arXiv2013

Coupled skinny baker's maps and the Kaplan-Yorke conjecture

The Kaplan-Yorke conjecture states that for "typical" dynamical systems with a physical measure, the information dimension and the Lyapunov dimension coincide. We explore this conjecture in a neighborhood of a system for which the two dimensions do not coincide because the system consists of two uncoupled subsystems. We are interested in whether coupling "typically" restores the equality of the dimensions. The particular subsystems we consider are skinny baker's maps, and we consider uni-directional coupling. For coupling in one of the possible directions, we prove that the dimensions coincide for a prevalent set of coupling functions, but for coupling in the other direction we show that the dimensions remain unequal for all coupling functions. We conjecture that the dimensions prevalently coincide for bi-directional coupling. On the other hand, we conjecture that the phenomenon we observe for a particular class of systems with uni-directional coupling, where the information and Lyapunov dimensions differ robustly, occurs more generally for many classes of uni-directionally coupled systems (also called skew-product systems) in higher dimensions.

math.DS↗

arXiv2011

Ensemble regional data assimilation using joint states

We propose a data assimilation scheme that produces the analyses for a global and an embedded limited area model simultaneously, considering forecast information from both models. The purpose of the proposed approach is twofold. First, we expect that the global analysis will benefit from incorporation of information from the higher resolution limited area model. Second, our method is expected to produce a limited area analysis that is more strongly constrained by the large scale flow than a conventional limited area analysis. The proposed scheme minimizes a cost function in which the control variable is the joint state of the global and the limited area models. In addition, the cost function includes a constraint term that penalizes large differences between the global and the limited area state estimates. The proposed approach is tested by idealized experiments, using `toy' models introduced by Lorenz in 2005. The results of these experiments suggest that the proposed approach improves the global analysis within and near the limited area domain and the regional analysis near the lateral boundaries. These analysis improvements lead to forecast improvements in both the global and the limited area models.

nlin.CD↗

arXiv2010

Comment on "Long Time Evolution of Phase Oscillator Systems" [Chaos 19,023117 (2009), arXiv:0902.2773]

A previous paper (arXiv:0902.2773, henceforth referred to as I) considered a general class of problems involving the evolution of large systems of globally coupled phase oscillators. It was shown there that, in an appropriate sense, the solutions to these problems are time asymptotically attracted toward a reduced manifold of system states (denoted M). This result has considerable utility in the analysis of these systems, as has been amply demonstrated in recent papers. In this note, we show that the analysis of I can be modified in a simple way that establishes significant extensions of the range of validity of our previous result. In particular, we generalize I in the following ways: (1) attraction to M is now shown for a very general class of oscillator frequency distribution functions g(ω), and (2) a previous restriction on the allowed class of initial conditions is now substantially relaxed.

nlin.CD↗

arXiv2009

Approximating invariant densities of metastable systems

We consider a piecewise smooth expanding map of the interval possessing two invariant subsets of positive Lebesgue measure and exactly two ergodic absolutely continuous invariant probability measures (ACIMs). When this system is perturbed slightly to make the invariant sets merge, we describe how the unique ACIM of the perturbed map can be approximated by a convex combination of the two initial ergodic ACIMs.

math.DS↗

arXiv2007

Approximating the largest eigenvalue of network adjacency matrices

The largest eigenvalue of the adjacency matrix of a network plays an important role in several network processes (e.g., synchronization of oscillators, percolation on directed networks, linear stability of equilibria of network coupled systems, etc.). In this paper we develop approximations to the largest eigenvalue of adjacency matrices and discuss the relationships between these approximations. Numerical experiments on simulated networks are used to test our results.

cond-mat.dis-nn↗

arXiv2007

Weighted percolation on directed networks

We present an analysis of the percolation transition for general node removal strategies valid for locally tree-like directed networks. On the basis of heuristic arguments we predict that, if the probability of removing node $i$ is $p_i$, the network disintegrates if $p_i$ is such that the largest eigenvalue of the matrix with entries $A_{ij}(1-p_i)$ is less than 1, where $A$ is the adjacency matrix of the network. The knowledge or applicability of a Markov network model is not required by our theory, thus making it applicable to situations not covered by previous works. We test our predicted percolation criterion against numerical results for different networks and node removal strategies.

cond-mat.soft↗

arXiv2007

Comparison between Local Ensemble Transform Kalman Filter and PSAS in the NASA finite volume GCM: perfect model experiments

This paper explores the potential of Local Ensemble Transform Kalman Filter (LETKF) by comparing the performance of LETKF with an operational 3D-Var assimilation system, Physical-Space Statistical Analysis System (PSAS), under a perfect model scenario. The comparison is carried out on the finite volume Global Circulation Model (fvGCM) with 72 grid points zonally, 46 grid points meridionally and 55 vertical levels. With only forty ensemble members, LETKF obtains an analysis and forecasts with lower RMS errors than those from PSAS. The performance of LETKF is further improved, especially over the oceans, by assimilating simulated temperature observations from rawinsondes and conventional surface pressure observations instead of geopotential heights. An initial decrease of the forecast errors in the NH observed in PSAS but not in LETKF suggests that the PSAS analysis is less balanced. The observed advantage of LETKF over PSAS is due to the ability of the forty-member ensemble from LETKF to capture flow-dependent errors and thus create a good estimate of the true background uncertainty. Furthermore, localization makes LETKF highly parallel and efficient, requiring only 5 minutes per analysis in a cluster of 20 PCs with forty ensemble members.

physics.ao-ph↗

arXiv2006

Efficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter

Data assimilation is an iterative approach to the problem of estimating the state of a dynamical system using both current and past observations of the system together with a model for the system's time evolution. Rather than solving the problem from scratch each time new observations become available, one uses the model to ``forecast'' the current state, using a prior state estimate (which incorporates information from past data) as the initial condition, then uses current data to correct the prior forecast to a current state estimate. This Bayesian approach is most effective when the uncertainty in both the observations and in the state estimate, as it evolves over time, are accurately quantified. In this article, we describe a practical method for data assimilation in large, spatiotemporally chaotic systems. The method is a type of ``ensemble Kalman filter'', in which the state estimate and its approximate uncertainty are represented at any given time by an ensemble of system states. We discuss both the mathematical basis of this approach and its implementation; our primary emphasis is on ease of use and computational speed rather than improving accuracy over previously published approaches to ensemble Kalman filtering. We include some numerical results demonstrating the efficiency and accuracy of our implementation for assimilating real atmospheric data with the global forecast model used by the U.S. National Weather Service.

physics.data-an↗

arXiv2006

Scale dependence of branching in arterial and bronchial trees

Although models of branching in arterial and bronchial trees often predict a dependence of bifurcation parameters on the scale of the bifurcating vessels, direct verifications of this dependence with data are uncommon. We compare measurements of bifurcation parameters in airways and arterial trees of different mammals as a function of scale to general features predicted by theoretical models. We find that the size dependence is more complex than existing theories based solely on energy minimization explain, and suggest additional factors that may govern the branching at different scales.

physics.bio-ph↗

arXiv2006

The emergence of coherence in complex networks of heterogeneous dynamical systems

We present a general theory for the onset of coherence in collections of heterogeneous maps interacting via a complex connection network. Our method allows the dynamics of the individual uncoupled systems to be either chaotic or periodic, and applies generally to networks for which the number of connections per node is large. We find that the critical coupling strength at which a transition to synchrony takes place depends separately on the dynamics of the individual uncoupled systems and on the largest eigenvalue of the adjacency matrix of the coupling network. Our theory directly generalizes the Kuramoto model of equal strength, all-to-all coupled phase oscillators to the case of oscillators with more realistic dynamics coupled via a large heterogeneous network.

cond-mat.dis-nn↗

arXiv2005

Synchronization in large directed networks of coupled phase oscillators

We extend recent theoretical approximations describing the transition to synchronization in large undirected networks of coupled phase oscillators to the case of directed networks. We also consider extensions to networks with mixed positive/negative coupling strengths. We compare our theory with numerical simulations and find good agreement.

cond-mat.stat-mech↗