arXiv subjects

Edward Ott

Publications and source records attributed to Edward Ott.

At least 19 recordsLinked to original sources

arXiv2025

Tailored Forecasting from Short Time Series via Meta-learning

Machine learning models can effectively forecast dynamical systems from time-series data, but they typically require large amounts of past data, making forecasting particularly challenging for systems with limited history. To overcome this, we introduce Meta-learning for Tailored Forecasting using Related Time Series (METAFORS), which generalizes knowledge across systems to enable forecasting in data-limited scenarios. By learning from a library of models trained on longer time series from potentially related systems, METAFORS builds and initializes a model tailored to short time-series data from the system of interest. Using a reservoir computing implementation and testing on simulated chaotic systems, we demonstrate that METAFORS can reliably predict both short-term dynamics and long-term statistics without requiring contextual labels. We see this even when test and related systems exhibit substantially different behaviors, highlighting METAFORS' strengths in data-limited scenarios.

cs.LG↗

arXiv2024

Prediction Beyond the Medium Range with an Atmosphere-Ocean Model that Combines Physics-based Modeling and Machine Learning

This paper explores the potential of a hybrid modeling approach that combines machine learning (ML) with conventional physics-based modeling for weather prediction beyond the medium range. It extends the work of Arcomano et al. (2022), which tested the approach for short- and medium-range weather prediction, and the work of Arcomano et al. (2023), which investigated its potential for climate modeling. The hybrid model used for the forecast experiments of the paper is based on the low-resolution, simplified parameterization atmospheric general circulation model SPEEDY. In addition to the hybridized prognostic variables of SPEEDY, the model has three purely ML-based prognostic variables: the 6h cumulative precipitation, the sea surface temperature, and the heat content of the top 300m deep layer of the ocean (a new addition compared to the model used in Arcomano et al., 2023). The model has skill in predicting the El Nino cycle and its global teleconnections with precipitation for 3-7 months depending on the season. The model captures equatorial variability of the precipitation associated with Kelvin and Rossby waves and MJO. Predictions of the precipitation in the equatorial region have skill for 15 days in the East Pacific and 11.5 days in the West Pacific. Though the model has low spatial resolution, for these tasks it has prediction skill comparable to what has been published for high-resolution, purely physics-based, conventional, operational forecast models.

physics.ao-ph↗

arXiv2022

Stabilizing Machine Learning Prediction of Dynamics: Noise and Noise-inspired Regularization

Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this technique can result in artificially rapid error growth. In this article, we systematically examine the technique of adding noise to the ML model input during training to promote stability and improve prediction accuracy. Furthermore, we introduce Linearized Multi-Noise Training (LMNT), a regularization technique that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. Our case study uses reservoir computing, a machine-learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while reservoir computers trained without regularization are unstable. Compared with other regularization techniques that yield stability in some cases, we find that both short-term and climate predictions from reservoir computers trained with noise or with LMNT are substantially more accurate. Finally, we show that the deterministic aspect of our LMNT regularization facilitates fast hyperparameter tuning when compared to training with noise.

cs.LG↗

arXiv2022

Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems

In this paper we consider the machine learning (ML) task of predicting tipping point transitions and long-term post-tipping-point behavior associated with the time evolution of an unknown (or partially unknown), non-stationary, potentially noisy and chaotic, dynamical system. We focus on the particularly challenging situation where the past dynamical state time series that is available for ML training predominantly lies in a restricted region of the state space, while the behavior to be predicted evolves on a larger state space set not fully observed by the ML model during training. In this situation, it is required that the ML prediction system have the ability to extrapolate to different dynamics past that which is observed during training. We investigate the extent to which ML methods are capable of accomplishing useful results for this task, as well as conditions under which they fail. In general, we found that the ML methods were surprisingly effective even in situations that were extremely challenging, but do (as one would expect) fail when ``too much" extrapolation is required. For the latter case, we investigate the effectiveness of combining the ML approach with conventional modeling based on scientific knowledge, thus forming a hybrid prediction system which we find can enable useful prediction even when its ML-based and knowledge-based components fail when acting alone. We also found that achieving useful results may require using very carefully selected ML hyperparameters and we propose a hyperparameter optimization strategy to address this problem. The main conclusion of this paper is that ML-based approaches are promising tools for predicting the behavior of non-stationary dynamical systems even in the case where the future evolution (perhaps due to the crossing of a tipping point) includes dynamics on a set outside of that explored by the training data.

cs.LG↗

arXiv2022

Short-wavelength Reverberant Wave Systems for Physical Realization of Reservoir Computing

Machine learning (ML) has found widespread application over a broad range of important tasks. To enhance ML performance, researchers have investigated computational architectures whose physical implementations promise compactness, high-speed execution, physical robustness, and low energy cost. Here, we experimentally demonstrate an approach that uses the high sensitivity of reverberant short wavelength waves for physical realization and enhancement of computational power of a type of ML known as reservoir computing (RC). The potential computation power of RC systems increases with their effective size. We here exploit the intrinsic property of short wavelength reverberant wave sensitivity to perturbations to expand the effective size of the RC system by means of spatial and spectral perturbations. Working in the microwave regime, this scheme is tested experimentally on different ML tasks. Our results indicate the general applicability of reverberant wave-based implementations of RC and of our effective reservoir size expansion technique

eess.SP↗

arXiv2021

Parallel Machine Learning for Forecasting the Dynamics of Complex Networks

Forecasting the dynamics of large complex networks from previous time-series data is important in a wide range of contexts. Here we present a machine learning scheme for this task using a parallel architecture that mimics the topology of the network of interest. We demonstrate the utility and scalability of our method implemented using reservoir computing on a chaotic network of oscillators. Two levels of prior knowledge are considered: (i) the network links are known; and (ii) the network links are unknown and inferred via a data-driven approach to approximately optimize prediction.

cs.LG↗

arXiv2021

Deep learning estimation of complex reverberant wave fields by a programmable metasurface

Electromagnetic environments are becoming increasingly complex and congested, creating a growing challenge for systems that rely on electromagnetic waves for communication, sensing, or imaging, particularly in reverberating environments. The use of programmable metasurfaces provides a potential means of directing waves to optimize wireless channels on-demand, ensuring reliable operation and protecting sensitive electronic components. Here we introduce a technique that combines a deep learning network with a binary programmable metasurface to shape waves in complex reverberant electromagnetic environments, in particular ones where there is no direct line of sight. We applied this technique for wavefront reconstruction and control, and accurately determined metasurface configurations based on measured system scattering responses in a chaotic microwave cavity. The state of the metasurface that realizes desired electromagnetic wave field distribution properties was successfully determined even in cases previously unseen by the deep learning algorithm. Our technique is enabled by the reverberant nature of the cavity, and is effective with a metasurface that covers only $\sim$1.5\% of the total cavity surface area.

physics.app-ph↗

arXiv2021

Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components

We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is in the form of noisy partial measurements of the past and present state of the dynamical system. Recently there have been several promising data-driven approaches to forecasting of chaotic dynamical systems using machine learning. Particularly promising among these are hybrid approaches that combine machine learning with a knowledge-based model, where a machine-learning technique is used to correct the imperfections in the knowledge-based model. Such imperfections may be due to incomplete understanding and/or limited resolution of the physical processes in the underlying dynamical system, e.g., the atmosphere or the ocean. Previously proposed data-driven forecasting approaches tend to require, for training, measurements of all the variables that are intended to be forecast. We describe a way to relax this assumption by combining data assimilation with machine learning. We demonstrate this technique using the Ensemble Transform Kalman Filter (ETKF) to assimilate synthetic data for the 3-variable Lorenz system and for the Kuramoto-Sivashinsky system, simulating model error in each case by a misspecified parameter value. We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.

cs.LG↗

arXiv2020

Machine Learning Link Inference of Noisy Delay-coupled Networks with Opto-Electronic Experimental Tests

We devise a machine learning technique to solve the general problem of inferring network links that have time-delays. The goal is to do this purely from time-series data of the network nodal states. This task has applications in fields ranging from applied physics and engineering to neuroscience and biology. To achieve this, we first train a type of machine learning system known as reservoir computing to mimic the dynamics of the unknown network. We formulate and test a technique that uses the trained parameters of the reservoir system output layer to deduce an estimate of the unknown network structure. Our technique, by its nature, is non-invasive, but is motivated by the widely-used invasive network inference method whereby the responses to active perturbations applied to the network are observed and employed to infer network links (e.g., knocking down genes to infer gene regulatory networks). We test this technique on experimental and simulated data from delay-coupled opto-electronic oscillator networks. We show that the technique often yields very good results particularly if the system does not exhibit synchrony. We also find that the presence of dynamical noise can strikingly enhance the accuracy and ability of our technique, especially in networks that exhibit synchrony.

nlin.AO↗

arXiv2020

Wavefront Shaping with a Tunable Metasurface: Creating Coldspots and Coherent Perfect Absorption at Arbitrary Frequencies

Modern electronic systems operate in complex electromagnetic environments and must handle noise and unwanted coupling. The capability to isolate or reject unwanted signals for mitigating vulnerabilities is critical in any practical application. In this work, we describe the use of a binary programmable metasurface to (i) control the spatial degrees of freedom for waves propagating inside an electromagnetic cavity and demonstrate the ability to create nulls in the transmission coefficient between selected ports; and (ii) create the conditions for coherent perfect absorption. Both objectives are performed at arbitrary frequencies. In the first case a novel and effective optimization algorithm is presented that selectively generates coldspots over a single frequency band or simultaneously over multiple frequency bands. We show that this algorithm is successful with multiple input port configurations and varying optimization bandwidths. In the second case we establish how this technique can be used to establish a multi-port coherent perfect absorption state for the cavity.

physics.app-ph↗

arXiv2020

Efficient Statistical Model for Predicting Electromagnetic Wave Distribution in Coupled Enclosures

The Random Coupling Model (RCM) has been successfully applied to predicting the statistics of currents and voltages at ports in complex electromagnetic (EM) enclosures operating in the short wavelength limit. Recent studies have extended the RCM to systems of multi-mode aperture-coupled enclosures. However, as the size (as measured in wavelengths) of a coupling aperture grows, the coupling matrix used in the RCM increases as well, and the computation becomes more complex and time-consuming. A simple Power Balance Model (PWB) can provide fast predictions for the \textit{averaged} power density of waves inside electrically-large systems for a wide range of cavity and coupling scenarios. However, the important interference induced fluctuations of the wavefield retained in the RCM is absent in PWB. Here we aim to combine the best aspects of each model to create a hybrid treatment and study the EM fields in coupled enclosure systems. The proposed hybrid approach provides both mean and fluctuation information of the EM fields without the full computational complexity of coupled-cavity RCM. We compare the hybrid model predictions with experiments on linear cascades of over-moded cavities. We find good agreement over a set of different loss parameters and for different coupling strengths between cavities. The range of validity and applicability of the hybrid method are tested and discussed.

physics.class-ph↗

arXiv2020

Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems

We consider the commonly encountered situation (e.g., in weather forecasting) where the goal is to predict the time evolution of a large, spatiotemporally chaotic dynamical system when we have access to both time series data of previous system states and an imperfect model of the full system dynamics. Specifically, we attempt to utilize machine learning as the essential tool for integrating the use of past data into predictions. In order to facilitate scalability to the common scenario of interest where the spatiotemporally chaotic system is very large and complex, we propose combining two approaches:(i) a parallel machine learning prediction scheme; and (ii) a hybrid technique, for a composite prediction system composed of a knowledge-based component and a machine-learning-based component. We demonstrate that not only can this method combining (i) and (ii) be scaled to give excellent performance for very large systems, but also that the length of time series data needed to train our multiple, parallel machine learning components is dramatically less than that necessary without parallelization. Furthermore, considering cases where computational realization of the knowledge-based component does not resolve subgrid-scale processes, our scheme is able to use training data to incorporate the effect of the unresolved short-scale dynamics upon the resolved longer-scale dynamics ("subgrid-scale closure").

cs.LG↗

arXiv2019

Using Machine Learning to Assess Short Term Causal Dependence and Infer Network Links

We introduce and test a general machine-learning-based technique for the inference of short term causal dependence between state variables of an unknown dynamical system from time series measurements of its state variables. Our technique leverages the results of a machine learning process for short time prediction to achieve our goal. The basic idea is to use the machine learning to estimate the elements of the Jacobian matrix of the dynamical flow along an orbit. The type of machine learning that we employ is reservoir computing. We present numerical tests on link inference of a network of interacting dynamical nodes. It is seen that dynamical noise can greatly enhance the effectiveness of our technique, while observational noise degrades the effectiveness. We believe that the competition between these two opposing types of noise will be the key factor determining the success of causal inference in many of the most important application situations.

nlin.AO↗

arXiv2019

Separation of Chaotic Signals by Reservoir Computing

We demonstrate the utility of machine learning in the separation of superimposed chaotic signals using a technique called Reservoir Computing. We assume no knowledge of the dynamical equations that produce the signals, and require only training data consisting of finite time samples of the component signals. We test our method on signals that are formed as linear combinations of signals from two Lorenz systems with different parameters. Comparing our nonlinear method with the optimal linear solution to the separation problem, the Wiener filter, we find that our method significantly outperforms the Wiener filter in all the scenarios we study. Furthermore, this difference is particularly striking when the component signals have similar frequency spectra. Indeed, our method works well when the component frequency spectra are indistinguishable - a case where a Wiener filter performs essentially no separation.

eess.SP↗

arXiv2019

Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics

We examine the efficiency of Recurrent Neural Networks in forecasting the spatiotemporal dynamics of high dimensional and reduced order complex systems using Reservoir Computing (RC) and Backpropagation through time (BPTT) for gated network architectures. We highlight advantages and limitations of each method and discuss their implementation for parallel computing architectures. We quantify the relative prediction accuracy of these algorithms for the longterm forecasting of chaotic systems using as benchmarks the Lorenz-96 and the Kuramoto-Sivashinsky (KS) equations. We find that, when the full state dynamics are available for training, RC outperforms BPTT approaches in terms of predictive performance and in capturing of the long-term statistics, while at the same time requiring much less training time. However, in the case of reduced order data, large scale RC models can be unstable and more likely than the BPTT algorithms to diverge. In contrast, RNNs trained via BPTT show superior forecasting abilities and capture well the dynamics of reduced order systems. Furthermore, the present study quantifies for the first time the Lyapunov Spectrum of the KS equation with BPTT, achieving similar accuracy as RC. This study establishes that RNNs are a potent computational framework for the learning and forecasting of complex spatiotemporal systems.

eess.SP↗

arXiv2019

Wave scattering properties of multiple weakly-coupled complex systems

The statistics of scattering of waves inside single ray-chaotic enclosures have been successfully described by the Random Coupling Model (RCM). We expand the RCM to systems consisting of multiple complex ray-chaotic enclosures with variable coupling scenarios. The statistical properties of the model-generated quantities are tested against measured data of electrically large multi-cavity systems of various designs. The statistics of model-generated trans-impedance and induced voltages on a load impedance agree well with the experimental results. The RCM coupled chaotic enclosure model is general and can be applied to other physical systems including coupled quantum dots, disordered nanowires, and short-wavelength electromagnetic propagation through rooms in buildings, aircraft and ships.

physics.class-ph↗

arXiv2019

Classification and prediction of wave chaotic systems with machine learning techniques

The wave properties of complex scattering systems that are large compared to the wavelength, and show chaos in the classical limit, are extremely sensitive to system details. A solution to the wave equation for a specific configuration can change substantially under small perturbations. Due to this extreme sensitivity, it is difficult to discern basic information about a complex system simply from scattering data as a function of energy or frequency, at least by eye. In this work, we employ supervised machine learning algorithms to reveal and classify hidden information about the complex scattering system presented in the data. As an example we are able to distinguish the total number of connected cavities in a linear chain of weakly coupled lossy enclosures from measured reflection data. A predictive machine learning algorithm for the future states of a perturbed complex scattering system is also trained with a recurrent neural network. Given a finite training data series, the reflection/transmission properties can be forecast by the proposed algorithm.

cond-mat.dis-nn↗

arXiv2019

Critical Network Cascades with Re-excitable nodes: Why tree-like approximations usually work, when they breakdown, and how to correct them

Network science is a rapidly expanding field, with a large and growing body of work on network-based dynamical processes. Most theoretical results in this area rely on the so-called \emph{locally tree-like approximation}. This is, however, usually an `uncontrolled' approximation, in the sense that the magnitudes of the error are typically unknown, although numerical results show that this error is often surprisingly small. In this paper, we place this approximation on more rigorous footing by calculating the magnitude of deviations away from tree-based theories in the context of discrete-time critical network cascades with re-excitable nodes. We discuss the conditions under which tree-like approximations give good results for calculating network criticality, and also explain the reasons for deviation from this approximation, in terms of the density of certain kinds of network motifs. Using this understanding, we derive results for network criticality that apply to general networks that explicitly do not satisfy the locally tree-like approximation. In particular, we focus on the bi-parallel motif, the smallest motif relevant to the failure of a tree-based theory in this context, and we derive the corrections due to such motifs on the conditions for criticality. We verify our claims on computer-generated networks, and we confirm that our theory accurately predicts the observed deviations from criticality. Using our theory, we explain why numerical simulations often show that deviations from a tree-based theory are surprisingly small. More specifically, we show that these deviations are negligible for networks whose average degree is even modestly large compared to one, justifying why tree-based theories appear to work well for most real-world networks.

physics.soc-ph↗