arXiv subjects

Henry D. I. Abarbanel

Publications and source records attributed to Henry D. I. Abarbanel.

At least 19 recordsLinked to original sources

arXiv2025

Convolutional method for data assimilation An improved method on neuronal electrophysiological data

We present a convolution-based data assimilation method tailored to neuronal electrophysiology, addressing the limitations of traditional value-based synchronization approaches. While conventional methods rely on nudging terms and pointwise deviation metrics, they often fail to account for spike timing precision, a key feature in neural signals. Our approach applies a Gaussian convolution to both measured data and model estimates, enabling a cost function that evaluates both amplitude and timing alignment via spike overlap. This formulation remains compatible with gradient-based optimization. Through twin experiments and real hippocampal neuron recordings, we demonstrate improved parameter estimation and prediction quality, particularly in capturing sharp, time-sensitive dynamics.

q-bio.NC↗

arXiv2023

Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networks

Drawing on ergodic theory, we introduce a novel training method for machine learning based forecasting methods for chaotic dynamical systems. The training enforces dynamical invariants--such as the Lyapunov exponent spectrum and fractal dimension--in the systems of interest, enabling longer and more stable forecasts when operating with limited data. The technique is demonstrated in detail using the recurrent neural network architecture of reservoir computing. Results are given for the Lorenz 1996 chaotic dynamical system and a spectral quasi-geostrophic model, both typical test cases for numerical weather prediction.

cs.LG↗

arXiv2022

A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics

A reservoir computer (RC) is a type of simplified recurrent neural network architecture that has demonstrated success in the prediction of spatiotemporally chaotic dynamical systems. A further advantage of RC is that it reproduces intrinsic dynamical quantities essential for its incorporation into numerical forecasting routines such as the ensemble Kalman filter -- used in numerical weather prediction to compensate for sparse and noisy data. We explore here the architecture and design choices for a "best in class" RC for a number of characteristic dynamical systems, and then show the application of these choices in scaling up to larger models using localization. Our analysis points to the importance of large scale parameter optimization. We also note in particular the importance of including input bias in the RC design, which has a significant impact on the forecast skill of the trained RC model. In our tests, the the use of a nonlinear readout operator does not affect the forecast time or the stability of the forecast. The effects of the reservoir dimension, spinup time, amount of training data, normalization, noise, and the RC time step are also investigated. While we are not aware of a generally accepted best reported mean forecast time for different models in the literature, we report over a factor of 2 increase in the mean forecast time compared to the best performing RC model of Vlachas et.al (2020) for the 40 dimensional spatiotemporally chaotic Lorenz 1996 dynamics, and we are able to accomplish this using a smaller reservoir size.

cs.NE↗

arXiv2021

Robust Forecasting using Predictive Generalized Synchronization in Reservoir Computing

Reservoir computers (RC) are a form of recurrent neural network (RNN) used for forecasting timeseries data. As with all RNNs, selecting the hyperparameters presents a challenge when training onnew inputs. We present a method based on generalized synchronization (GS) that gives direction in designing and evaluating the architecture and hyperparameters of an RC. The 'auxiliary method' for detecting GS provides a computationally efficient pre-training test that guides hyperparameterselection. Furthermore, we provide a metric for RC using the reproduction of the input system's Lyapunov exponentsthat demonstrates robustness in prediction.

physics.comp-ph↗

arXiv2021

Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation

Data assimilation (DA) is integrated with machine learning in order to perform entirely data-driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as surrogate models to replace key components of the DA cycle in numerical weather prediction (NWP), including the conventional numerical forecast model, the forecast error covariance matrix, and the tangent linear and adjoint models. It is shown how these RNNs can be initialized using DA methods to directly update the hidden/reservoir state with observations of the target system. The results indicate that these techniques can be applied to estimate the state of a system for the repeated initialization of short-term forecasts, even in the absence of a traditional numerical forecast model. Further, it is demonstrated how these integrated RNN-DA methods can scale to higher dimensions by applying domain localization and parallelization, providing a path for practical applications in NWP.

cs.LG↗

arXiv2021

Forecasting Using Reservoir Computing: The Role of Generalized Synchronization

Reservoir computers (RC) are a form of recurrent neural network (RNN) used for forecasting time series data. As with all RNNs, selecting the hyperparameters presents a challenge when training on new inputs. We present a method based on generalized synchronization (GS) that gives direction in designing and evaluating the architecture and hyperparameters of a RC. The 'auxiliary method' for detecting GS provides a pre-training test that guides hyperparameter selection. Furthermore, we provide a metric for a "well trained" RC using the reproduction of the input system's Lyapunov exponents.

cs.NE↗

arXiv2020

Machine Learning Classification Informed by a Functional Biophysical System

We present a novel machine learning architecture for classification suggested by experiments on olfactory systems. The network separates input stimuli, represented as spatially distinct currents, via winnerless competition---a process based on the intrinsic sequential dynamics of the neural system---then uses a support vector machine (SVM) to provide precision to the space-time separation of the output. The combined network uses biophysical models of neurons and shows high discrimination among inputs and robustness to noise. While using the SVM alone does not permit determination of the components of mixtures of classified inputs, the combined network is able to tell the precise concentrations of the constituent parts.

physics.bio-ph↗

arXiv2020

Precision annealing Monte Carlo methods for statistical data assimilation and machine learning

In statistical data assimilation (SDA) and supervised machine learning (ML), we wish to transfer information from observations to a model of the processes underlying those observations. For SDA, the model consists of a set of differential equations that describe the dynamics of a physical system. For ML, the model is usually constructed using other strategies. In this paper, we develop a systematic formulation based on Monte Carlo sampling to achieve such information transfer. Following the derivation of an appropriate target distribution, we present the formulation based on the standard Metropolis-Hasting (MH) procedure and the Hamiltonian Monte Carlo (HMC) method for performing the high dimensional integrals that appear. To the extensive literature on MH and HMC, we add (1) an annealing method using a hyperparameter that governs the precision of the model to identify and explore the highest probability regions of phase space dominating those integrals, and (2) a strategy for initializing the state space search. The efficacy of the proposed formulation is demonstrated using a nonlinear dynamical model with chaotic solutions widely used in geophysics.

physics.data-an↗

arXiv2019

Machine Learning of Time Series Using Time-delay Embedding and Precision Annealing

Tasking machine learning to predict segments of a time series requires estimating the parameters of a ML model with input/output pairs from the time series. Using the equivalence between statistical data assimilation and supervised machine learning, we revisit this task. The training method for the machine utilizes a precision annealing approach to identifying the global minimum of the action (-log[P]). In this way we are able to identify the number of training pairs required to produce good generalizations (predictions) for the time series. We proceed from a scalar time series $s(t_n); t_n = t_0 + n Δt$ and using methods of nonlinear time series analysis show how to produce a $D_E > 1$ dimensional time delay embedding space in which the time series has no false neighbors as does the observed $s(t_n)$ time series. In that $D_E$-dimensional space we explore the use of feed forward multi-layer perceptrons as network models operating on $D_E$-dimensional input and producing $D_E$-dimensional outputs.

cs.LG↗

arXiv2019

Precision Annealing Monte Carlo Methods for Statistical Data Assimilation: Metropolis-Hastings Procedures

Statistical Data Assimilation (SDA) is the transfer of information from field or laboratory observations to a user selected model of the dynamical system producing those observations. The data is noisy and the model has errors; the information transfer addresses properties of the conditional probability distribution of the states of the model conditioned on the observations. The quantities of interest in SDA are the conditional expected values of functions of the model state, and these require the approximate evaluation of high dimensional integrals. We introduce a conditional probability distribution and use the Laplace method with annealing to identify the maxima of the conditional probability distribution. The annealing method slowly increases the precision term of the model as it enters the Laplace method. In this paper, we extend the idea of precision annealing (PA) to Monte Carlo calculations of conditional expected values using Metropolis-Hastings methods.

stat.CO↗

arXiv2018

Strategic Monte Carlo Methods for State and Parameter Estimation in High Dimensional Nonlinear Problems

In statistical data assimilation one seeks the largest maximum of the conditional probability distribution $P(\mathbf{X},\mathbf{p}|\mathbf{Y})$ of model states, $\mathbf{X}$, and parameters,$\mathbf{p}$, conditioned on observations $\mathbf{Y}$ through minimizing the `action', $A(\mathbf{X}) = -\log P(\mathbf{X},\mathbf{p}|\mathbf{Y})$. This determines the dominant contribution to the expected values of functions of $\mathbf{X}$ but does not give information about the structure of $P(\mathbf{X},\mathbf{p}|\mathbf{Y})$ away from the maximum. We introduce a Monte Carlo sampling method, called Strategic Monte Carlo (SMC) sampling, for estimating $P(\mathbf{X}, \mathbf{p}|\mathbf{Y})$ in the neighborhood of its largest maximum to remedy this limitation. SMC begins with a systematic variational annealing (VA) procedure for finding the smallest minimum of $A(\mathbf{X})$. SMC generates accurate estimates for the mean, standard deviation and other higher moments of $P(\mathbf{X},\mathbf{p}|\mathbf{Y})$. Additionally, the random search allows for an understanding of any multimodal structure that may underly the dynamics of the problem. SMC generates a gaussian probability control term based on the paths determined by VA to minimize a cost function $A(\mathbf{X},\mathbf{p})$. This probability is sampled during the Monte Carlo search of the cost function to constrain the search to high probability regions of the surface thus substantially reducing the time necessary to sufficiently explore the space.

stat.ME↗

arXiv2017

An optimization-based approach to calculating neutrino flavor evolution

We assess the utility of an optimization-based data assimilation (D.A.) technique for treating the problem of nonlinear neutrino flavor transformation in core collapse supernovae. D.A. uses measurements obtained from a physical system to estimate the state variable evolution and parameter values of the associated model. Formulated as an optimization procedure, D.A. can offer an integration-blind approach to predicting model evolution, which offers an advantage for models that thwart solution via traditional numerical integration techniques. Further, D.A. performs most optimally for models whose equations of motion are nonlinearly coupled. In this exploratory work, we consider a simple steady-state model with two mono-energetic neutrino beams coherently interacting with each other and a background medium. As this model can be solved via numerical integration, we have an independent consistency check for D.A. solutions. We find that the procedure can capture key features of flavor evolution over the entire trajectory, even given measurements of neutrino flavor only at the endpoint, and with an assumed known initial flavor distribution. Further, the procedure permits an examination of the sensitivity of flavor evolution to estimates of unknown model parameters, locates degeneracies in parameter space, and can identify the specific measurements required to break those degeneracies.

astro-ph.HE↗

arXiv2016

Model of the Songbird Nucleus HVC as a Network of Central Pattern Generators

We propose a functional architecture of the adult songbird nucleus HVC in which the core element is a "functional syllable unit" (FSU). In this model, HVC is organized into FSUs, each of which provides the basis for the production of one syllable in vocalization. Within each FSU, the inhibitory neuron population takes one of two operational states: (A) simultaneous firing wherein all inhibitory neurons fire simultaneously, and (B) competitive firing of the inhibitory neurons. Switching between these basic modes of activity is accomplished via changes in the synaptic strengths among the inhibitory neurons. The inhibitory neurons connect to excitatory projection neurons such that during state (A) the activity of projection neurons is suppressed, while during state (B) patterns of sequential firing of projection neurons can occur. The latter state is stabilized by feedback from the projection to the inhibitory neurons. Song composition for specific species is distinguished by the manner in which different FSUs are functionally connected to each other. Ours is a computational model built with biophysically based neurons. We illustrate that many observations of HVC activity are explained by the dynamics of the proposed population of FSUs, and we identify aspects of the model that are currently testable experimentally. In addition, and standing apart from the core features of an FSU, we propose that the transition between modes may be governed by the biophysical mechanism of neuromodulation.

q-bio.NC↗

arXiv2015

Basin structure of optimization based state and parameter estimation

Most data based state and parameter estimation methods require suitable initial values or guesses to achieve convergence to the desired solution, which typically is a global minimum of some cost function. Unfortunately, however, other stable solutions (e.g., local minima) may exist and provide suboptimal or even wrong estimates. Here we demonstrate for a 9-dimensional Lorenz-96 model how to characterize the basin size of the global minimum when applying some particular optimization based estimation algorithm. We compare three different strategies for generating suitable initial guesses and we investigate the dependence of the solution on the given trajectory segment (underlying the measured time series). To address the question of how many state variables have to be measured for optimal performance, different types of multivariate time series are considered consisting of 1, 2, or 3 variables. Based on these time series the local observability of state variables and parameters of the Lorenz-96 model is investigated and confirmed using delay coordinates. This result is in good agreement with the observation that correct state and parameter estimation results are obtained if the optimization algorithm is initialized with initial guesses close to the true solution. In contrast, initialization with other exact solutions of the model equations (different from the true solution used to generate the time series) typically fails, i.e. the optimization procedure ends up in local minima different from the true solution. Initialization using random values in a box around the attractor exhibits success rates depending on the number of observables and the available time series (trajectory segment).

nlin.CD↗

arXiv2014

Accurately Estimating the State of a Geophysical System with Sparse Observations: Predicting the Weather

Utilizing the information in observations of a complex system to make accurate predictions through a quantitative model when observations are completed at time $T$, requires an accurate estimate of the full state of the model at time $T$. When the number of measurements $L$ at each observation time within the observation window is larger than a sufficient minimum value $L_s$, the impediments in the estimation procedure are removed. As the number of available observations is typically such that $L \ll L_s$, additional information from the observations must be presented to the model. We show how, using the time delays of the measurements at each observation time, one can augment the information transferred from the data to the model, removing the impediments to accurate estimation and permitting dependable prediction. We do this in a core geophysical fluid dynamics model, the shallow water equations, at the heart of numerical weather prediction. The method is quite general, however, and can be utilized in the analysis of a broad spectrum of complex systems where measurements are sparse. When the model of the complex system has errors, the method still enables accurate estimation of the state of the model and thus evaluation of the model errors in a manner separated from uncertainties in the data assimilation procedure.

nlin.CD↗

arXiv2011

Data Assimilation using a GPU Accelerated Path Integral Monte Carlo Approach

The answers to data assimilation questions can be expressed as path integrals over all possible state and parameter histories. We show how these path integrals can be evaluated numerically using a Markov Chain Monte Carlo method designed to run in parallel on a Graphics Processing Unit (GPU). We demonstrate the application of the method to an example with a transmembrane voltage time series of a simulated neuron as an input, and using a Hodgkin-Huxley neuron model. By taking advantage of GPU computing, we gain a parallel speedup factor of up to about 300, compared to an equivalent serial computation on a CPU, with performance increasing as the length of the observation time used for data assimilation increases.

physics.comp-ph↗

arXiv2010

Self-Consistent Stochastic Model Errors in Data Assimilation

In using data assimilation to import information from observations to estimate parameters and state variables of a model, one must assume a distribution for the noise in the measurements and in the model errors. Using the path integral formulation of data assimilation~ cite{abar2009}, we introduce the idea of self consistency of the distribution of stochastic model errors: the distribution of model errors from the path integral with observed data should be consistent with the assumption made in formulating the the path integral. The path integral setting for data assimilation is discussed to provide the setting for the consistency test. Using two examples drawn from the 1996 Lorenz model, for $D = 100$ and for $D = 20$ we show how one can test for this inconsistency with essential no additional effort than that expended in extracting answers to interesting questions from data assimilation itself. \end{abstract}

nlin.CD↗

arXiv2009

State and parameter estimation using Monte Carlo evaluation of path integrals

Transferring information from observations of a dynamical system to estimate the fixed parameters and unobserved states of a system model can be formulated as the evaluation of a discrete time path integral in model state space. The observations serve as a guiding potential working with the dynamical rules of the model to direct system orbits in state space. The path integral representation permits direct numerical evaluation of the conditional mean path through the state space as well as conditional moments about this mean. Using a Monte Carlo method for selecting paths through state space we show how these moments can be evaluated and demonstrate in an interesting model system the explicit influence of the role of transfer of information from the observations. We address the question of how many observations are required to estimate the unobserved state variables, and we examine the assumptions of Gaussianity of the underlying conditional probability.

nlin.CD↗