arXiv subjects

Themistoklis P. Sapsis

Publications and source records attributed to Themistoklis P. Sapsis.

At least 37 records · Page 2Linked to original sources

arXiv2020

Sparse Methods for Automatic Relevance Determination

This work considers methods for imposing sparsity in Bayesian regression with applications in nonlinear system identification. We first review automatic relevance determination (ARD) and analytically demonstrate the need to additional regularization or thresholding to achieve sparse models. We then discuss two classes of methods, regularization based and thresholding based, which build on ARD to learn parsimonious solutions to linear problems. In the case of orthogonal covariates, we analytically demonstrate favorable performance with regards to learning a small set of active terms in a linear system with a sparse solution. Several example problems are presented to compare the set of proposed methods in terms of advantages and limitations to ARD in bases with hundreds of elements. The aim of this paper is to analyze and understand the assumptions that lead to several algorithms and to provide theoretical and empirical results so that the reader may gain insight and make more informed choices regarding sparse Bayesian regression.

stat.ML↗

arXiv2020

Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics

We examine the efficiency of Recurrent Neural Networks in forecasting the spatiotemporal dynamics of high dimensional and reduced order complex systems using Reservoir Computing (RC) and Backpropagation through time (BPTT) for gated network architectures. We highlight advantages and limitations of each method and discuss their implementation for parallel computing architectures. We quantify the relative prediction accuracy of these algorithms for the longterm forecasting of chaotic systems using as benchmarks the Lorenz-96 and the Kuramoto-Sivashinsky (KS) equations. We find that, when the full state dynamics are available for training, RC outperforms BPTT approaches in terms of predictive performance and in capturing of the long-term statistics, while at the same time requiring much less training time. However, in the case of reduced order data, large scale RC models can be unstable and more likely than the BPTT algorithms to diverge. In contrast, RNNs trained via BPTT show superior forecasting abilities and capture well the dynamics of reduced order systems. Furthermore, the present study quantifies for the first time the Lyapunov Spectrum of the KS equation with BPTT, achieving similar accuracy as RC. This study establishes that RNNs are a potent computational framework for the learning and forecasting of complex spatiotemporal systems.

eess.SP↗

arXiv2019

A Gaussian moment method and its augmentation via LSTM recurrent neural networks for the statistics of cavitating bubble populations

Phase-averaged dilute bubbly flow models require high-order statistical moments of the bubble population. The method of classes, which directly evolve bins of bubbles in the probability space, are accurate but computationally expensive. Moment-based methods based upon a Gaussian closure present an opportunity to accelerate this approach, particularly when the bubble size distributions are broad (polydisperse). For linear bubble dynamics a Gaussian closure is exact, but for bubbles undergoing large and nonlinear oscillations, it results in a large error from misrepresented higher-order moments. Long short-term memory recurrent neural networks, trained on Monte Carlo truth data, are proposed to improve these model predictions. The networks are used to correct the low-order moment evolution equations and improve prediction of higher-order moments based upon the low-order ones. Results show that the networks can reduce model errors to less than $1\%$ of their unaugmented values.

physics.flu-dyn↗

arXiv2019

Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples

For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify its statistics, using the minimum number of function evaluations. This problem can been seen in the context of active learning or optimal experimental design. We employ Bayesian regression to represent the derived model uncertainty due to finite and small number of input-output pairs. In this context we evaluate existing methods for optimal sample selection, such as model error minimization and mutual information maximization. We show that for the case of known output variance, the commonly employed criteria in the literature do not take into account the output values of the existing input-output pairs, while for the case of unknown output variance this dependence can be very weak. We introduce a criterion that takes into account the values of the output for the existing samples and adaptively selects inputs from regions of the parameter space which have important contribution to the output. The new method allows for application to high-dimensional inputs, paving the way for optimal experimental design in high-dimensions.

stat.ML↗

arXiv2019

Learning the Tangent Space of Dynamical Instabilities from Data

For a large class of dynamical systems, the optimally time-dependent (OTD) modes, a set of deformable orthonormal tangent vectors that track directions of instabilities along any trajectory, are known to depend "pointwise" on the state of the system on the attractor, and not on the history of the trajectory. We leverage the power of neural networks to learn this "pointwise" mapping from phase space to OTD space directly from data. The result of the learning process is a cartography of directions associated with strongest instabilities in phase space. Implications for data-driven prediction and control of dynamical instabilities are discussed.

physics.comp-ph↗

arXiv2019

Bubbles in Turbulent Flows: Data-driven, kinematic models with memory terms

We present data driven kinematic models for the motion of bubbles in high-Re turbulent fluid flows based on recurrent neural networks with long-short term memory enhancements. The models extend empirical relations, such as Maxey-Riley (MR) and its variants, whose applicability is limited when either the bubble size is large or the flow is very complex. The recurrent neural networks are trained on the trajectories of bubbles obtained by Direct Numerical Simulations (DNS) of the Navier Stokes equations for a two-component incompressible flow model. Long short term memory components exploit the time history of the flow field that the bubbles have encountered along their trajectories and the networks are further augmented by imposing rotational invariance to their structure. We first train and validate the formulated model using DNS data for a turbulent Taylor-Green vortex. Then we examine the model predictive capabilities and its generalization to Reynolds numbers that are different from those of the training data on benchmark problems, including a steady (Hill's spherical vortex) and an unsteady (Gaussian vortex ring) flow field. We find that the predictions of the developed model are significantly improved compared with those obtained by the MR equation. Our results indicate that data-driven models with history terms are well suited in capturing the trajectories of bubbles in turbulent flows.

physics.flu-dyn↗

arXiv2019

Machine-Learning Ocean Dynamics from Lagrangian Drifter Trajectories

Lagrangian ocean drifters provide highly accurate approximations of ocean surface currents but are sparsely located across the globe. As drifters passively follow ocean currents, there is minimal control on where they will be making measurements, providing limited temporal coverage for a given region. Complementary Eulerian velocity data is available with global coverage, but is itself limited by the spatial and temporal resolution possible with satellite altimetry measurements. Additionally, altimetry measurements approximate geostrophic components of ocean currents but neglect smaller sub-mesoscale motions and require smoothing and interpolation from raw satellite track measurements. In an effort to harness the rich dynamics available in ocean drifter datasets, we have trained a recurrent neural network on the time history of drifter motion to minimize the error in a reduced-order Maxey-Riley drifter model. This approach relies on a slow-manifold approximation to determine the most mathematically-relevant variables with which to train, subsequently improving the temporal and spatial resolution of the underlying velocity field. By adding this neural network component, we also correct drifter trajectories near sub-mesoscale features missed by deterministic models using only satellite and wind reanalysis data. The effect of varying similarity between training and testing trajectory datasets for the blended model was evaluated, as was the effect of seasonality in the Gulf of Mexico.

math.DS↗

arXiv2019

Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.

physics.comp-ph↗

arXiv2019

Closed-loop adaptive control of extreme events in a turbulent flow

Extreme events that arise spontaneously in chaotic dynamical systems often have an adverse impact on the system or the surrounding environment. As such, their mitigation is highly desirable. Here, we introduce a novel control strategy for mitigating extreme events in a turbulent shear flow. The controller combines a probabilistic prediction of the extreme events with a deterministic actuator. The predictions are used to actuate the controller only when an extreme event is imminent. When actuated, the controller only acts on the degrees of freedom that are involved in the formation of the extreme events, exerting minimal interference with the flow dynamics. As a result, the attractors of the controlled and uncontrolled systems share the same chaotic core (containing the non-extreme events) and only differ in the tail of their distributions. We propose that such adaptive low-dimensional controllers should be used to mitigate extreme events in general chaotic dynamical systems, beyond the shear flow considered here.

physics.flu-dyn↗

arXiv2018

Are extreme dissipation events predictable in turbulent fluid flows?

We derive precursors of extreme dissipation events in a turbulent channel flow. Using a recently developed method that combines dynamics and statistics for the underlying attractor, we extract a characteristic state that precedes laminarization events that subsequently lead to extreme dissipation episodes. Our approach utilizes coarse statistical information for the turbulent attractor, in the form of second order statistics, to identify high-likelihood regions in the state space. We then search within this high probability manifold for the state that leads to the most finite-time growth of the flow kinetic energy. This state has both high probability of occurrence and leads to extreme values of dissipation. We use the alignment between a given turbulent state and this critical state as a precursor for extreme events and demonstrate its favorable properties for prediction of extreme dissipation events. Finally, we analyze the physical relevance of the derived precursor and show its robust character for different Reynolds numbers. Overall, we find that our choice of precursor works well at the Reynolds number it is computed at and at higher Reynolds number flows with similar extreme events.

physics.flu-dyn↗

arXiv2018

Data-assisted reduced-order modeling of extreme events in complex dynamical systems

Dynamical systems with high intrinsic dimensionality are often characterized by extreme events having the form of rare transitions several standard deviations away from the mean. For such systems, order-reduction methods through projection of the governing equations have limited applicability due to the large intrinsic dimensionality of the underlying attractor but also the complexity of the transient events. An alternative approach is data-driven techniques that aim to quantify the dynamics of specific modes utilizing data-streams. Several of these approaches have improved performance by expanding the state representation using delayed coordinates. However, such strategies are limited in regions of the phase space where there is a small amount of data available, as is the case for extreme events. In this work, we develop a blended framework that integrates an imperfect model, obtained from projecting equations into a subspace that still contains crucial dynamical information, with data-streams through a recurrent neural network (RNN) architecture. In particular, we employ the long-short-term memory (LSTM), to model portions of the dynamics which cannot be accounted by the equations. The RNN is trained by analyzing the mismatch between the imperfect model and the data-streams, projected in the reduced-order space. In this way, the data-driven model improves the imperfect model in regions where data is available, while for locations where data is sparse the imperfect model still provides a baseline for the prediction of the system dynamics. We assess the developed framework on two challenging prototype systems exhibiting extreme events and show that the blended approach has improved performance compared with methods that use either data streams or the imperfect model alone. The improvement is more significant in regions associated with extreme events, where data is sparse.

nlin.CD↗

arXiv2018

A sequential sampling strategy for extreme event statistics in nonlinear dynamical systems

We develop a method for the evaluation of extreme event statistics associated with nonlinear dynamical systems, using a small number of samples. From an initial dataset of design points, we formulate a sequential strategy that provides the 'next-best' data point (set of parameters) that when evaluated results in improved estimates of the probability density function (pdf) for a scalar quantity of interest. The approach utilizes Gaussian process regression to perform Bayesian inference on the parameter-to-observation map describing the quantity of interest. We then approximate the desired pdf along with uncertainty bounds utilizing the posterior distribution of the inferred map. The 'next-best' design point is sequentially determined through an optimization procedure that selects the point in parameter space that maximally reduces uncertainty between the estimated bounds of the pdf prediction. Since the optimization process utilizes only information from the inferred map it has minimal computational cost. Moreover, the special form of the metric emphasizes the tails of the pdf. The method is practical for systems where the dimensionality of the parameter space is of moderate size, i.e. order O(10). We apply the method to estimate the extreme event statistics for a very high-dimensional system with millions of degrees of freedom: an offshore platform subjected to three-dimensional irregular waves. It is demonstrated that the developed approach can accurately determine the extreme event statistics using limited number of samples.

cs.LG↗

arXiv2018

Extreme Events: Mechanisms and Prediction

Extreme events, such as rogue waves, earthquakes and stock market crashes, occur spontaneously in many dynamical systems. Because of their usually adverse consequences, quantification, prediction and mitigation of extreme events are highly desirable. Here, we review several aspects of extreme events in phenomena described by high-dimensional, chaotic dynamical systems. We specially focus on two pressing aspects of the problem: (i) Mechanisms underlying the formation of extreme events and (ii) Real-time prediction of extreme events. For each aspect, we explore methods relying on models, data or both. We discuss the strengths and limitations of each approach as well as possible future research directions.

nlin.CD↗

arXiv2018

Model order reduction for stochastic dynamical systems with continuous symmetries

Stochastic dynamical systems with continuous symmetries arise commonly in nature and often give rise to coherent spatio-temporal patterns. However, because of their random locations, these patterns are not well captured by current order reduction techniques and a large number of modes is typically necessary for an accurate solution. In this work, we introduce a new methodology for efficient order reduction of such systems by combining (i) the method of slices, a symmetry reduction tool, with (ii) any standard order reduction technique, resulting in efficient mixed symmetry-dimensionality reduction schemes. In particular, using the Dynamically Orthogonal (DO) equations in the second step, we obtain a novel nonlinear Symmetry-reduced Dynamically Orthogonal (SDO) scheme. We demonstrate the performance of the SDO scheme on stochastic solutions of the 1D Korteweg-de Vries and 2D Navier-Stokes equations.

physics.comp-ph↗

arXiv2017

A variational approach to probing extreme events in turbulent dynamical systems

Extreme events are ubiquitous in a wide range of dynamical systems, including turbulent fluid flows, nonlinear waves, large scale networks and biological systems. Here, we propose a variational framework for probing conditions that trigger intermittent extreme events in high-dimensional nonlinear dynamical systems. We seek the triggers as the probabilistically feasible solutions of an appropriately constrained optimization problem, where the function to be maximized is a system observable exhibiting intermittent extreme bursts. The constraints are imposed to ensure the physical admissibility of the optimal solutions, i.e., significant probability for their occurrence under the natural flow of the dynamical system. We apply the method to a body-forced incompressible Navier--Stokes equation, known as the Kolmogorov flow. We find that the intermittent bursts of the energy dissipation are independent of the external forcing and are instead caused by the spontaneous transfer of energy from large scales to the mean flow via nonlinear triad interactions. The global maximizer of the corresponding variational problem identifies the responsible triad, hence providing a precursor for the occurrence of extreme dissipation events. Specifically, monitoring the energy transfers within this triad, allows us to develop a data-driven short-term predictor for the intermittent bursts of energy dissipation. We assess the performance of this predictor through direct numerical simulations.

physics.flu-dyn↗

arXiv2017

Probabilistic response and rare events in Mathieu's equation under correlated parametric excitation

We derive an analytical approximation to the probability distribution function (pdf) for the response of Mathieu's equation under parametric excitation by a random process with a spectrum peaked at the main resonant frequency, motivated by the problem of large amplitude ship roll resonance in random seas. The inclusion of random stochastic excitation renders the otherwise straightforward response to a system undergoing intermittent resonances: randomly occurring large amplitude bursts. Intermittent resonance occurs precisely when the random parametric excitation pushes the system into the instability region, causing an extreme magnitude response. As a result, the statistics are characterized by heavy-tails. We apply a recently developed mathematical technique, the probabilistic decomposition-synthesis method, to derive an analytical approximation to the non-Gaussian pdf of the response. We illustrate the validity of this analytic approximation through comparisons with Monte-Carlo simulations that demonstrate our result accurately captures the strong non-Gaussianity of the response.

math.DS↗

arXiv2017

Extreme events and their optimal mitigation in nonlinear structural systems excited by stochastic loads: Application to ocean engineering systems

We develop an efficient numerical method for the probabilistic quantification of the response statistics of nonlinear multi-degree-of-freedom structural systems under extreme forcing events, emphasizing accurate heavy-tail statistics. The response is decomposed to a statistically stationary part and an intermittent component. The stationary part is quantified using a statistical linearization method while the intermittent part, associated with extreme transient responses, is quantified through i) either a few carefully selected simulations or ii) through the use of effective measures (effective stiffness and damping). The developed approach is able to accurately capture the extreme response statistics orders of magnitude faster compared with direct methods. The scheme is applied to the design and optimization of small attachments that can mitigate and suppress extreme forcing events delivered to a primary structural system. Specifically, we consider the problem of suppression of extreme responses in two prototype ocean engineering systems. First, we consider linear and cubic springs and perform parametric optimization by minimizing the forth-order moments of the response. We then consider a more generic, possibly asymmetric, piecewise linear spring and optimize its nonlinear characteristics. The resulting asymmetric spring design far outperforms the optimal cubic energy sink and the linear tuned mass dampers.

cs.CE↗

arXiv2017

Heavy-tailed response of structural systems subjected to stochastic excitation containing extreme forcing events

We characterize the complex, heavy-tailed probability distribution functions (pdf) describing the response and its local extrema for structural systems subjected to random forcing that includes extreme events. Our approach is based on the recent probabilistic decomposition-synthesis technique, where we decouple rare events regimes from the background fluctuations. The result of the analysis has the form of a semi-analytical approximation formula for the pdf of the response (displacement, velocity, and acceleration) and the pdf of the local extrema. For special limiting cases (lightly damped or heavily damped systems) our analysis provides fully analytical approximations. We also demonstrate how the method can be applied to high dimensional structural systems through a two-degrees-of-freedom structural system undergoing rare events due to intermittent forcing. The derived formulas can be evaluated with very small computational cost and are shown to accurately capture the complicated heavy-tailed and asymmetrical features in the probability distribution many standard deviations away from the mean, through comparisons with expensive Monte-Carlo simulations.

nlin.CD↗