SearcharxivSearch

arXiv subjects

Adrian Sandu

Publications and source records attributed to Adrian Sandu.

At least 19 recordsLinked to original sources

Benchmarking Machine Learning Uncertainty Quantification Methodologies for Predicting Turbine Gas Temperature Degradation

Effective prognostics and health management of modern engines relies on accurate turbine gas temperature predictions and robust uncertainty quantification to ensure reliability and safety. This paper investigates five major approaches for constructing prediction intervals -- namely the Delta method, Bayesian Monte Carlo Dropout, Bootstrap method, Lower-Upper Bound Estimation, and Mean-Variance Estimation -- as a means of capturing the uncertainty in neural network predictions of turbine gas temperature. Each approach is implemented within a unified experimental framework that employs cross-validation for hyperparameter selection, repeated train-test splits for performance robustness, and multiple metrics to evaluate both the accuracy and tightness of the intervals. In particular, Coverage Probability, Normalized Mean Prediction Interval Width, and the Coverage Width-based Criterion are measured to comprehensively assess each method's reliability and sharpness. Experiments conducted on a representative turbine gas temperature dataset reveal distinct trade-offs among the five methods in terms of interval coverage, width, and stability. These findings provide a practical guide for selecting and tuning prediction interval methods in engine health management and prognostics, ensuring both interpretability and precision in real-world applications.

cs.LG

Scientific Machine Learning for Engine Health Management and Remaining Useful Life Prediction

Engine Health Management (EHM) depends on reliable forecasting of Remaining Useful Life (RUL) and on tracking thermal indicators such as turbine gas temperature (TGT). In practice, real-world fleet data are heterogeneous and non-stationary, and point predictions alone are insufficient for risk-aware maintenance decisions. This paper presents a multi-task scientific machine learning framework for turbine prognostics that jointly predicts turbine gas temperature untrimmed (TGTU), Delta Turbine Gas Temperature (DTGT), and RUL, with quantified uncertainty in the form of prediction intervals whose empirical coverage is evaluated. A shared sequence encoder (convolutional front-end with residual bidirectional LSTM layers and attention pooling) feeds task-specific heads, including mean--variance estimation for probabilistic regression and, optionally, a survival head for threshold-based event modeling. The framework is designed to be tunable via a small set of practitioner-facing parameters (e.g., DTGT thresholding rules and RUL target construction) so that deployment can align with in-house policies and proprietary criteria. The predictive performance of the proposed framework is evaluated using both point and interval metrics, including mean absolute error (MAE), prediction interval coverage probability (PICP), mean prediction interval width (MPIW), and the coverage--width criterion (CWC). Results are reported both in aggregate and stratified by flight phase and maintenance segment to highlight operational-context effects and to support uncertainty-aware monitoring.

cs.LG

A Patankar predictor-corrector approach for positivity-preserving time integration

Many natural processes, such as chemical reactions and wave dynamics, are modeled as production-destruction (PD) systems that obey positivity and linear conservation laws. Classical time integrators do not guarantee positivity and can produce negative or nonphysical numerical solutions. This paper presents a modular correction strategy that can be applied to stiffly implicit Runge-Kutta schemes, in particular stiffly accurate SDIRK methods. The strategy combines stage-wise clipping with a ratio-based scaling that enforces invariants and is guaranteed to yield non-negative, conservative solutions. We provide a theoretical analysis of the corrected schemes and characterize their worst-case order of accuracy relative to the underlying base method. Numerical experiments on stiff ODE systems (Robertson, MAPK, stratospheric chemistry) and a nonlinear PDE (the Korteweg-De Vries equation) demonstrate that the corrected SDIRK methods preserve positivity and invariants without significant loss of accuracy. Importantly, corrections applied only to the final stage are sufficient in practice, while applying them at all stages may distort dynamics in some cases. These results show that the proposed framework provides a simple and effective way to construct positivity-preserving integrators for stiff PD systems.

math.NA

Ensemble based Closed-Loop Optimal Control using Physics-Informed Neural Networks

The objective of designing a control system is to steer a dynamical system with a control signal, guiding it to exhibit the desired behavior. The Hamilton-Jacobi-Bellman (HJB) partial differential equation offers a framework for optimal control system design. However, numerical solutions to this equation are computationally intensive, and analytical solutions are frequently unavailable. Knowledge-guided machine learning methodologies, such as physics-informed neural networks (PINNs), offer new alternative approaches that can alleviate the difficulties of solving the HJB equation numerically. This work presents a multistage ensemble framework to learn the optimal cost-to-go, and subsequently the corresponding optimal control signal, through the HJB equation. Prior PINN-based approaches rely on a stabilizing the HJB enforcement during training. Our framework does not use stabilizer terms and offers a means of controlling the nonlinear system, via either a singular learned control signal or an ensemble control signal policy. Success is demonstrated in closed-loop control, using both ensemble- and singular-control, of a steady-state time-invariant two-state continuous nonlinear system with an infinite time horizon, accounting of noisy, perturbed system states and varying initial conditions.

cs.LG

Multirate methods for ordinary differential equations

This survey provides an overview of state-of-the art multirate schemes, which exploit the different time scales in the dynamics of a differential equation model by adapting the computational costs to different activity levels of the system. We start the discussion with the straightforward approach based on interpolating and extrapolating the slow--fast coupling variables; the multirate Euler scheme, used as a base example, falls into this class. Next we discuss higher order multirate schemes that generalize classical singlerate linear multistep, Runge-Kutta, and extrapolation methods.

math.NA

A copula-based rank histogram ensemble filter

Serial ensemble filters implement triangular probability transport maps to reduce high-dimensional inference problems to sequences of state-by-state univariate inference problems. The univariate inference problems are solved by sampling posterior probability densities obtained by combining constructed prior densities with observational likelihoods according to Bayes' rule. Many serial filters in the literature focus on representing the marginal posterior densities of each state. However, rigorously capturing the conditional dependencies between the different univariate inferences is crucial to correctly sampling multidimensional posteriors. This work proposes a new serial ensemble filter, called the copula rank histogram filter (CoRHF), that seeks to capture the conditional dependency structure between variables via empirical copula estimates; these estimates are used to rigorously implement the triangular (state-by-state univariate) Bayesian inference. The success of the CoRHF is demonstrated on two-dimensional examples and the Lorenz '63 problem. A practical extension to the high-dimensional setting is developed by localizing the empirical copula estimation, and is demonstrated on the Lorenz '96 problem.

stat.ME

Feature preserving data assimilation via feature alignment

Data assimilation combines information from physical observations and numerical simulation results to obtain better estimates of the state and parameters of a physical system. A wide class of physical systems of interest have solutions that exhibit the formation of structures, called features, which have to be accurately captured by the assimilation framework. For example, fluids can develop features such as shockwaves and contact discontinuities that need to be tracked and preserved during data assimilation. State-of-the-art data assimilation techniques are agnostic of such features. Current ensemble-based methods construct state estimates by taking linear combinations of multiple ensemble states; repeated averaging tends to smear the features over multiple assimilation cycles, leading to nonphysical state estimates. A novel feature-preserving data assimilation methodology that combines sequence alignment with the ensemble transform particle filter is proposed to overcome this limitation of existing assimilation algorithms. Specifically, optimal transport of particles is performed along feature-aligned characteristics. The strength of the proposed feature-preserving filtering approach is demonstrated on multiple test problems described by the compressible Euler equations.

math.OC

Preserving Nonlinear Constraints in Variational Flow Filtering Data Assimilation

Data assimilation aims to estimate the states of a dynamical system by optimally combining sparse and noisy observations of the physical system with uncertain forecasts produced by a computational model. The states of many dynamical systems of interest obey nonlinear physical constraints, and the corresponding dynamics is confined to a certain sub-manifold of the state space. Standard data assimilation techniques applied to such systems yield posterior states lying outside the manifold, violating the physical constraints. This work focuses on particle flow filters which use stochastic differential equations to evolve state samples from a prior distribution to samples from an observation-informed posterior distribution. The variational Fokker-Planck (VFP) -- a generic particle flow filtering framework -- is extended to incorporate non-linear, equality state constraints in the analysis. To this end, two algorithmic approaches that modify the VFP stochastic differential equation are discussed: (i) VFPSTAB, to inexactly preserve constraints with the addition of a stabilizing drift term, and (ii) VFPDAE, to exactly preserve constraints by treating the VFP dynamics as a stochastic differential-algebraic equation (SDAE). Additionally, an implicit-explicit time integrator is developed to evolve the VFPDAE dynamics. The strength of the proposed approach for constraint preservation in data assimilation is demonstrated on three test problems: the double pendulum, Korteweg-de-Vries, and the incompressible Navier-Stokes equations.

math.OC

Improving the Adaptive Moment Estimation (ADAM) stochastic optimizer through an Implicit-Explicit (IMEX) time-stepping approach

The Adam optimizer, often used in Machine Learning for neural network training, corresponds to an underlying ordinary differential equation (ODE) in the limit of very small learning rates. This work shows that the classical Adam algorithm is a first-order implicit-explicit (IMEX) Euler discretization of the underlying ODE. Employing the time discretization point of view, we propose new extensions of the Adam scheme obtained by using higher-order IMEX methods to solve the ODE. Based on this approach, we derive a new optimization algorithm for neural network training that performs better than classical Adam on several regression and classification problems.

cs.CE

Simultaneous Optimal System and Controller Design for Multibody Systems with Joint Friction using Direct Sensitivities

Real-world multibody systems are often subject to phenomena like friction, joint clearances, and external events. These phenomena can significantly impact the optimal design of the system and its controller. This work addresses the gradient-based optimization methodology for multibody dynamic systems with joint friction using a direct sensitivity approach for gradient computation. After a thorough review of various friction models developed over the years, the Brown McPhee model has been found to be the most suitable for the study due to its accuracy for dynamic simulation and its compatibility with sensitivity analysis. The methodology supports co-design of the system and its controller, which is especially relevant for applications like robotics and servo-mechanical systems where the actuation and the design are highly dependent on each other. Numerical results are obtained using a new implementation of the MBSVT (Multi-Body Systems at Virginia Tech) software package; MBSVT 2.0 is reprogrammed in Julia for ease of implementation while maintaining high computational efficiency. Three case studies are provided to demonstrate the attractive properties of simultaneous optimal design and control approach for certain applications.

eess.SY

Adversarial Training Using Feedback Loops

Deep neural networks (DNN) have found wide applicability in numerous fields due to their ability to accurately learn very complex input-output relations. Despite their accuracy and extensive use, DNNs are highly susceptible to adversarial attacks due to limited generalizability. For future progress in the field, it is essential to build DNNs that are robust to any kind of perturbations to the data points. In the past, many techniques have been proposed to robustify DNNs using first-order derivative information of the network. This paper proposes a new robustification approach based on control theory. A neural network architecture that incorporates feedback control, named Feedback Neural Networks, is proposed. The controller is itself a neural network, which is trained using regular and adversarial data such as to stabilize the system outputs. The novel adversarial training approach based on the feedback control architecture is called Feedback Looped Adversarial Training (FLAT). Numerical results on standard test problems empirically show that our FLAT method is more effective than the state-of-the-art to guard against adversarial attacks.

cs.LG

Symplectic multirate generalized additive Runge-Kutta methods for Hamiltonian systems

The generalized additive Runge-Kutta (GARK) framework provides a powerful approach for solving additively partitioned ordinary differential equations. This work combines the ideas of symplectic GARK schemes and multirate GARK schemes to efficiently solve additively partitioned Hamiltonian systems with multiple time scales. Order conditions, as well as conditions for symplecticity and time-reversibility, are derived in the general setting of non-separable Hamiltonian systems. Investigations of the special case of separable Hamiltonian systems are also carried out. We show that particular partitions may introduce stability issues, and discuss partitions that enable an implicit-explicit integration leading to improved stability properties. Higher-order symplectic multirate GARK schemes based on advanced composition techniques are discussed. The performance of the schemes is demonstrated by means of the Fermi-Pasta-Ulam problem.

math.NA

Neural Network Reduction with Guided Regularizers

Regularization techniques such as $\mathcal{L}_1$ and $\mathcal{L}_2$ regularizers are effective in sparsifying neural networks (NNs). However, to remove a certain neuron or channel in NNs, all weight elements related to that neuron or channel need to be prunable, which is not guaranteed by traditional regularization. This paper proposes a simple new approach named "Guided Regularization" that prioritizes the weights of certain NN units more than others during training, which renders some of the units less important and thus, prunable. This is different from the scattered sparsification of $\mathcal{L}_1$ and $\mathcal{L}_2$ regularizers where the the components of a weight matrix that are zeroed out can be located anywhere. The proposed approach offers a natural reduction of NN in the sense that a model is being trained while also neutralizing unnecessary units. We empirically demonstrate that our proposed method is effective in pruning NNs while maintaining performance.

cs.LG

Adjoint-Matching Neural Network Surrogates for Fast 4D-Var Data Assimilation

Data assimilation is the process of fusing information from imperfect computer simulations with noisy, sparse measurements of reality to obtain improved estimates of the state or parameters of a dynamical system of interest. The data assimilation procedures used in many geoscience applications, such as numerical weather forecasting, are variants of the our-dimensional variational (4D-Var) algorithm. The cost of solving the underlying 4D-Var optimization problem is dominated by the cost of repeated forward and adjoint model runs. This motivates substituting the evaluations of the physical model and its adjoint by fast, approximate surrogate models. Neural networks offer a promising approach for the data-driven creation of surrogate models. The accuracy of the surrogate 4D-Var solution depends on the accuracy with each the surrogate captures both the forward and the adjoint model dynamics. We formulate and analyze several approaches to incorporate adjoint information into the construction of neural network surrogates. The resulting networks are tested on unseen data and in a sequential data assimilation problem using the Lorenz-63 system. Surrogates constructed using adjoint information demonstrate superior performance on the 4D-Var data assimilation problem compared to a standard neural network surrogate that uses only forward dynamics information.

cs.LG

A Two-Level Galerkin Reduced Order Model for the Steady Navier-Stokes Equations

We propose, analyze, and investigate numerically a novel two-level Galerkin reduced order model (2L-ROM) for the efficient and accurate numerical simulation of the steady Navier-Stokes equations. In the first step of the 2L-ROM, a relatively low-dimensional nonlinear system is solved. In the second step, the Navier-Stokes equations are linearized around the solution found in the first step, and a higher-dimensional system for the linearized problem is solved. We prove an error bound for the new 2L-ROM and compare it to the standard one level ROM (1L-ROM) in the numerical simulation of the steady Burgers equation. The 2L-ROM significantly decreases (by a factor of $2$ and even $3$) the 1L-ROM computational cost, without compromising its numerical accuracy.

math.NA

The Model Forest Ensemble Kalman Filter

Traditional data assimilation uses information obtained from the propagation of one physics-driven model and combines it with information derived from real-world observations in order to obtain a better estimate of the truth of some natural process. However, in many situations multiple simulation models that describe the same physical phenomenon are available. Such models can have different sources. On one hand there are theory-guided models are constructed from first physical principles, while on the other there are data-driven models that are constructed from snapshots of high fidelity information. In this work we provide a possible way to make use of this collection of models in data assimilation by generalizing the idea of model hierarchies into model forests -- collections of high fidelity and low fidelity models organized in a groping of model trees such as to capture various relationships between different models. We generalize the multifidelity ensemble Kalman filter that previously operated on model hierarchies into the model forest ensemble Kalman filter through a generalized theory of linear control variates. This new filter allows for much more freedom when treading the line between accuracy and speed. Numerical experiments with a high fidelity quasi-geostrophic model and two of its low fidelity reduced order models validate the accuracy of our approach.

cs.CE

Physics-informed neural networks for PDE-constrained optimization and control

A fundamental problem in science and engineering is designing optimal control policies that steer a given system towards a desired outcome. This work proposes Control Physics-Informed Neural Networks (Control PINNs) that simultaneously solve for a given system state, and for the optimal control signal, in a one-stage framework that conforms to the underlying physical laws. Prior approaches use a two-stage framework that first models and then controls a system in sequential order. In contrast, a Control PINN incorporates the required optimality conditions in its architecture and in its loss function. The success of Control PINNs is demonstrated by solving the following open-loop optimal control problems: (i) an analytical problem, (ii) a one-dimensional heat equation, and (iii) a two-dimensional predator-prey problem.

cs.LG

A Meta-learning Formulation of the Autoencoder Problem for Non-linear Dimensionality Reduction

A rapidly growing area of research is the use of machine learning approaches such as autoencoders for dimensionality reduction of data and models in scientific applications. We show that the canonical formulation of autoencoders suffers from several deficiencies that can hinder their performance. Using a meta-learning approach, we reformulate the autoencoder problem as a bi-level optimization procedure that explicitly solves the dimensionality reduction task. We prove that the new formulation corrects the identified deficiencies with canonical autoencoders, provide a practical way to solve it, and showcase the strength of this formulation with a simple numerical illustration.

cs.LG