Searcharxiv⌕ Search

arXiv subjects

Karthik Duraisamy

Publications and source records attributed to Karthik Duraisamy.

At least 19 recordsLinked to original sources

Dynamic Quadtree Tokenization and Transformer for Adaptive Mesh PDE Forecasting

The quadratic attention cost of Vision Transformers (ViTs) forces a trade-off between spatial resolution and rollout horizon, particularly for fine-scale PDEs where shocks, reaction fronts, and material interfaces occupy small, evolving regions of the domain. Conventional neural surrogates also lack mechanisms to adapt resolution dynamically. We propose WAMRViT, a ViT that tokenizes inputs as balanced quadtrees using a wavelet-inspired refinement criterion, jointly encodes position and refinement level with 3D rotary positional embeddings, and regrids in cell space during inference for stable long-horizon rollouts. A multi-scale variant retains each leaf at its native source resolution and lets the model learn across resolution levels. Unlike fixed-budget adaptive-tokenization methods, WAMRViT imposes no predetermined token count and supports fully adaptive topology throughout autoregressive rollout. To our knowledge, it is the first machine-learning surrogate to natively tokenize multi-level Adaptive Mesh Refinement (AMR) data. On uniform-grid benchmarks, uniform-patch WAMRViT improves finest-level region-of-interest VRMSE over a finest-patch uniform ViT while using substantially fewer tokens. The multi-scale variant achieves the lowest first-step full-field RMSE and VRMSE on both benchmarks and improves rollout-averaged full-field and refined-region accuracy at long horizons. With parallelized regridding, its end-to-end rollout cost lies between finest-patch and approximately token-matched coarser-patch ViTs. On a complex AMR combustion problem whose finest features cannot be represented natively by the evaluated uniform-grid baselines, WAMRViT operates directly on adaptive cells and substantially reduces finest-level error at matched transformer capacity. Code: https://github.com/tonyzyl/wamrvit

cs.CV↗

In-context learning from self-generated trajectories for adaptive model reduction

High-fidelity simulations of complex physical systems are often too expensive for repeated prediction, design, and control. Reduced-order models address this computational cost by compressing high-dimensional dynamics into low-dimensional representations that can be evaluated rapidly, but they often lose accuracy when online dynamics drift beyond the offline training data. Adaptive methods address this limitation by updating the reduced subspace online using external, out-of-span information, such as full-order corrections or sensor snapshots. We discovered that a complementary and previously unexploited in-span adaptation channel exists within the current reduced subspace. To exploit this channel, we continually update the reduced representation using the model's own predictions through an incremental singular-value decomposition with a forgetting factor. This produces a trajectory-informed spectral preconditioner in which the reduced subspace remains unchanged, while the basis is reweighted and realigned according to the directions visited by the evolving dynamics. This internal reorganization prepares the basis to absorb future out-of-span corrections more effectively, improving adaptation without requiring additional external information. We expose the mechanism through a three-dimensional spiral example and demonstrate its benefits on nonlinear partial differential equations, including viscous Burgers and Fisher--KPP dynamics. We also discuss how in-span learning can be interpreted as a dynamical-systems analogue of in-context learning. More broadly, in-span learning suggests a new principle for computational science, revealing that model-generated trajectories contain more usable information than previously recognized.

cs.LG↗

The Minimal Attached Eddy in Wall Turbulence: Statistical Foundations, Inverse Identification and Influence Kernels

Townsend's attached eddy hypothesis models the logarithmic region of high Reynolds number wall turbulence as a random superposition of wall-attached, self-similar eddies whose sizes obey a scale-invariant population law. Building on the statistical framework of Woodcock & Marusic (2015), the present work (i) poses an inverse problem to infer the ideal single-eddy contribution (influence) functions for the mean velocity and Reynolds stresses from DNS moments, (ii) uses these inferred kernels to guide a minimal Biot--Savart-consistent hairpin-type eddy built from Rankine vortex rods with an inviscid image system, and (iii) introduces and infers a spectral Influence kernel that maps a self-similar eddy footprint to its one-dimensional energy spectrum. The Influence-kernel viewpoint yields an explanation for the emergence of the linear part of the energy spectrum, provides a clear scale-by-scale decomposition and helps rationalize why simple eddy templates can reproduce a broad set of log-layer statistics once the mean-flow anchoring is fixed. Closed-form expressions for the mean influence function and the Fourier-space streamwise velocity of a general straight-segment hairpin family with image are derived, revealing a mean-variance duality: For a rectangular hairpin, the horizontal head determines the entire mean kernel while the inclined legs dominate the spectral energy. It is shown that the rectangular hairpin yields a precise plateau in the influence function, resulting in a logarithmic mean velocity profile and desirable properties for the streamwise fluctuations. The model is further extended by calibrating the eddy population density as a function of scale, yielding accurate descriptions of mean velocity and streamwise variance across $Re_τ=6000-20000$.

physics.flu-dyn↗

Fully Discrete Active Flux Method based on Transported Acoustic Increments for the Compressible Euler Equations

A fully discrete Active Flux method is proposed for the two-dimensional compressible Euler equations. The method builds on the evolution-operator formulation of multidimensional Active Flux proposed by Roe in which conservative cell averages are updated by unsplit flux quadrature while primitive point values are evolved by acoustic and advective subsolvers. This has recently been extended to the Euler equations by Barsukow, who developed an efficient approach by adding acoustic increments at the Eulerian target point. The proposed method reconstructs the acoustic increment as a cellwise Q2 field, using vertex, edg--midpoint, and cell-center acoustic increments, and evaluates this field at the convective foot of the target point. For constant frozen coefficients, the resulting point update reduces to the transported composition, eliminating the additive split defect and yielding the exact unsplit frozen evolution when the acoustic and advective generators commute. The resulting method preserves the exact locally linearized acoustic evolution operator of Barsukow, the compact stencil, and the conservative one-stage average update. Numerical experiments probe several facets of the numerical method. A mixed Fourier wave packet with the linearized Euler equations isolates the split error and shows third-order point accuracy for the transported update. Isentropic vortex convection confirms reduced error constants, and an enlarged empirical CFL range. Nonlinear Gaussian acoustic pulse evolution demonstrates preservation of radial symmetry and near-third-order decay of the symmetry error. Low-Mach shear-layer tests show coherent vorticity evolution, low entropy dissipation, and absence of the coarse-grid secondary vortices seen in displayed DG/CG comparisons. A compressible under-resolved Kelvin-Helmholtz test demonstrates robust evolution with consistent entropy dissipation.

math.NA↗

Residual-augmented flow matching operators for probabilistic partial differential equations

Learning surrogate models for physical systems with latent uncertainty remains challenging in data-scarce regimes: deterministic neural operators fail to characterize uncertainty, while generative approaches require large ensembles of high-fidelity solution operator simulations and often sacrifice resolution generalizability. In this work, we propose a residual-augmented probabilistic operator learning framework that casts flow-matching-based generative modeling in infinite-dimensional function spaces while leveraging inexpensive low-fidelity solution operators as an inductive bias. Rather than learning the full high-fidelity stochastic solution operator directly, the proposed framework learns probabilistic residual operators that characterize the discrepancy between low- and high-fidelity solutions. By parameterizing the vector field in flow matching using neural operators conditioned on both the known system input and low-fidelity solution, the framework amortizes probabilistic inference across input conditions while enabling uncertainty-aware and resolution-generalizable predictions across spatial discretizations. Numerical experiments on stochastic advection, Burgers', and Darcy flow systems demonstrate that the residual-augmented formulation improves predictive accuracy under the same high-fidelity data budget, while the probabilistic operator learning formulation enables accurate characterization of uncertainty in low-data regimes compared to learning high-fidelity stochastic operators directly from data.

stat.CO↗

Evolutionary Feature Engineering for Structured Data

Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using LLM-based evolution to discover preprocessing transformations for structured data. EFE represents transformations as Python programs with a standardized fit/transform interface, allowing them to be inserted directly into existing machine learning pipelines. During evolution, candidate programs are refined using dataset context, summary statistics, and downstream performance feedback on validation set. We instantiate EFE in two settings. For time-series forecasting, EFE-Time learns invertible, dataset-specific normalizations that improve off-the-shelf time-series foundation models. It reduces forecasting errors (MASE, WQL, MAE) 3% or more when averaged across datasets and improvements are as much as 19% on the COVID-Deaths dataset. Notably, these improvements occur with recent TSFMs such as Chronos-2. For tabular prediction, EFE-Tab evolves compact feature programs that add useful interpretable features and remove redundant ones, improving or matching existing LLM-based feature-engineering methods. We found EFE-Tab to be particularly effective on classical decision trees, where small sets of evolved features yield competitive accuracy while preserving interpretability. Overall, EFE demonstrates that LLM-based evolution can improve both accuracy and interpretability when automatically tackling structured data.

cs.LG↗

History-aware adaptive reduced-order models via incremental singular value decomposition

Reduced-order models (ROMs) can accelerate high-dimensional dynamical simulations, but their accuracy often deteriorates when online dynamics leave the regime represented by offline training data. We develop a projection-based adaptive ROM framework based on incremental singular value decomposition (iSVD), in which occasional full-order operator evaluations provide correction snapshots for online basis updates. The intrusive ROMs considered here are fully parameterized by the basis, so each update naturally propagates to reduced operators and hyper-reduction machinery. Through its evolving singular structure, iSVD retains an encoded history of the observed dynamics and is history-aware in this sense. We study the method on three nonlinear problems of increasing complexity: the one-dimensional viscous Burgers equation, the Sod shock tube, and a stiff one-dimensional ten-species rotating detonation engine (RDE). The Burgers problem is used to analyze the method and compare iSVD with alternative basis adaptation rules, showing that history-aware updates outperform instantaneous updates and that iSVD gives the strongest overall performance. The Sod and RDE cases demonstrate that these advantages persist in more challenging compressible-flow settings. For the RDE problem, the iSVD adaptive ROM improves upon the current state-of-the-art Direct adaptive ROM baseline in both predictive accuracy and computational efficiency. A cost analysis shows that the dominant online cost comes from interacting with the full-order model to obtain correction snapshots, while the iSVD update itself is negligible. These results identify iSVD as an effective mechanism for online learning of reduced subspaces and suggest a path toward ROMs that remain predictive over horizons several orders of magnitude longer than their initial training window.

cs.LG↗

Asymptotic-preserving semi-implicit finite volume scheme for Extended Magnetohydrodynamics

A Finite Volume (FV) scheme is developed for solving the extended magnetohydrodynamic (XMHD) equations, yielding accurate results in the ideal, resistive, and Hall MHD limits. This is accomplished by first re-writing the XMHD equations such that it allows the algorithm to retain the use of ideal MHD Riemann solvers and the constrained transport method to preserve divergence-free magnetic fields. Incorporation of electron inertia and displacement current introduces additional numerical stiffness which motivates a semi-implicit FV scheme that re-formulates the XMHD model as a relaxation system. The equations are then advanced in time using an explicit 2nd-order Runge-Kutta scheme with operator splitting applied to the implicit source term updates at each sub-stage. For additional numerical stability, a density-dependent slope limiter is implemented to increase flux diffusivity at low density regions where non-ideal effects become significant. The algorithm is subsequently implemented in a scalable adaptive mesh refinement (AMR) framework. As the new algorithm retains many aspects of the ideal MHD formulations, it asymptotes naturally to the ideal MHD limit. Moreover, it shows promising results at the resistive and Hall MHD limits. This is verified against reference test problems for ideal, resistive and Hall MHD.

physics.plasm-ph↗

Foundation Models for Discovery and Exploration in Chemical Space

Accurate prediction of atomistic, thermodynamic, and kinetic properties from molecular structures underpins materials innovation. Existing computational and experimental approaches lack the scalability required to navigate chemical space efficiently. Scientific foundation models trained on large unlabelled datasets offer a path towards navigating chemical space across application domains. Here, we develop MIST, a family of molecular foundation models with up to an order of magnitude more parameters and data than prior works. Trained using a novel tokenizer, Smirk, which comprehensively captures nuclear, electronic, and geometric information, MIST learns a diverse range of molecules. MIST models have been fine-tuned to predict more than 400 structure-property relationships and have been shown to match or exceed state-of-the-art performance across diverse benchmarks, from physiology to electrochemistry. We demonstrate the ability of these models to solve real-world problems across chemical space from multiobjective electrolyte solvent screening to stereochemical reasoning for organometallics and mixture property prediction. The clearest demonstration of a foundation model is its ability to solve problems that were neither explicit targets of training nor central to the intentions of its developers. We identify olfactory perception mapping as such a problem, and show that MIST accurately predicted scent profiles and learned a hierarchical representation of olfactory space consistent with hyperbolic geometry. We formulated hyperparameter aware Bayesian neural scaling laws which eliminate the need for hyperparameter sweeps at every scale, making training large compute-optimal models feasible on a limited compute budget. The methods and findings presented here represent a significant step towards accelerating materials discovery, design, and optimization using foundation models.

physics.chem-ph↗

Gauss Principle in Incompressible Flow: Unified Variational Perspective on Pressure and Projection

Following recent work, this manuscript clarifies what the Gauss-Appell principle determines in incompressible, inviscid flow and how it connects to classical projection methods. At a fixed time, freezing the velocity and varying only the material acceleration leads to minimization of a quadratic subject to acceleration-level constraints. First-order conditions yield a Poisson-Neumann problem for a reaction pressure whose gradient removes the non-solenoidal and wall-normal content of the provisional residual, precisely the well-known Leray-Hodge projection. Thus, Gauss-Appell enforces the instantaneous kinematic constraints and recovers Euler at the instant. Once the impressed physics is specified, for instance via external body forces, the reaction pressure is uniquely determined (up to an additive constant) as the Lagrange multiplier enforcing incompressibility and wall impermeability; it does no work on divergence-free, wall-tangent motions. This is the well-established interpretation of pressure in incompressible flow. The direct, fixed-time application of this principle determines the reaction pressure for an already-specified velocity field and does not, by itself, select circulation or stagnation points, because these are properties of the velocity state, not the instantaneous acceleration correction. The formal decomposition of the pressure into impressed and reaction components admits representational freedom that does not imply physical non-uniqueness of the constraint force. Orthogonality conventions such as Dirichlet orthogonality can fix the representational freedom as an additional modeling choice. This variational viewpoint also yields a simple computational diagnostic: the minimized Appellian equals a L2 norm of the reaction-pressure gradient which vanishes for constraint-compatible updates and grows with the magnitude of divergence and wall-flux mismatch.

physics.flu-dyn↗

Predictivity and Utility of Neural Surrogates of Multiscale PDEs

Scientific machine learning is increasingly being spoken of as universal emulators for classical numerical solvers for multi-scale partial differential equations, but most apparent successes can be explained by facts that also define their limits. Many successful benchmarks live on low-dimensional solution manifolds where any competent reduced model will interpolate well. More fundamentally, neural surrogates systematically under-resolve high-frequency content due to spectral bias, and coarse-graining compounds this problem through irreversible information loss. In many multi-scale problems, no architecture or training procedure can fully recover what the coarse representation discards. Two simple examples are used to characterize spectral bias, coarse-graining and error accumulation. We discuss why medium-range weather prediction on reanalysis data sits in a favorable sweet spot and why this will not generalize to genuinely chaotic multi-scale scenarios. We identify domains where neural surrogates offer genuine value, propose further research on neural-classical hybrids, and call for better reporting standards.

math-ph↗

Bias- and Variance-Aware Probabilistic Rounding Error Analysis for Floating-Point Arithmetic

Probabilistic rounding error analysis can yield much sharper bounds than classical worst-case theory, but existing results typically rely on zero-mean rounding errors and often leave the confidence parameter implicit. This work revisits probabilistic rounding error analysis in a moment-aware setting. We first derive a confidence-calibrated reformulation of the Higham and Mary [16] bound that makes its confidence parameter explicit. We then introduce a variance-informed probabilistic backward error bound based on the first two moments of $\log(1+δ)$, where $δ$ is the relative rounding error. This allows the analysis to accommodate biased rounding error models rather than relying on a zero-mean assumption. To illustrate this framework, we study both a uniform model and a log-space $\operatorname{Beta}$ model for rounding errors, the latter of which provides a simple way to represent bias. This perspective shows that the growth of probabilistic rounding error bounds is not universal: near-zero-mean regimes recover $\sqrt{n}$-like behavior, while biased models can exhibit faster accumulation. $\texttt{CUDA}$ experiments in single and half precision on dot products, sparse matrix-vector products, and a stochastic boundary-value problem show that the proposed framework is especially useful in low-precision regimes where deterministic bounds are overly conservative and where bias-aware modeling better matches observed error growth.

stat.CO↗

Toward Adaptive Non-Intrusive Reduced-Order Models: Design and Challenges

Projection-based Reduced Order Models (ROMs) are often deployed as static surrogates, which limits their practical utility once a system leaves the training manifold. We formalize and study adaptive non-intrusive ROMs that update both the latent subspace and the reduced dynamics online. Building on ideas from static non-intrusive ROMs, specifically, Operator Inference (OpInf) and the recently-introduced Non-intrusive Trajectory-based optimization of Reduced-Order Models (NiTROM), we propose three formulations: Adaptive OpInf (sequential basis/operator refits), Adaptive NiTROM (joint Riemannian optimization of encoder/decoder and polynomial dynamics), and a hybrid that initializes NiTROM with an OpInf update. We describe the online data window, adaptation window, and computational budget, and analyze cost scaling. On a transiently perturbed lid-driven cavity flow, static Galerkin/OpInf/NiTROM drift or destabilize when forecasting beyond training. In contrast, Adaptive OpInf robustly suppresses amplitude drift with modest cost; Adaptive NiTROM is shown to attain near-exact energy tracking under frequent updates but is sensitive to its initialization and optimization depth; the hybrid is most reliable under regime changes and minimal offline data, yielding physically coherent fields and bounded energy. We argue that predictive claims for ROMs must be cost-aware and transparent, with clear separation of training/adaptation/deployment regimes and explicit reporting of online budgets and full-order model queries. This work provides a practical template for building self-correcting, non-intrusive ROMs that remain effective as the dynamics evolve well beyond the initial manifold.

cs.LG↗

Active Inference AI Systems for Scientific Discovery

The rapid evolution of artificial intelligence has led to expectations of transformative impact on science, yet current systems remain fundamentally limited in enabling genuine scientific discovery. This perspective contends that progress turns on closing three mutually reinforcing gaps in abstraction, reasoning and empirical grounding. Central to addressing these gaps is recognizing complementary cognitive modes: thinking as slow, iterative hypothesis generation -- exploring counterfactual spaces where physical laws can be temporarily violated to discover new patterns -- and reasoning as fast, deterministic validation, traversing established knowledge graphs to test consistency with known principles. Abstractions in this loop should be manipulable models that enable counterfactual prediction, causal attribution, and refinement. Design principles -- rather than a monolithic recipe -- are proposed for systems that reason in imaginary spaces and learn from the world: causal, multimodal models for internal simulation; persistent, uncertainty-aware scientific memory that distinguishes hypotheses from established claims; formal verification pathways coupled to computations and experiments. It is also argued that the inherent ambiguity in feedback from simulations and experiments, and underlying uncertainties make human judgment indispensable, not as a temporary scaffold but as a permanent architectural component. Evaluations must assess the system's ability to identify novel phenomena, propose falsifiable hypotheses, and efficiently guide experimental programs toward genuine discoveries.

cs.AI↗

Attention-Enhanced Convolutional Autoencoder and Structured Delay Embeddings for Weather Prediction

Weather prediction is a quintessential problem involving the forecasting of a complex, nonlinear, and chaotic high-dimensional dynamical system. This work introduces an efficient reduced-order modeling (ROM) framework for short-range weather prediction and investigates fundamental questions in dimensionality reduction and reduced order modeling of such systems. Unlike recent AI-driven models, which require extensive computational resources, our framework prioritizes efficiency while achieving reasonable accuracy. Specifically, a ResNet-based convolutional autoencoder augmented by block attention modules is developed to reduce the dimensionality of high-dimensional weather data. Subsequently, a linear operator is learned in the time-delayed embedding of the latent space to efficiently capture the dynamics. Using the ERA5 reanalysis dataset, we demonstrate that this framework performs well in-distribution as evidenced by effectively predicting weather patterns within training data periods. We also identify important limitations in generalizing to future states, particularly in maintaining prediction accuracy beyond the training window. Our analysis reveals that weather systems exhibit strong temporal correlations that can be effectively captured through linear operations in an appropriately constructed embedding space, and that projection error rather than inference error is the main bottleneck. These findings shed light on some key challenges in reduced-order modeling of chaotic systems and point toward opportunities for hybrid approaches that combine efficient reduced-order models as baselines with more sophisticated AI architectures, particularly for applications in long-term climate modeling where computational efficiency is paramount.

cs.LG↗

Two-point Turbulence Closures in Physical Space

This work presents a predictive two-point statistical closure framework for turbulence formulated in physical space. A closure model for ensemble-averaged, incompressible homogeneous isotropic turbulence (HIT) is developed as a starting point to demonstrate the viability of the approach in more general flows. The evolution equation for the longitudinal correlation function is derived in a discrete form, circumventing the need for a Fourier transformation. The formulation preserves the near-exact representation of the linear terms, a defining feature of rapid distortion theory. The closure of the nonlinear higher-order moments follows the phenomenological principles of the Eddy-Damped Quasi-Normal Markovian (EDQNM) model of Orszag (1970). Several key differences emerge from the physical-space treatment, including the need to evaluate a matrix exponential in the evolution equation and the appearance of triple integrals arising from the non-local nature of the pressure-Poisson equation. This framework naturally incorporates non-local length-scale information into the evolution of turbulence statistics. Verification of the physical-space two-point closure is performed by comparison with direct numerical simulations of statistically stationary forced HIT and with classical EDQNM predictions for decaying HIT. Finally, extensions to inhomogeneous and anisotropic turbulence are discussed, emphasizing advantages in applications where spectral methods are ill-conditioned, such as compressible flows with discontinuities.

physics.flu-dyn↗

LaDCast: A Latent Diffusion Model for Medium-Range Ensemble Weather Forecasting

Accurate probabilistic weather forecasting demands both high accuracy and efficient uncertainty quantification, challenges that overburden both ensemble numerical weather prediction (NWP) and recent machine-learning methods. We introduce LaDCast, the first global latent-diffusion framework for medium-range ensemble forecasting, which generates hourly ensemble forecasts entirely in a learned latent space. An autoencoder compresses high-dimensional ERA5 reanalysis fields into a compact representation, and a transformer-based diffusion model produces sequential latent updates with arbitrary hour initialization. The model incorporates Geometric Rotary Position Embedding (GeoRoPE) to account for the Earth's spherical geometry, a dual-stream attention mechanism for efficient conditioning, and sinusoidal temporal embeddings to capture seasonal patterns. LaDCast achieves deterministic and probabilistic skill close to that of the European Centre for Medium-Range Forecast IFS-ENS, without any explicit perturbations. Notably, LaDCast demonstrates superior performance in tracking rare extreme events such as cyclones, capturing their trajectories more accurately than established models. By operating in latent space, LaDCast reduces storage and compute by orders of magnitude, demonstrating a practical path toward forecasting at kilometer-scale resolution in real time. We open-source our code and models and provide the training and evaluation pipelines at: https://github.com/tonyzyl/ladcast.

cs.LG↗

A Digital Twin Framework for Generation-IV Reactors with Reinforcement Learning-Enabled Health-Aware Supervisory Control

Generation IV (Gen-IV) nuclear power plants are envisioned to replace the current reactor fleet, bringing improvements in performance, safety, reliability, and sustainability. However, large cost investments currently inhibit the deployment of these advanced reactor concepts. Digital twins bridge real-world systems with digital tools to reduce costs, enhance decision-making, and boost operational efficiency. In this work, a digital twin framework is designed to operate the Gen-IV Fluoride-salt-cooled High-temperature Reactor, utilizing data-enhanced methods to optimize operational and maintenance policies while adhering to system constraints. The closed-loop framework integrates surrogate modeling, reinforcement learning, and Bayesian inference to streamline end-to-end communication for online regulation and self-adjustment. Reinforcement learning is used to consider component health and degradation to drive the target power generations, with constraints enforced through a Reference Governor control algorithm that ensures compliance with pump flow rate and temperature limits. These input driving modules benefit from detailed online simulations that are assimilated to measurement data with Bayesian filtering. The digital twin is demonstrated in three case studies: a one-year long-term operational period showcasing maintenance planning capabilities, short-term accuracy refinement with high-frequency measurements, and system shock capturing that demonstrates real-time recalibration capabilities when change in boundary conditions. These demonstrations validate robustness for health-aware and constraint-informed nuclear plant operation, with general applicability to other advanced reactor concepts and complex engineering systems.

eess.SY↗