arXiv subjects

Seid Koric

Publications and source records attributed to Seid Koric.

At least 19 recordsLinked to original sources

arXiv2026

Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

Deep neural operators learn mappings between input functions and complete PDE solution fields, enabling forward evaluations of new problem instances orders of magnitude faster than conventional numerical solvers. Attention mechanisms have recently been introduced into neural operators, but most studies change several architectural components at once, making it difficult to identify what actually improves accuracy. This work presents a controlled and systematic study of five deep operator network (DeepONet) variants with distinct attention mechanisms, trained under both data-driven and physics-informed regimes, to isolate the effects of cross-attention, self-attention, tokenization, and attention depth. We evaluate them on a source-driven transient one-dimensional nonlinear diffusion-reaction equation, a transient one-dimensional viscous Burgers equation with variable initial conditions, and a two-dimensional Poisson heat-conduction problem with heterogeneous source fields. Per-sensor tokenization with cross-attention reduces the mean relative L_2 error of the classical DeepONet in all benchmark-training combinations by factors of 2.4-28.0, while the best attention configurations reach 3.5-32.3. Branch self-attention paired only with dot-product fusion is inconsistent, degrading the one-dimensional problems while helping the more complex two-dimensional source field; added on top of cross-attention it improves all six cases, though by less than cross-attention fusion alone. Global pre-mixing provides no consistent benefit. Increasing cross-attention depth further improves accuracy, but with diminishing returns and a substantially higher cost under physics-informed training. Overall, query-dependent cross-attention is the most reliable mechanism, whereas branch self-attention is most useful for large, spatially complex functional inputs.

cs.LG↗

arXiv2025

Agentic Physical AI toward a Domain-Specific Foundation Model for Energy Systems: A Case Study on Nuclear Reactor Control

The prevailing paradigm in AI for physical systems: scaling general-purpose foundation models toward universal multimodal reasoning, confronts a barrier at the control interface. Frontier vision-language models achieve only 50-53% accuracy on basic quantitative physics tasks, behaving as approximate guessers that preserve semantic plausibility while violating physical constraints. Safety-critical control demands outcome-space guarantees over executed actions, not parameter-space imitation. Here we present a pathway toward domain-specific foundation models through compact language models operating as Agentic Physical AI: policy optimization driven by physics-based simulator validation rather than perceptual inference. We train a 360M-parameter model on synthetic nuclear reactor scenarios scaled from 10^3 to 10^5 examples. Scaling produces strong, regime-dependent reliability gains under nominal simulated conditions, with variance collapse of approximately 500x and elimination of >10% terminal-power excursions on the sampled distribution. Despite balanced exposure to four actuation families, the model concentrates 95% of runtime execution on a single-bank strategy, without reinforcement learning or reward engineering. Representations transfer across simulators without architectural change. We position the system as a candidate decision component within a verification, monitoring, and defense-in-depth architecture, not as a stand-alone safety solution: the demonstrated behavior speaks to closed-loop reliability on a single-step task in simulation and does not yet address off-nominal operation, sensor faults, or uncertainty quantification.

cs.AI↗

arXiv2025

A comprehensive comparison of neural operators for 3D industry-scale engineering designs

Neural operators have emerged as powerful tools for learning nonlinear mappings between function spaces, enabling real-time prediction of complex dynamics in diverse scientific and engineering applications. With their growing adoption in engineering design evaluation, a wide range of neural operator architectures have been proposed for various problem settings. However, model selection remains challenging due to the absence of fair and comprehensive comparisons. To address this, we propose and standardize six representative 3D industry-scale engineering design datasets spanning thermal analysis, linear elasticity, elasto-plasticity, time-dependent plastic problems, and computational fluid dynamics. All datasets include fully preprocessed inputs and outputs for model training, making them directly usable across diverse neural operator architectures. Using these datasets, we conduct a systematic comparison of four types of neural operator variants, including Branch-Trunk-based Neural Operators inspired by DeepONet, Graph-based Neural Operators inspired by Graph Neural Networks, Grid-based Neural Operators inspired by Fourier Neural Operators, and Point-based Neural Operators inspired by PointNet. We further introduce practical enhancements to adapt these models to different engineering settings, improving the fairness of the comparison. Our benchmarking study evaluates each model strengths and limitations in terms of predictive performance, computational efficiency, memory usage, and deployment complexity. The findings provide actionable insights to guide future neural operator development.

cs.CE↗

arXiv2025

Bridging Sequential Deep Operator Network and Video Diffusion: Residual Refinement of Spatio-Temporal PDE Solutions

Video-diffusion models have recently set the standard in video generation, inpainting, and domain translation thanks to their training stability and high perceptual fidelity. Building on these strengths, we repurpose conditional video diffusion as a physics surrogate for spatio-temporal fields governed by partial differential equations (PDEs). Our two-stage surrogate first applies a Sequential Deep Operator Network (S-DeepONet) to produce a coarse, physics-consistent prior from the prescribed boundary or loading conditions. The prior is then passed to a conditional video diffusion model that learns only the residual: the point-wise difference between the ground truth and the S-DeepONet prediction. By shifting the learning burden from the full solution to its much smaller residual space, diffusion can focus on sharpening high-frequency structures without sacrificing global coherence. The framework is assessed on two disparate benchmarks: (i) vortex-dominated lid-driven cavity flow and (ii) tensile plastic deformation of dogbone specimens. Across these data sets the hybrid surrogate consistently outperforms its single-stage counterpart, cutting the mean relative L2 error from 4.57% to 0.83% for the flow problem and from 4.42% to 2.94% for plasticity, a relative improvements of 81.8% and 33.5% respectively. The hybrid approach not only lowers quantitative errors but also improves visual quality, visibly recovering fine spatial details. These results show that (i) conditioning diffusion on a physics-aware prior enables faithful reconstruction of localized features, (ii) residual learning reduces the problem, accelerating convergence and enhancing accuracy, and (iii) the same architecture transfers seamlessly from incompressible flow to nonlinear elasto-plasticity without problem-specific architectural modifications, highlighting its broad applicability to nonlinear, time-dependent continua.

cs.CE↗

arXiv2025

Sequential Neural Operator Transformer for High-Fidelity Surrogates of Time-Dependent Non-linear Partial Differential Equations

Partial differential equations (PDEs) are fundamental to modeling complex and nonlinear physical phenomena, but their numerical solution often requires significant computational resources, particularly when a large number of forward full solution evaluations are necessary, such as in design, optimization, sensitivity analysis, and uncertainty quantification. Recent progress in operator learning has enabled surrogate models that efficiently predict full PDE solution fields; however, these models often struggle with accuracy and robustness when faced with highly nonlinear responses driven by sequential input functions. To address these challenges, we propose the Sequential Neural Operator Transformer (S-NOT), a architecture that combines gated recurrent units (GRUs) with the self-attention mechanism of transformers to address time-dependent,nonlinear PDEs. Unlike S-DeepONet (S-DON), which uses a dot product to merge encoded outputs from the branch and trunk sub-networks, S-NOT leverages attention to better capture intricate dependencies between sequential inputs and spatial query points. We benchmark S-NOT on three challenging datasets from real-world applications with plastic and thermo-viscoplastic highly nonlinear material responses: multiphysics steel solidification, a 3D lug specimen, and a dogbone specimen under temporal and path-dependent loadings. The results show that S-NOT consistently achieves a higher prediction accuracy than S-DON even for data outliers, demonstrating its accuracy and robustness for drastically accelerating computational frameworks in scientific and engineering applications.

physics.comp-ph↗

arXiv2025

Single vs. Multiple Branches in DeepONet and S-DeepONet: Network Architecture Follows Coupling in Multiphysics Systems

`Real-time prediction of complex physical systems requires surrogate models that learn from data while representing strong multiphysics coupling. Deep Operator Networks have shown success in single-physics problems, yet their effectiveness in capturing nonlinear interactions in coupled systems (such as thermo-mechanical or electro-thermal coupling) remains underexplored. Here we pose a practical question: should the architecture of a neural operator reflect the strength of physical coupling it aims to model? We compare single-branch and multi-branch designs, in both feedforward and sequential recurrent forms, across three representative systems: a reaction--diffusion problem with heterogeneous sources, a nonlinear thermo-electrical problem with temperature-dependent conductivity and Joule heating, and a viscoplastic thermo-mechanical model of steel solidification. Single-branch networks consistently outperform multi-branch variants in tightly coupled regimes by encouraging shared latent representations, whereas multi-branch designs remain favorable for decoupled or single-physics tasks. Once trained, these surrogates deliver full-field predictions up to $1.8 \times 10^4$ times faster than physics-based solvers.

cs.LG↗

arXiv2025

From Proxies to Fields: Spatiotemporal Reconstruction of Global Radiation from Sparse Sensor Sequences

Accurate reconstruction of latent environmental fields from sparse, indirect observations is a fundamental challenge across scientific domains, from atmospheric science and geophysics to public health and aerospace safety. Existing approaches typically rely on physics-based simulations or dense sensor networks; however, these methods are hampered by high computational cost, latency, and limited spatial coverage. Here we introduce the \textbf{Temporal Radiation Operator Network (TRON)}, a spatiotemporal neural operator architecture that infers continuous global scalar fields solely from sequences of sparse, non-uniform proxy measurements. Unlike recent prediction models that require dense, gridded inputs to predict system states, TRON tackles sparse-to-dense, cross-domain field reconstruction. It reconstructs the current global field in real time from sparse, temporally evolving sensor data, without access to any future observations or dense ground-truth fields. We demonstrate this approach on global cosmic radiation dose mapping: TRON, trained on daily reference fields spanning 2001 to 2023, generalizes across 65,341 spatial locations with input sequences ranging from 7 to 90 days. It achieves sub-second inference with relative $L_2$ errors below 0.1\%, representing over a 58,000$\times$ speedup compared to physics-based estimators. Although showcased in the radiation application, TRON provides a domain-agnostic framework for continuous field reconstruction from sparse data, with broad applications in atmospheric modeling, geophysical hazard monitoring, and real-time environmental risk prediction.

cs.LG↗

arXiv2025

Geometry-Informed Neural Operator Transformer

Machine-learning-based surrogate models offer significant computational efficiency and faster simulations compared to traditional numerical methods, especially for problems requiring repeated evaluations of partial differential equations. This work introduces the Geometry-Informed Neural Operator Transformer (GINOT), which integrates the transformer architecture with the neural operator framework to enable forward predictions on arbitrary geometries. GINOT employs a sampling and grouping strategy together with an attention mechanism to encode surface point clouds that are unordered, exhibit non-uniform point densities, and contain varying numbers of points for different geometries. The geometry information is seamlessly integrated with query points in the solution decoder through the attention mechanism. The performance of GINOT is validated on multiple challenging datasets, showcasing its high accuracy and strong generalization capabilities for complex and arbitrary 2D and 3D geometries.

cs.LG↗

arXiv2025

Towards Signed Distance Function based Metamaterial Design: Neural Operator Transformer for Forward Prediction and Diffusion Model for Inverse Design

The inverse design of metamaterial architectures presents a significant challenge, particularly for nonlinear mechanical properties involving large deformations, buckling, contact, and plasticity. Traditional methods, such as gradient-based optimization, and recent generative deep-learning approaches often rely on binary pixel-based representations, which introduce jagged edges that hinder finite element (FE) simulations and 3D printing. To overcome these challenges, we propose an inverse design framework that utilizes a signed distance function (SDF) representation combined with a conditional diffusion model. The SDF provides a smooth boundary representation, eliminating the need for post-processing and ensuring compatibility with FE simulations and manufacturing methods. A classifier-free guided diffusion model is trained to generate SDFs conditioned on target macroscopic stress-strain curves, enabling efficient one-shot design synthesis. To assess the mechanical response of the generated designs, we introduce a forward prediction model based on Neural Operator Transformers (NOT), which accurately predicts homogenized stress-strain curves and local solution fields for arbitrary geometries with irregular query meshes. This approach enables a closed-loop process for general metamaterial design, offering a pathway for the development of advanced functional materials.

physics.comp-ph↗

arXiv2024

Virtual Sensing to Enable Real-Time Monitoring of Inaccessible Locations & Unmeasurable Parameters

Real-time monitoring of safety-critical interior states is an open problem across energy, environmental and industrial systems where direct instrumentation is infeasible. Approaches based on governing equations, discrete state vectors or fixed sensor locations cannot deliver mesh-independent, field-level reconstruction at arbitrary interior coordinates in real time. Here we introduce neural operator-based virtual sensing, a framework for recovering inaccessible interior fields from sparse boundary measurements, instantiated with MIMONet, a multi-input, multi-output neural operator that fuses heterogeneous inputs and decodes coupled fields through a shared latent representation. Across three engineering-grade evaluations of escalating complexity - confined recirculating flow, pressurized water reactor subchannels and compact power-system heat exchangers - MIMONet attains relative reconstruction error below 5% with sub-millisecond inference, with calibrated uncertainty and noise resilience. We validate the framework on three independent real-world datasets spanning electrochemical energy, atmospheric science and physical oceanography: current-density mapping in a hydrogen fuel cell, wind-speed profiling on meteorological towers, and North Atlantic ocean state from a global reanalysis. MIMONet improves on classical virtual-sensing baselines by 72-84%, including 84% for fuel-cell internal fields recovered from 10% sensor coverage and 83% for hub-height wind speed recovered from lower-level tower sensors. Most notably, it recovers sea-surface height, a field with no direct sensors, from its learned coupling to observed temperature and salinity at 4.1x lower error than the best classical predictor. These results establish operator-based virtual sensing as a practical route to real-time field observability in systems where the states that matter most cannot be measured directly.

cs.LG↗

arXiv2024

Univariate Conditional Variational Autoencoder for Morphogenic Patterns Design in Frontal Polymerization-Based Manufacturing

Under some initial and boundary conditions, the rapid reaction-thermal diffusion process taking place during frontal polymerization (FP) destabilizes the planar mode of front propagation, leading to spatially varying, complex hierarchical patterns in thermoset polymeric materials. Although modern reaction-diffusion models can predict the patterns resulting from unstable FP, the inverse design of patterns, which aims to retrieve process conditions that produce a desired pattern, remains an open challenge due to the non-unique and non-intuitive mapping between process conditions and manufactured patterns. In this work, we propose a probabilistic generative model named univariate conditional variational autoencoder (UcVAE) for the inverse design of hierarchical patterns in FP-based manufacturing. Unlike the cVAE, which encodes both the design space and the design target, the UcVAE encodes only the design space. In the encoder of the UcVAE, the number of training parameters is significantly reduced compared to the cVAE, resulting in a shorter training time while maintaining comparable performance. Given desired pattern images, the trained UcVAE can generate multiple process condition solutions that produce high-fidelity hierarchical patterns.

physics.comp-ph↗

arXiv2024

Virtual Sensing-Enabled Digital Twin Framework for Real-Time Monitoring of Nuclear Systems Leveraging Deep Neural Operators

Effective real-time monitoring is a foundation of digital twin technology, crucial for detecting material degradation and maintaining the structural integrity of nuclear systems to ensure both safety and operational efficiency. Traditional physical sensor systems face limitations such as installation challenges, high costs, and difficulty measuring critical parameters in hard-to-reach or harsh environments, often resulting in incomplete data coverage. Machine learning-driven virtual sensors, integrated within a digital twin framework, offer a transformative solution by enhancing physical sensor capabilities to monitor critical degradation indicators like pressure, velocity, and turbulence. However, conventional machine learning models struggle with real-time monitoring due to the high-dimensional nature of reactor data and the need for frequent retraining. This paper introduces the use of Deep Operator Networks (DeepONet) as a core component of a digital twin framework to predict key thermal-hydraulic parameters in the hot leg of an AP-1000 Pressurized Water Reactor (PWR). DeepONet serves as a dynamic and scalable virtual sensor by accurately mapping the interplay between operational input parameters and spatially distributed system behaviors. In this study, DeepONet is trained with different operational conditions, which relaxes the requirement of continuous retraining, making it suitable for online and real-time prediction components for digital twin. Our results show that DeepONet achieves accurate predictions with low mean squared error and relative L2 error and can make predictions on unknown data 1400 times faster than traditional CFD simulations. This speed and accuracy enable DeepONet to synchronize with the physical system in real-time, functioning as a dynamic virtual sensor that tracks degradation-contributing conditions.

cs.LG↗

arXiv2024

Nonlinear Inverse Design of Mechanical Multi-Material Metamaterials Enabled by Video Denoising Diffusion and Structure Identifier

Metamaterials, synthetic materials with customized properties, have emerged as a promising field due to advancements in additive manufacturing. These materials derive unique mechanical properties from their internal lattice structures, which are often composed of multiple materials that repeat geometric patterns. While traditional inverse design approaches have shown potential, they struggle to map nonlinear material behavior to multiple possible structural configurations. This paper presents a novel framework leveraging video diffusion models, a type of generative artificial Intelligence (AI), for inverse multi-material design based on nonlinear stress-strain responses. Our approach consists of two key components: (1) a fields generator using a video diffusion model to create solution fields based on target nonlinear stress-strain responses, and (2) a structure identifier employing two UNet models to determine the corresponding multi-material 2D design. By incorporating multiple materials, plasticity, and large deformation, our innovative design method allows for enhanced control over the highly nonlinear mechanical behavior of metamaterials commonly seen in real-world applications. It offers a promising solution for generating next-generation metamaterials with finely tuned mechanical characteristics.

cs.AI↗

arXiv2024

Advanced Deep Operator Networks to Predict Multiphysics Solution Fields in Materials Processing and Additive Manufacturing

Unlike classical artificial neural networks, which require retraining for each new set of parametric inputs, the Deep Operator Network (DeepONet), a lately introduced deep learning framework, approximates linear and nonlinear solution operators by taking parametric functions (infinite-dimensional objects) as inputs and mapping them to complete solution fields. In this paper, two newly devised DeepONet formulations with sequential learning and Residual U-Net (ResUNet) architectures are trained for the first time to simultaneously predict complete thermal and mechanical solution fields under variable loading, loading histories, process parameters, and even variable geometries. Two real-world applications are demonstrated: 1- coupled thermo-mechanical analysis of steel continuous casting with multiple visco-plastic constitutive laws and 2- sequentially coupled direct energy deposition for additive manufacturing. Despite highly challenging spatially variable target stress distributions, DeepONets can infer reasonably accurate full-field temperature and stress solutions several orders of magnitude faster than traditional and highly optimized finite-element analysis (FEA), even when FEA simulations are run on the latest high-performance computing platforms. The proposed DeepONet model's ability to provide field predictions almost instantly for unseen input parameters opens the door for future preliminary evaluation and design optimization of these vital industrial processes.

cs.CE↗

arXiv2024

Geom-DeepONet: A Point-cloud-based Deep Operator Network for Field Predictions on 3D Parameterized Geometries

Modern digital engineering design process commonly involves expensive repeated simulations on varying three-dimensional (3D) geometries. The efficient prediction capability of neural networks (NNs) makes them a suitable surrogate to provide design insights. Nevertheless, few available NNs can handle solution prediction on varying 3D shapes. We present a novel deep operator network (DeepONet) variant called Geom-DeepONet, which encodes parameterized 3D geometries and predicts full-field solutions on an arbitrary number of nodes. To the best of the authors' knowledge, this is the first attempt in the literature and is our primary novelty. In addition to expressing shapes using mesh coordinates, the signed distance function for each node is evaluated and used to augment the inputs to the trunk network of the Geom-DeepONet, thereby capturing both explicit and implicit representations of the 3D shapes. The powerful geometric encoding capability of a sinusoidal representation network (SIREN) is also exploited by replacing the classical feedforward neural networks in the trunk with SIREN. Additional data fusion between the branch and trunk networks is introduced by an element-wise product. A numerical benchmark was conducted to compare Geom-DeepONet to PointNet and vanilla DeepONet, where results show that our architecture trains fast with a small memory footprint and yields the most accurate results among the three with less than 2 MPa stress error. Results show a much lower generalization error of our architecture on unseen dissimilar designs than vanilla DeepONet. Once trained, the model can predict vector solutions, and speed can be over $10^5$ times faster than implicit finite element simulations for large meshes.

cs.CE↗

arXiv2024

Material-Response-Informed DeepONet and its Application to Polycrystal Stress-strain Prediction in Crystal Plasticity

Crystal plasticity (CP) simulations are a tool for understanding how microstructure morphology and texture affect mechanical properties and are an essential component of elucidating the structure-property relations. However, it can be computationally expensive. Hence, data-driven machine learning models have been applied to predict the mean-field response of a polycrystal representative volume element to reduce computation time. In this work, we proposed a novel Deep Operator Network (DeepONet) architecture for predicting microstructure stress-strain response. It employs a convolutional neural network in the trunk to encode the microstructure. To account for different material properties, boundary conditions, and loading, we proposed using single crystal stress-strain curves as inputs to the branch network, furnishing a material-response-informed DeepONet. Using four numerical examples, we demonstrate that the current DeepONet can be trained on a single material and loading and then generalized to new conditions via transfer learning. Results show that using single crystal responses as input outperforms a similar model using material properties as inputs and overcomes limitations with changing boundary conditions and temporal resolution. In all cases, the new model achieved a $R^2$ value of above 0.99, and over 95\% of predicted stresses have a relative error of $\le$ 5\%, indicating superior accuracy. With as few as 20 new data points and under 1min training time, the trained DeepONet can be fine-tuned to generate accurate predictions on different materials and loading. Once trained, the prediction speed is almost $1\times10^{4}$ times faster the CP simulations. The efficiency and high generalizability of our DeepONet render it a powerful data-driven surrogate model for CP simulations in multi-scale analyses.

cs.CE↗

arXiv2023

Predictions of Transient Vector Solution Fields with Sequential Deep Operator Network

The Deep Operator Network (DeepONet) structure has shown great potential in approximating complex solution operators with low generalization errors. Recently, a sequential DeepONet (S-DeepONet) was proposed to use sequential learning models in the branch of DeepONet to predict final solutions given time-dependent inputs. In the current work, the S-DeepONet architecture is extended by modifying the information combination mechanism between the branch and trunk networks to simultaneously predict vector solutions with multiple components at multiple time steps of the evolution history, which is the first in the literature using DeepONets. Two example problems, one on transient fluid flow and the other on path-dependent plastic loading, were shown to demonstrate the capabilities of the model to handle different physics problems. The use of a trained S-DeepONet model in inverse parameter identification via the genetic algorithm is shown to demonstrate the application of the model. In almost all cases, the trained model achieved an $R^2$ value of above 0.99 and a relative $L_2$ error of less than 10\% with only 3200 training data points, indicating superior accuracy. The vector S-DeepONet model, having only 0.4\% more parameters than a scalar model, can predict two output components simultaneously at an accuracy similar to the two independently trained scalar models with a 20.8\% faster training time. The S-DeepONet inference is at least three orders of magnitude faster than direct numerical simulations, and inverse parameter identifications using the trained model is highly efficient and accurate.

cs.CE↗

arXiv2023

Towards Exascale Computation for Turbomachinery Flows

A state-of-the-art large eddy simulation code has been developed to solve compressible flows in turbomachinery. The code has been engineered with a high degree of scalability, enabling it to effectively leverage the many-core architecture of the new Sunway system. A consistent performance of 115.8 DP-PFLOPs has been achieved on a high-pressure turbine cascade consisting of over 1.69 billion mesh elements and 865 billion Degree of Freedoms (DOFs). By leveraging a high-order unstructured solver and its portability to large heterogeneous parallel systems, we have progressed towards solving the grand challenge problem outlined by NASA, which involves a time-dependent simulation of a complete engine, incorporating all the aerodynamic and heat transfer components.

cs.DC↗