arXiv subjects

Qibang Liu

Publications and source records attributed to Qibang Liu.

11 recordsLinked to original sources

arXiv2026

Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

Deep neural operators learn mappings between input functions and complete PDE solution fields, enabling forward evaluations of new problem instances orders of magnitude faster than conventional numerical solvers. Attention mechanisms have recently been introduced into neural operators, but most studies change several architectural components at once, making it difficult to identify what actually improves accuracy. This work presents a controlled and systematic study of five deep operator network (DeepONet) variants with distinct attention mechanisms, trained under both data-driven and physics-informed regimes, to isolate the effects of cross-attention, self-attention, tokenization, and attention depth. We evaluate them on a source-driven transient one-dimensional nonlinear diffusion-reaction equation, a transient one-dimensional viscous Burgers equation with variable initial conditions, and a two-dimensional Poisson heat-conduction problem with heterogeneous source fields. Per-sensor tokenization with cross-attention reduces the mean relative L_2 error of the classical DeepONet in all benchmark-training combinations by factors of 2.4-28.0, while the best attention configurations reach 3.5-32.3. Branch self-attention paired only with dot-product fusion is inconsistent, degrading the one-dimensional problems while helping the more complex two-dimensional source field; added on top of cross-attention it improves all six cases, though by less than cross-attention fusion alone. Global pre-mixing provides no consistent benefit. Increasing cross-attention depth further improves accuracy, but with diminishing returns and a substantially higher cost under physics-informed training. Overall, query-dependent cross-attention is the most reliable mechanism, whereas branch self-attention is most useful for large, spatially complex functional inputs.

cs.LG↗

arXiv2025

A comprehensive comparison of neural operators for 3D industry-scale engineering designs

Neural operators have emerged as powerful tools for learning nonlinear mappings between function spaces, enabling real-time prediction of complex dynamics in diverse scientific and engineering applications. With their growing adoption in engineering design evaluation, a wide range of neural operator architectures have been proposed for various problem settings. However, model selection remains challenging due to the absence of fair and comprehensive comparisons. To address this, we propose and standardize six representative 3D industry-scale engineering design datasets spanning thermal analysis, linear elasticity, elasto-plasticity, time-dependent plastic problems, and computational fluid dynamics. All datasets include fully preprocessed inputs and outputs for model training, making them directly usable across diverse neural operator architectures. Using these datasets, we conduct a systematic comparison of four types of neural operator variants, including Branch-Trunk-based Neural Operators inspired by DeepONet, Graph-based Neural Operators inspired by Graph Neural Networks, Grid-based Neural Operators inspired by Fourier Neural Operators, and Point-based Neural Operators inspired by PointNet. We further introduce practical enhancements to adapt these models to different engineering settings, improving the fairness of the comparison. Our benchmarking study evaluates each model strengths and limitations in terms of predictive performance, computational efficiency, memory usage, and deployment complexity. The findings provide actionable insights to guide future neural operator development.

cs.CE↗

arXiv2025

Sequential Neural Operator Transformer for High-Fidelity Surrogates of Time-Dependent Non-linear Partial Differential Equations

Partial differential equations (PDEs) are fundamental to modeling complex and nonlinear physical phenomena, but their numerical solution often requires significant computational resources, particularly when a large number of forward full solution evaluations are necessary, such as in design, optimization, sensitivity analysis, and uncertainty quantification. Recent progress in operator learning has enabled surrogate models that efficiently predict full PDE solution fields; however, these models often struggle with accuracy and robustness when faced with highly nonlinear responses driven by sequential input functions. To address these challenges, we propose the Sequential Neural Operator Transformer (S-NOT), a architecture that combines gated recurrent units (GRUs) with the self-attention mechanism of transformers to address time-dependent,nonlinear PDEs. Unlike S-DeepONet (S-DON), which uses a dot product to merge encoded outputs from the branch and trunk sub-networks, S-NOT leverages attention to better capture intricate dependencies between sequential inputs and spatial query points. We benchmark S-NOT on three challenging datasets from real-world applications with plastic and thermo-viscoplastic highly nonlinear material responses: multiphysics steel solidification, a 3D lug specimen, and a dogbone specimen under temporal and path-dependent loadings. The results show that S-NOT consistently achieves a higher prediction accuracy than S-DON even for data outliers, demonstrating its accuracy and robustness for drastically accelerating computational frameworks in scientific and engineering applications.

physics.comp-ph↗

arXiv2025

Single vs. Multiple Branches in DeepONet and S-DeepONet: Network Architecture Follows Coupling in Multiphysics Systems

`Real-time prediction of complex physical systems requires surrogate models that learn from data while representing strong multiphysics coupling. Deep Operator Networks have shown success in single-physics problems, yet their effectiveness in capturing nonlinear interactions in coupled systems (such as thermo-mechanical or electro-thermal coupling) remains underexplored. Here we pose a practical question: should the architecture of a neural operator reflect the strength of physical coupling it aims to model? We compare single-branch and multi-branch designs, in both feedforward and sequential recurrent forms, across three representative systems: a reaction--diffusion problem with heterogeneous sources, a nonlinear thermo-electrical problem with temperature-dependent conductivity and Joule heating, and a viscoplastic thermo-mechanical model of steel solidification. Single-branch networks consistently outperform multi-branch variants in tightly coupled regimes by encouraging shared latent representations, whereas multi-branch designs remain favorable for decoupled or single-physics tasks. Once trained, these surrogates deliver full-field predictions up to $1.8 \times 10^4$ times faster than physics-based solvers.

cs.LG↗

arXiv2025

Geometry-Informed Neural Operator Transformer

Machine-learning-based surrogate models offer significant computational efficiency and faster simulations compared to traditional numerical methods, especially for problems requiring repeated evaluations of partial differential equations. This work introduces the Geometry-Informed Neural Operator Transformer (GINOT), which integrates the transformer architecture with the neural operator framework to enable forward predictions on arbitrary geometries. GINOT employs a sampling and grouping strategy together with an attention mechanism to encode surface point clouds that are unordered, exhibit non-uniform point densities, and contain varying numbers of points for different geometries. The geometry information is seamlessly integrated with query points in the solution decoder through the attention mechanism. The performance of GINOT is validated on multiple challenging datasets, showcasing its high accuracy and strong generalization capabilities for complex and arbitrary 2D and 3D geometries.

cs.LG↗

arXiv2025

Towards Signed Distance Function based Metamaterial Design: Neural Operator Transformer for Forward Prediction and Diffusion Model for Inverse Design

The inverse design of metamaterial architectures presents a significant challenge, particularly for nonlinear mechanical properties involving large deformations, buckling, contact, and plasticity. Traditional methods, such as gradient-based optimization, and recent generative deep-learning approaches often rely on binary pixel-based representations, which introduce jagged edges that hinder finite element (FE) simulations and 3D printing. To overcome these challenges, we propose an inverse design framework that utilizes a signed distance function (SDF) representation combined with a conditional diffusion model. The SDF provides a smooth boundary representation, eliminating the need for post-processing and ensuring compatibility with FE simulations and manufacturing methods. A classifier-free guided diffusion model is trained to generate SDFs conditioned on target macroscopic stress-strain curves, enabling efficient one-shot design synthesis. To assess the mechanical response of the generated designs, we introduce a forward prediction model based on Neural Operator Transformers (NOT), which accurately predicts homogenized stress-strain curves and local solution fields for arbitrary geometries with irregular query meshes. This approach enables a closed-loop process for general metamaterial design, offering a pathway for the development of advanced functional materials.

physics.comp-ph↗

arXiv2025

Effects of Prompt Length on Domain-specific Tasks for Large Language Models

In recent years, Large Language Models have garnered significant attention for their strong performance in various natural language tasks, such as machine translation and question answering. These models demonstrate an impressive ability to generalize across diverse tasks. However, their effectiveness in tackling domain-specific tasks, such as financial sentiment analysis and monetary policy understanding, remains a topic of debate, as these tasks often require specialized knowledge and precise reasoning. To address such challenges, researchers design various prompts to unlock the models' abilities. By carefully crafting input prompts, researchers can guide these models to produce more accurate responses. Consequently, prompt engineering has become a key focus of study. Despite the advancements in both models and prompt engineering, the relationship between the two-specifically, how prompt design impacts models' ability to perform domain-specific tasks-remains underexplored. This paper aims to bridge this research gap.

cs.CL↗

arXiv2024

Univariate Conditional Variational Autoencoder for Morphogenic Patterns Design in Frontal Polymerization-Based Manufacturing

Under some initial and boundary conditions, the rapid reaction-thermal diffusion process taking place during frontal polymerization (FP) destabilizes the planar mode of front propagation, leading to spatially varying, complex hierarchical patterns in thermoset polymeric materials. Although modern reaction-diffusion models can predict the patterns resulting from unstable FP, the inverse design of patterns, which aims to retrieve process conditions that produce a desired pattern, remains an open challenge due to the non-unique and non-intuitive mapping between process conditions and manufactured patterns. In this work, we propose a probabilistic generative model named univariate conditional variational autoencoder (UcVAE) for the inverse design of hierarchical patterns in FP-based manufacturing. Unlike the cVAE, which encodes both the design space and the design target, the UcVAE encodes only the design space. In the encoder of the UcVAE, the number of training parameters is significantly reduced compared to the cVAE, resulting in a shorter training time while maintaining comparable performance. Given desired pattern images, the trained UcVAE can generate multiple process condition solutions that produce high-fidelity hierarchical patterns.

physics.comp-ph↗

arXiv2024

Nonlinear Inverse Design of Mechanical Multi-Material Metamaterials Enabled by Video Denoising Diffusion and Structure Identifier

Metamaterials, synthetic materials with customized properties, have emerged as a promising field due to advancements in additive manufacturing. These materials derive unique mechanical properties from their internal lattice structures, which are often composed of multiple materials that repeat geometric patterns. While traditional inverse design approaches have shown potential, they struggle to map nonlinear material behavior to multiple possible structural configurations. This paper presents a novel framework leveraging video diffusion models, a type of generative artificial Intelligence (AI), for inverse multi-material design based on nonlinear stress-strain responses. Our approach consists of two key components: (1) a fields generator using a video diffusion model to create solution fields based on target nonlinear stress-strain responses, and (2) a structure identifier employing two UNet models to determine the corresponding multi-material 2D design. By incorporating multiple materials, plasticity, and large deformation, our innovative design method allows for enhanced control over the highly nonlinear mechanical behavior of metamaterials commonly seen in real-world applications. It offers a promising solution for generating next-generation metamaterials with finely tuned mechanical characteristics.

cs.AI↗

arXiv2022

Adaptive coupling peridynamic least-square minimization with finite element method for fracture analysis

This study presents an adaptive coupling peridynamic least-square minimization with the finite element method (PDLSM-FEM) for fracture analysis. The presented method utilizes the PDLSM modeling discontinuities while maximizing the FEM region for computational efficiency. Within the presented adaptive PDLSM-FEM, only elements intersecting with the crack path and their neighboring elements are defined as PD elements, whose stiffness matrices are derived based on PDLSM equations. The remaining elements are conventional finite elements. Numerical integration of interaction integral is proposed and implemented to evaluate the stress intensity factors (SIFs) for 2-D problems. The criterion of maximum hoop tensile stress is employed for failure prediction. New contributions of this work include the adaptive coupling of PDLSM with FEM for minimizing the PD region and the application of the adaptive PDLSM-FEM to quasi-static crack propagation analysis. Simulations of three 2-D plane stress plates and one 3-D block with static or quasi-static cracks propagation are performed. Results show the proposed method improves computational efficiency substantially and has reasonable accuracy and good capability of crack propagation prediction.

math.NA↗

arXiv2022

An extended ordinary state-based peridynamics for non-spherical horizons

This work presents an extended ordinary state-based peridynamics (XOSBPD) model for the non-spherical horizons. Based on the OSBPD, we derive the XOSBPD by introducing the Lagrange multipliers to guarantee the non-local dilatation and non-local strain energy density (SED) are equal to local dilatation and local SED, respectively. In this formulation, the XOSBPD removes the limitation of spherical horizons and is suitable for arbitrary horizon shapes. In addition, the presented XOSBPD does not need volume and surface correction and allows non-uniform discretization implementation with various horizon sizes. Three classic examples demonstrate the accuracy and capability for complex dynamical fracture analysis. The proposed method provides an efficient tool and in-depth insight into the failure mechanism of structure components and solid materials.

cs.CE↗