arXiv subjects

Shengze Cai

Publications and source records attributed to Shengze Cai.

At least 19 recordsLinked to original sources

arXiv2026

Acoustic-driven millimetric helical robot: ultrasonic synergistic manipulation in confined fluidic environment

Acoustic field-driven manipulation provides a non-contact and non-invasive strategy for controlling microscale and nanoscale objects, yet its extension to millimeter-scale robots was limited by insufficient propulsion efficiency in confined biological environments. Here, a coordinated multi-acoustic-field approach is introduced, which harnesses the synergistic action of acoustic radiation forces and acoustic streaming flows to enable controlled locomotion of millimeter-scale helical robots and enhance propulsion. Multiphysics simulations captured the dynamics of millimeter-scale helical robots under combined acoustic fields, and experimental validation demonstrated their locomotion capabilities, including planar navigation, inclined climbing, and vertical motion. Semi-autonomous navigation experiments further confirmed that ultrasonic synergy substantially improved maneuverability. In vitro tests in porcine venous vessels demonstrated that coordinated acoustic fields supported both unidirectional and reciprocating motion under biologically relevant confinement. These findings provide mechanistic insight into scaling acoustic micromanipulation to the millimetre regime and support biomedical applications requiring versatile and controllable robotic mobility.

cs.RO↗

arXiv2026

DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling

Modern language models are built primarily from Transformers, recurrent models, and their hybrid architectures. Transformers rely on token-level attention memories, while recurrent models such as state space models (SSMs) and linear attention maintain compact recurrent states. These architectures are typically instantiated separately or interleaved at the layer level, leaving open whether a shared memory representation can support both recurrent compression and attention-style retrieval. We study this question through the state space duality (SSD) view of Mamba-2, where the SSM state can be interpreted as a compressed associative key--value (KV) cache. We observe that Mamba-2 decodes token-conditioned values from this state but does not decode token-conditioned keys. Based on this observation, we propose DART (Decoded Attention over Recurrent sTates), which retains the chunk state contributions produced by the Mamba-2 chunked scan as chunk state memories, decodes token-conditioned keys and values from these memories, and performs state-memory attention (SMA) over the resulting KV pairs. The retrieved output is then combined with the native Mamba-2 output through a gated residual connection. DART supports practical training by reusing the Mamba-2 chunked scan and implementing SMA as a FlashAttention-style computation. Our analysis and experiments show that DART substantially reduces the length-dependent inference cache compared with a matched attention baseline (e.g., $75\%$ savings when the chunk size is $S=256$ and the state size is $N=128$). Compared with Mamba-2, DART substantially improves associative recall and retrieval while preserving general language-modeling quality.

cs.LG↗

arXiv2026

DSSMs: State Space Models with Explicit Memory via Delay Differential Equations

State Space Models (SSMs) have emerged as a powerful paradigm for efficient long-sequence modeling, offering parallel training and fast linear-time recurrent inference. However, like other recurrent architectures, SSMs must compress an unbounded history into a fixed-size state, which limits context retention and makes precise retrieval over long-range context inherently difficult. To overcome this limitation, we propose Delay State Space Models (DSSMs), a delay differential equation (DDE)-inspired extension of diagonal SSMs that augments discrete SSM recurrences with explicit delayed-state feedback. Making explicit delayed feedback practical requires new stability parameterization, history management, and FFT-training tools. We address these challenges with a practical discretization and parameterization grounded in a simple delay-independent stability condition. To bypass direct time-domain kernel construction, we derive the DSSM transfer function and compute kernels in the frequency domain, using a kernel contour shift to suppress aliasing and recover accurate FFT training. Empirically, DSSMs substantially improve targeted delayed-retrieval tasks while outperforming S4D on most standard sequence metrics and remaining close on the others.

cs.LG↗

arXiv2026

Transformer-based Neural Operators for 3D Wind Field Prediction over Complex Mountainous Terrain

Accurate prediction of three-dimensional (3D) wind fields over complex mountainous terrain is essential for renewable energy deployment and regional weather modeling. Traditional computational fluid dynamics (CFD) simulations face two fundamental bottlenecks: expert-intensive mesh generation around irregular topography, and iterative solvers that require hours to days even on high-performance clusters. Recent neural operator approaches accelerate inference, but typically fail to resolve the sharp, localized velocity gradients induced by complex terrain features. Here, we present a transformer-based dual-attention neural-operator framework for 3D wind field prediction over complex mountainous terrain, and validate its effectiveness through two instantiations on representative point-based (mesh-free) and graph-based neural-operator architectures, namely Patch-solver and Patch-GTO. Trained on a large CFD-generated dataset spanning diverse terrain geometries and inflow conditions, the framework enables rapid prediction of steady-state wind field while maintaining competitive accuracy. It also demonstrates robust zero-shot transfer to real-world mountainous sites across several diverse locations, outperforming existing neural operator baselines by 10% in relative error. We further verify that incorporating sparse observational data (1% spatial coverage) reduces prediction error by 16.89% relative to the corresponding model without sparse data input and by 32.75% relative to advanced neural operator baselines on unseen terrains. This framework establishes a generalizable computational paradigm across domains, promising to be a real-time tool for wind resource assessment over complex mountainous terrain and related atmosphere-surface interaction studies.

physics.flu-dyn↗

arXiv2026

BadmintonGRF: A Multimodal Dataset and Benchmark for Markerless Ground Reaction Force Estimation in Badminton

Multimodal resources for non-periodic court sports with laboratory-grade sensing remain scarce: few publicly pair instrumented ground reaction force (GRF) with high-frame-rate multi-view video, limiting markerless load estimation in realistic training settings. BadmintonGRF records eight synchronized RGB views at ~120 FPS, four Kistler force plates, and Vicon motion capture (C3D) without hardware genlock across modalities; alignment combines human-verified events, automated quality assurance, and per-camera time offsets with uncertainty metadata. Tier 1 distributes pose, time-aligned GRF, metadata, and splits under CC BY-NC 4.0, enabling the primary benchmark without raw RGB or C3D; we report a Tier 1 task that maps 2D pose to GRF. Tier 2 provides raw RGB and C3D under controlled access for studies that require appearance or full kinematics. The public release contains 17,425 impact-segment archives in the 10-subject benchmark tree (156 instrumented trials; raw multi-view RGB alone exceeds 1 TB); benchmark loader gates retain 12,867 view-specific instances and 1,732 unique impacts after multi-view deduplication. We are not aware of prior public badminton corpora that combine this sensing layout with audited video--GRF alignment for impact-centric GRF estimation. We distribute preprocessing code, leave-one-subject-out splits, ten reference baselines, and optional late fusion (one deterministic test-time pass per instance; no test-time augmentation), with a within-trial diagnostic in the supplementary material.

cs.CV↗

arXiv2026

Deep learning accelerated solutions of incompressible Navier-Stokes equations on non-uniform Cartesian grids

In incompressible flow simulations, non-uniform grids efficiently capture localized flow features; however, their spatially varying resolutions severely exacerbate computational complexity. The pressure Poisson equation (PPE) formulated on these grids yields highly complex linear systems, forming the primary computational bottleneck in fractional step method. To address this, we develop an extended hybrid framework tailored for non-uniform Cartesian grids, integrating deep learning with classical iterative solvers to accelerate PPE solutions. Specifically, the framework employs a deep operator network with a U-Net-based branch network. To effectively capture spatially varying resolutions, we propose a multi-level distance vector map construction strategy that computes discrete grid-spacing information corresponding to each hierarchical level of the U-Net. This grid-spacing information is explicitly fused into feature maps prior to convolution operations. Empowered by this grid-spacing-aware architecture, the framework seamlessly extends to simulate flows interacting with solid structures using a decoupled immersed boundary projection method. By training exclusively on fabricated linear systems rather than conventional flow-dependent datasets, the model generalizes effortlessly across diverse immersed obstacle geometries with fixed network weights. Benchmark results demonstrate that the framework significantly outperforms standalone preconditioned conjugate gradient methods and its standard convolution counterpart, underscoring its exceptional potential for real-world computational fluid dynamics applications.

physics.flu-dyn↗

arXiv2026

Distributed physics-informed neural networks via domain decomposition for fast flow reconstruction

Physics-Informed Neural Networks (PINNs) offer a powerful paradigm for flow reconstruction, seamlessly integrating sparse velocity measurements with the governing Navier-Stokes equations to recover complete velocity and latent pressure fields. However, scaling such models to large spatiotemporal domains is hindered by computational bottlenecks and optimization instabilities. In this work, we propose a robust distributed PINNs framework designed for efficient flow reconstruction via spatiotemporal domain decomposition. A critical challenge in such distributed solvers is pressure indeterminacy, where independent sub-networks drift into inconsistent local pressure baselines. We address this issue through a reference anchor normalization strategy coupled with decoupled asymmetric weighting. By enforcing a unidirectional information flow from designated master ranks where the anchor point lies to neighboring ranks, our approach eliminates gauge freedom and guarantees global pressure uniqueness while preserving temporal continuity. Furthermore, to mitigate the Python interpreter overhead associated with computing high-order physics residuals, we implement a high-performance training pipeline accelerated by CUDA graphs and JIT compilation. Extensive validation on complex flow benchmarks demonstrates that our method achieves near-linear strong scaling and high-fidelity reconstruction, establishing a scalable and physically rigorous pathway for flow reconstruction and understanding of complex hydrodynamics.

cs.LG↗

arXiv2025

An Efficient Graph-Transformer Operator for Learning Physical Dynamics with Manifolds Embedding

Accurate and efficient physical simulations are essential in science and engineering, yet traditional numerical solvers face significant challenges in computational cost when handling simulations across dynamic scenarios involving complex geometries, varying boundary/initial conditions, and diverse physical parameters. While deep learning offers promising alternatives, existing methods often struggle with flexibility and generalization, particularly on unstructured meshes, which significantly limits their practical applicability. To address these challenges, we propose PhysGTO, an efficient Graph-Transformer Operator for learning physical dynamics through explicit manifold embeddings in both physical and latent spaces. In the physical space, the proposed Unified Graph Embedding module aligns node-level conditions and constructs sparse yet structure-preserving graph connectivity to process heterogeneous inputs. In the latent space, PhysGTO integrates a lightweight flux-oriented message-passing scheme with projection-inspired attention to capture local and global dependencies, facilitating multilevel interactions among complex physical correlations. This design ensures linear complexity relative to the number of mesh points, reducing both the number of trainable parameters and computational costs in terms of floating-point operations (FLOPs), and thereby allowing efficient inference in real-time applications. We introduce a comprehensive benchmark spanning eleven datasets, covering problems with unstructured meshes, transient flow dynamics, and large-scale 3D geometries. PhysGTO consistently achieves state-of-the-art accuracy while significantly reducing computational costs, demonstrating superior flexibility, scalability, and generalization in a wide range of simulation tasks.

cs.CE↗

arXiv2025

Lagrangian-Eulerian learning of flow field and trajectories with TrajectoryFlowNet

Predicting particle transport in complex flows is traditionally achieved by solving the Navier-Stokes equations. While various numerical and experimental methods exist, they typically require deep physical insights and incur high computational costs. Machine learning offers an alternative by learning predictive patterns directly from data, avoiding explicit physical modeling. However, purely data-driven approaches often lack interpretability, physical consistency, and generalizability in sparse data regimes. To this end, we propose TrajectoryFlowNet, a Lagrangian-Eulerian physics-informed neural network architecture, for fluid flow velocimetry and imaging via learning to predict spatiotemporal flow fields and long-range particle trajectories. The salient features of our model include its ability to handle complex flow patterns with irregular boundaries, predict the full-field flows, image the long-range flow trajectory of any arbitrary particle, and ensure physical consistency in predictions based only on very scarce measurement of flow trajectories. We validate TrajectoryFlowNet via both numerical examples (e.g., lid-driven cavity flow and complex cylinder flow) and experimental test cases (e.g., aortic and ventricle blood flows) across diverse flow scenarios. The results demonstrate our model's effectiveness in capturing intricate particle-laden flow dynamics, enabling long-range tracking of particles and accurate construction of flow fields in real-world applications.

physics.flu-dyn↗

arXiv2025

Accelerated Distributed Aggregative Optimization

This paper delves into the investigation of a distributed aggregative optimization problem within a network. In this scenario, each agent possesses its own local cost function, which relies not only on the local state variable but also on an aggregated function of state variables from all agents. To expedite the optimization process, we amalgamate the heavy ball and Nesterovs accelerated method with distributed aggregative gradient tracking, resulting in the proposal of two innovative algorithms, aimed at resolving the distributed aggregative optimization problem. Our analysis demonstrates that the proposed algorithms can converge to an optimal solution at a global linear convergence rate when the objective function is strongly convex with the Lipschitz-continuous gradient, and when the parameters (e.g., step size and momentum coefficients) are chosen within specific ranges. Additionally, we present several numerical experiments to verify the effectiveness, robustness and superiority of our proposed algorithms.

math.OC↗

arXiv2024

PiRD: Physics-informed Residual Diffusion for Flow Field Reconstruction

The use of machine learning in fluid dynamics is becoming more common to expedite the computation when solving forward and inverse problems of partial differential equations. Yet, a notable challenge with existing convolutional neural network (CNN)-based methods for data fidelity enhancement is their reliance on specific low-fidelity data patterns and distributions during the training phase. In addition, the CNN-based method essentially treats the flow reconstruction task as a computer vision task that prioritizes the element-wise precision which lacks a physical and mathematical explanation. This dependence can dramatically affect the models' effectiveness in real-world scenarios, especially when the low-fidelity input deviates from the training data or contains noise not accounted for during training. The introduction of diffusion models in this context shows promise for improving performance and generalizability. Unlike direct mapping from a specific low-fidelity to a high-fidelity distribution, diffusion models learn to transition from any low-fidelity distribution towards a high-fidelity one. Our proposed model - Physics-informed Residual Diffusion, demonstrates the capability to elevate the quality of data from both standard low-fidelity inputs, to low-fidelity inputs with injected Gaussian noise, and randomly collected samples. By integrating physics-based insights into the objective function, it further refines the accuracy and the fidelity of the inferred high-quality data. Experimental results have shown that our approach can effectively reconstruct high-quality outcomes for two-dimensional turbulent flows from a range of low-fidelity input conditions without requiring retraining.

physics.flu-dyn↗

arXiv2023

Accelerated Distributed Aggregative Optimization

In this paper, we investigate a distributed aggregative optimization problem in a network, where each agent has its own local cost function which depends not only on the local state variable but also on an aggregated function of state variables from all agents. To accelerate the optimization process, we combine heavy ball and Nesterov's accelerated methods with distributed aggregative gradient tracking, and propose two novel algorithms named DAGT-HB and DAGT-NES for solving the distributed aggregative optimization problem. We analyse that the DAGT-HB and DAGT-NES algorithms can converge to an optimal solution at a global $\mathbf{R}-$linear convergence rate when the objective function is smooth and strongly convex, and when the parameters (e.g., step size and momentum coefficients) are selected within certain ranges. A numerical experiment on the optimal placement problem is given to verify the effectiveness and superiority of our proposed algorithms.

math.OC↗

arXiv2023

The Novel Adaptive Fractional Order Gradient Decent Algorithms Design via Robust Control

The vanilla fractional order gradient descent may oscillatively converge to a region around the global minimum instead of converging to the exact minimum point, or even diverge, in the case where the objective function is strongly convex. To address this problem, a novel adaptive fractional order gradient descent (AFOGD) method and a novel adaptive fractional order accelerated gradient descent (AFOAGD) method are proposed in this paper. Inspired by the quadratic constraints and Lyapunov stability analysis from robust control theory, we establish a linear matrix inequality to analyse the convergence of our proposed algorithms. We prove that the proposed algorithms can achieve R-linear convergence when the objective function is $\textbf{L-}$smooth and $\textbf{m-}$strongly-convex. Several numerical simulations are demonstrated to verify the effectiveness and superiority of our proposed algorithms.

math.OC↗

arXiv2023

Neural Observer with Lyapunov Stability Guarantee for Uncertain Nonlinear Systems

In this paper, we propose a novel nonlinear observer based on neural networks, called neural observer, for observation tasks of linear time-invariant (LTI) systems and uncertain nonlinear systems. In particular, the neural observer designed for uncertain systems is inspired by the active disturbance rejection control, which can measure the uncertainty in real-time. The stability analysis (e.g., exponential convergence rate) of LTI and uncertain nonlinear systems (involving neural observers) are presented and guaranteed, where it is shown that the observation problems can be solved only using the linear matrix inequalities (LMIs). Also, it is revealed that the observability and controllability of the system matrices are required to demonstrate the existence of solutions of LMIs. Finally, the effectiveness of neural observers is verified on three simulation cases, including the X-29A aircraft model, the nonlinear pendulum, and the four-wheel steering vehicle.

math.OC↗

arXiv2022

GotFlow3D: Recurrent Graph Optimal Transport for Learning 3D Flow Motion in Particle Tracking

Flow visualization technologies such as particle tracking velocimetry (PTV) are broadly used in understanding the all-pervasiveness three-dimensional (3D) turbulent flow from nature and industrial processes. Despite the advances in 3D acquisition techniques, the developed motion estimation algorithms in particle tracking remain great challenges of large particle displacements, dense particle distributions and high computational cost. By introducing a novel deep neural network based on recurrent Graph Optimal Transport, called GotFlow3D, we present an end-to-end solution to learn the 3D fluid flow motion from double-frame particle sets. The proposed network constructs two graphs in the geometric and feature space and further enriches the original particle representations with the fused intrinsic and extrinsic features learnt from a graph neural network. The extracted deep features are subsequently utilized to make optimal transport plans indicating the correspondences of particle pairs, which are then iteratively and adaptively retrieved to guide the recurrent flow learning. Experimental evaluations, including assessments on numerical experiments and validations on real-world experiments, demonstrate that the proposed GotFlow3D achieves state-of-the-art performance against both recently-developed scene flow learners and particle tracking algorithms, with impressive accuracy, robustness and generalization ability, which can provide deeper insight into the complex dynamics of broad physical and biological systems.

physics.flu-dyn↗

arXiv2021

A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019, and the second one, named Fourier Neural Operator or FNO, was published in 2020. In order to compare FNO with DeepONet for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also endow DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO. We consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries and especially noisy data, the performance of FNO deteriorates greatly. For example, for the instability wave analysis with only 0.1% noise added to the input data, the error of FNO increases 10000 times making it inappropriate for such important applications, while there is hardly any effect of such noise on the DeepONet. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

physics.comp-ph↗

arXiv2021

AOSLO-net: A deep learning-based method for automatic segmentation of retinal microaneurysms from adaptive optics scanning laser ophthalmoscope images

Microaneurysms (MAs) are one of the earliest signs of diabetic retinopathy (DR), a frequent complication of diabetes that can lead to visual impairment and blindness. Adaptive optics scanning laser ophthalmoscopy (AOSLO) provides real-time retinal images with resolution down to 2 $μm$ and thus allows detection of the morphologies of individual MAs, a potential marker that might dictate MA pathology and affect the progression of DR. In contrast to the numerous automatic models developed for assessing the number of MAs on fundus photographs, currently there is no high throughput image protocol available for automatic analysis of AOSLO photographs. To address this urgency, we introduce AOSLO-net, a deep neural network framework with customized training policies to automatically segment MAs from AOSLO images. We evaluate the performance of AOSLO-net using 87 DR AOSLO images and our results demonstrate that the proposed model outperforms the state-of-the-art segmentation model both in accuracy and cost and enables correct MA morphological classification.

eess.IV↗

arXiv2021

Physics-informed neural networks (PINNs) for fluid mechanics: A review

Despite the significant progress over the last 50 years in simulating flow problems using numerical discretization of the Navier-Stokes equations (NSE), we still cannot incorporate seamlessly noisy data into existing algorithms, mesh-generation is complex, and we cannot tackle high-dimensional problems governed by parametrized NSE. Moreover, solving inverse flow problems is often prohibitively expensive and requires complex and expensive formulations and new computer codes. Here, we review flow physics-informed learning, integrating seamlessly data and mathematical models, and implementing them using physics-informed neural networks (PINNs). We demonstrate the effectiveness of PINNs for inverse problems related to three-dimensional wake flows, supersonic flows, and biomedical flows.

physics.flu-dyn↗