arXiv subjects

Yilang Liu

Publications and source records attributed to Yilang Liu.

14 recordsLinked to original sources

arXiv2026

Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient

We present the stochastic decoupled policy gradient (SDPG), a lightweight visual reinforcement learning (RL) method that trains diverse visuomotor control policies end-to-end within a few hours on a single NVIDIA RTX 4080 GPU. SDPG estimates policy gradients via random perturbations of trajectory rollouts, requiring orders of magnitude fewer batch-rendered environments and substantially reducing compute and memory overhead. On visual MuJoCo benchmarks, SDPG consistently outperforms baseline methods in training time, memory usage, and rewards. Finally, to support future research, we introduce a suite of realistic visual robotics benchmarks spanning dexterous manipulation, challenging locomotion, and demonstrate effective sim-to-real transfer on physical hardware.

cs.RO↗

arXiv2026

Asymptotically Optimal Ergodic Coverage on Generalized Motion Fields

Autonomous robotic exploration in remote and extreme environments allows scientists to model complex transport phenomena and collective behaviors described by continuously deforming flow fields. Although these environments are naturally modeled as time-varying domains, most adaptive exploration methods assume static environments and fail to provide adequate coverage or satisfy any formal guarantees. This is especially the case in oceanography where autonomous underwater systems (UxS) have highly restrictive compute and payload requirements that necessitate path planning methods that yield robust data collection strategies in open-loop and underactuated settings. In this work, to address the aforementioned issues, we propose to formulate adaptive search as an ergodic coverage problem and investigate certifying coverage in the ergodic sense over evolving domains with flow-induced dynamics. We expand upon recent work demonstrating maximum mean discrepancy (MMD) as a functional ergodic metric, and derive a flow-adaptive formulation that explicitly accounts for domain evolution within the coverage objective. We show that this approach preserves ergodic coverage guarantees in ambient flows and enables effective exploration in under-actuated, and even open-loop planning settings by integrating environment dynamics. Experiments validate that our method generalizes to diverse spatiotemporal processes including ocean exploration, and tracking human and cattle movement. Physical experiments on aerial and legged robotic platforms validate our ability to obtain ergodic coverage in non-convex, flow-restricted environments while respecting robot dynamics.

cs.RO↗

arXiv2025

Sample-Based Hybrid Mode Control: Asymptotically Optimal Switching of Algorithmic and Non-Differentiable Control Modes

This paper investigates a sample-based solution to the hybrid mode control problem across non-differentiable and algorithmic hybrid modes. Our approach reasons about a set of hybrid control modes as an integer-based optimization problem where we select what mode to apply, when to switch to another mode, and the duration for which we are in a given control mode. A sample-based variation is derived to efficiently search the integer domain for optimal solutions. We find our formulation yields strong performance guarantees that can be applied to a number of robotics-related tasks. In addition, our approach is able to synthesize complex algorithms and policies to compound behaviors and achieve challenging tasks. Last, we demonstrate the effectiveness of our approach in real-world robotic examples that require reactive switching between long-term planning and high-frequency control.

cs.RO↗

arXiv2025

A novel parallelizable convergence accelerating method: Pointwise Frequency Damping

This paper proposes a novel class of data-driven acceleration methods for steady-state flow field solvers. The core innovation lies in predicting and assigning the asymptotic limit value for each parameter during iterations based on its own historical data, rather than processing and assigning the entire flow field at once. This approach fundamentally guarantees identical results between serial and parallel computations. Subsequently, a formula for representing the asymptotic limit based on historical data is derived and discretized, yielding a purely algebraic expression.Furthermore, the applicability scope of the method is discussed, along with the underlying reasons for its acceleration capability. A quantitative expression for estimating the speedup ratio is also provided. Extensive validation cases were tested, ranging from the simplest inviscid airfoil flow to complex three-dimensional viscous transonic cruise flow around a aircraft, and solving asymmetric linear systems via GMRES. These tests consistently demonstrate significant acceleration effects with speedup factors ranging from 2.5 to 4. Combined with the near-zero computational overhead of the purely algebraic formulation during the solving process and the inherently parallel-compatible pointwise prediction principle, the results strongly indicate that this method is highly suitable for large-scale industrial mesh computations.

physics.flu-dyn↗

arXiv2025

Accelerating Visual-Policy Learning through Parallel Differentiable Simulation

In this work, we propose a computationally efficient algorithm for visual policy learning that leverages differentiable simulation and first-order analytical policy gradients. Our approach decouple the rendering process from the computation graph, enabling seamless integration with existing differentiable simulation ecosystems without the need for specialized differentiable rendering software. This decoupling not only reduces computational and memory overhead but also effectively attenuates the policy gradient norm, leading to more stable and smoother optimization. We evaluate our method on standard visual control benchmarks using modern GPU-accelerated simulation. Experiments show that our approach significantly reduces wall-clock training time and consistently outperforms all baseline methods in terms of final returns. Notably, on complex tasks such as humanoid locomotion, our method achieves a $4\times$ improvement in final return, and successfully learns a humanoid running policy within 4 hours on a single GPU.

cs.LG↗

arXiv2025

A data-driven convergence booster for accelerating and stabilizing pseudo time-stepping

This paper introduces a novel data-driven convergence booster that not only accelerates convergence but also stabilizes solutions in cases where obtaining a steady-state solution is otherwise challenging. The method constructs a reduced-order model (ROM) of the solution residual using intermediate solutions and periodically solves a least-square problem in the low-dimensional ROM subspace. The second-order approximation of the residual and the use of normal equation distinguish this work from similar approaches in the literature from the methodology perspective. From the application perspective, in contrast to prior studies that focus on linear systems or idealized problems, we rigorously assess the method's performance on realistic computational fluid dynamics (CFD) applications. In addition to reducing the time complexity of point-iterative solvers for linear systems, we demonstrate substantial reductions in the number of pseudo-time steps required for implicit schemes solving the nonlinear Navier-Stokes equations. Across a range of two- and three-dimensional flows-including subsonic inviscid and transonic turbulent cases-the method consistently achieves a 3 to 4 times speedup in wall-clock time. Lastly, the proposed method acts as a robust stabilizer, capable of converging to steady solutions in flows that would otherwise exhibit persistent unsteadiness-such as vortex shedding or transonic buffet-without relying on symmetry boundary conditions.

physics.flu-dyn↗

arXiv2025

Towards a Generalized SA Model: Symbolic Regression-Based Correction for Separated Flows

This study focuses on the numerical simulation of high Reynolds number separated flows and proposes a data-driven approach to improve the predictive capability of the SA turbulence model. First, data assimilation was performed on two typical airfoils with high angle-of-attack separated flows to obtain a high-fidelity flow field dataset. Based on this dataset, a white-box model was developed using symbolic regression to modify the production term of the SA model. To validate the effectiveness of the modified model, multiple representative airfoils and wings, such as the SC1095 airfoil, DU91-W2-250 airfoil, and ONERA-M6 wing, were selected as test cases. A wide range of flow conditions was considered, including subsonic to transonic regimes, Reynolds numbers ranging from hundreds of thousands to tens of millions, and angles of attack varying from small to large. The results indicate that the modified model significantly improves the prediction accuracy of separated flows while maintaining the predictive capability for attached flows. It notably enhances the reproduction of separated vortex structures and flow separation locations, reducing the mean relative error in lift prediction at stall angles by 69.2% and improving computational accuracy by more than three times. Furthermore, validation using a zero-pressure-gradient flat plate case confirms the modified model's ability to accurately predict the turbulent boundary layer velocity profile and skin friction coefficient distribution. The findings of this study provide new insights and methodologies for the numerical simulation of high Reynolds number separated flows, contributing to more accurate modeling of complex flow phenomena in engineering applications.

physics.flu-dyn↗

arXiv2022

A novel convergence enhancement method based on Online Dimension Reduction Optimization

Iterative steady-state solvers are widely used in computational fluid dynamics. Unfortunately, it is difficult to obtain steady-state solution for unstable problem caused by physical instability and numerical instability. Optimization is a better choice for solving unstable problem because steady-state solution is always the extreme point of optimization regardless of whether the problem is unstable or ill-conditioned, but it is difficult to solve partial differential equations (PDEs) due to too many optimization variables. In this study, we propose an Online Dimension Reduction Optimization (ODRO) method to enhance the convergence of the traditional iterative method to obtain the steady-state solution of unstable problem. This method performs proper orthogonal decomposition (POD) on the snapshots collected from a few iteration steps, optimizes PDE residual in the POD subspace to get a solution with lower residual, and then continues to iterate with the optimized solution as the initial value, repeating the above three steps until the residual converges. Several typical cases show that the proposed method can efficiently calculate the steady-state solution of unstable problem with both the high efficiency and robustness of the iterative method and the good convergence of the optimization method. In addition, this method is easy to implement in almost any iterative solver with minimal code modification.

cs.CE↗

arXiv2021

High Reynolds number airfoil turbulence modeling method based on machine learning technique

In this paper, a turbulence model based on deep neural network is developed for turbulent flow around airfoil at high Reynolds numbers. According to the data got from the Spalart-Allmaras (SA) turbulence model, we build a neural network model that maps flow features to eddy viscosity. The model is then used to replace the SA turbulence model to mutually couple with the CFD solver. We build this suitable data-driven turbulence model mainly from the inputs, outputs features and loss function of the model. A feature selection method based on feature importance is also implemented. The results show that this feature selection method can effectively remove redundant features. The model based on the new input features has better accuracy and stability in mutual coupling with the CFD solver. The force coefficient obtained from solution can match the sample data well. The developed model also shows strong generalization at different inflow condition (angle of attack, Mach number, Reynolds number and airfoil).

physics.flu-dyn↗

arXiv2021

Analysis on numerical stability and convergence of RANS turbulence models from the perspective of coupling modes

Reynolds-averaged Navier-Stokes simulations are still the main method to study complex flows in engineering. However, traditional turbulence models cannot accurately predict flow fields with separations. In such situation, machine learning methods provide an effective way to build new data-driven turbulence closure models. Nevertheless, a bottleneck that the data-driven turbulence models encounter is how to ensure the stability and convergence of the RANS equations in posterior iterations. This paper studies the effects of different coupling modes on the convergence and stability between the RANS equations and turbulence models. Numerical results demonstrate that the frozen coupling mode, commonly used in machine learning turbulence models, may lead to divergence and instability in posterior iterations; while the mutual coupling mode can maintain good convergence and stability in the process of iterations. This research can provide a new perspective to the coupling mode for machine learning turbulence models with RANS equations in posterior iterations.

physics.flu-dyn↗

arXiv2021

An Energy-Saving Snake Locomotion Gait Policy Obtained Using Deep Reinforcement Learning

Snake robots, comprised of sequentially connected joint actuators, have recently gained increasing attention in the industrial field, like life detection in narrow space. Such robots can navigate through the complex environment via the cooperation of multiple motors located on the backbone. However, controlling the robots in an unknown environment is challenging, and conventional control strategies can be energy inefficient or even fail to navigate to the destination. In this work, a snake locomotion gait policy is developed via deep reinforcement learning (DRL) for energy-efficient control. We apply proximal policy optimization (PPO) to each joint motor parameterized by angular velocity and the DRL agent learns the standard serpenoid curve at each timestep. The robot simulator and task environment are built upon PyBullet. Comparing to conventional control strategies, the snake robots controlled by the trained PPO agent can achieve faster movement and more energy-efficient locomotion gait. This work demonstrates that DRL provides an energy-efficient solution for robot control.

cs.LG↗

arXiv2020

A new data assimilation method of recovering turbulent flow field at high-Reynolds numbers for turbulence machine learning

This paper proposes a new data assimilation method for recovering high fidelity turbulent flow field around airfoil at high Reynolds numbers based on experimental data, which is called Proper Orthogonal Decomposition Inversion (POD-Inversion) data assimilation method. Aiming at the flows including shock wave discontinuities or separated flows at high angle of attack, the proposed method can reconstruct high-fidelity turbulent flow field combining with experimental distributed force coefficients. We firstly perform the POD analysis to the turbulent eddy viscosity fields computed by SA model and obtain the base POD modes. Then optimized the POD coefficients by global optimization algorithm coupling with the Navier-Stokes equations solver. The high-fidelity turbulent flied are recovered by several main modes, which can dramatically reduce the dimensions of the system. The effectiveness of the method is verified by the cases of transonic flow around the RAE2822 airfoil at high Reynolds numbers and the separated flow at high angles of attack. The results demonstrate that the proposed assimilation method can recover the turbulent flow field which optimally match the experimental data, and significantly reduce the error of pressure coefficients. The proposed data assimilation method can offer high-fidelity field data for turbulent model based on machine learning.

physics.flu-dyn↗

arXiv2018

Machine learning methods for turbulence modeling in subsonic flows over airfoils

Reynolds-Averaged Navier-Stokes(RANS) method will still play a vital role in the following several decade in aerospace engineering. Although RANS models are widely used, empiricism and large discrepancies between models reduce the reliability of simulating complex flows. Therefore, in recent years, data-driven turbulence model has aroused widespread concern in fluid mechanics. Based on the experimental/numerical simulation results, this approach aims to modify or construct the turbulence model for specific purposes by machine learning technologies. In this paper, we take the results calculated by SA model as training data. Different from low Reynolds number turbulent flows, the data from high Reynolds number flows shows an apparent scaling effect, thus leading to difficulties in the data-driven modeling. In order to improve the fitting accuracy, we divided the flow field into near-wall region, wake region, and far-field region, and built individual model for every region. In this paper, we adopted the radial basis function neural network (RBFNN) and some auxiliary optimization algorithms to reconstruct a mapping function between mean variables and the eddy viscosity. Since this model reflects the relationship between local flow characteristics and turbulent eddy viscosity, it is independent on the airfoil shape and flow condition. The training data in this paper is generated from only three subsonic flow calculations of NACA0012 airfoil. By coupling the proposed approach with N-S equations, we calculated various flow cases as well as two different airfoils and showed the eddy viscosity contours, velocity profiles along the normal direction of wall and skin friction coefficient distributions, etc. Compared with the SA model, the results show a reasonable accuracy and better efficiency, which indicates the positive prospect of data-driven methods in turbulence modeling.

physics.flu-dyn↗

arXiv2018

Mode Multigrid - A novel convergence acceleration method

This paper proposes a mode multigrid (MMG) method, and applies it to accelerate the convergence of the steady state flow on unstructured grids. The dynamic mode decomposition (DMD) technique is used to analyze the convergence process of steady flow field according to the solution vectors from the previous time steps. Unlike the traditional multigrid method, we project the flowfield solutions from the physical space into the modal space, and truncate all the high-frequency modes but only the first-order mode are retained based on the DMD analysis. The real solutions in the physical space can be obtained simply by the inverse transformation from the modal space. The developed MMG method ingeniously avoids the complicated process of coarsening computational mesh, and does not need to make any change for the grid in physical space. Therefore, it is very convenient to be applied to any numerical schemes with just little change for the flow solver, which is also suitable for unstructured grids and easy for parallel computing. Several typical test cases have been used to verify the effectiveness of the proposed method, which demonstrates that the MMG can dramatically reduce the number of iterative steps for the different mesh types, different accuracy of spatial discretization and different time-marching schemes. The method is 3 to 6 times faster than the original method while ensuring the computational accuracy.

physics.comp-ph↗