Reinforcement Learning-Based Filters for Convection-Dominated Flows: Reference-Free and Reference-Guided Training

Abstract

We propose a reinforcement learning (RL) framework for the dynamic selection of the filter parameter in Evolve-Filter (EF) regularization strategies for incompressible turbulent flows. Instead of prescribing the filter radius heuristically, the RL agent learns to adaptively control the filtering intensity in time, balancing numerical stability and physical accuracy. The methodology is assessed on two benchmark problems with fundamentally different dynamics: flow past a cylinder and decaying homogeneous turbulence. Both reference-guided and reference-free reward formulations are investigated. In the reference-guided setting, the agent is trained using direct numerical simulation (DNS) data over a limited time window and then evaluated in extrapolation. In the reference-free setting, the reward relies exclusively on physics-based quantities, without access to reference solutions, i.e., completely eliminating the computational costs related to DNS simulations. The results show that the proposed RL-EF strategies prevent numerical blow-up while avoiding the excessive dissipation typical of standard EF approaches based on a fixed Kolmogorov length scale. The learned policies accurately reproduce the relevant flow dynamics across scales, preserving the correct balance between large-scale structures and small-scale dissipation. Notably, the reference-free reward achieves performance comparable to the reference-guided approach, demonstrating that stable and spectrally consistent filtering strategies can be learned even without DNS data, drastically reducing the computational costs of the training phase. The proposed framework provides a robust and flexible alternative to manually tuned regularization parameters, enabling adaptive, physically consistent control of filtering in turbulent flow simulations.