arXiv subjects

Sijie Huang

Publications and source records attributed to Sijie Huang.

3 recordsLinked to original sources

arXiv2026

Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents

Reinforcement learning from verifiable rewards (RLVR) is a promising paradigm for improving large language model (LLM) agents on long-horizon interactive tasks. However, in partially observable environments, incomplete observations cause agent beliefs to drift over time, while delayed rewards obscure the causal impact of intermediate decisions, exacerbating temporal credit assignment challenges. To address this, we propose ReBel (Reward Belief), a process-level reinforcement learning algorithm that explicitly models structured belief states to summarize interaction history and guide subsequent policy learning. ReBel introduces belief-consistency supervision, converting discrepancies between predicted beliefs and observed feedback into dense self-supervised signals without requiring external step-wise annotations or verifiers. It also employs belief-aware grouping to compare trajectories under similar belief states, yielding more robust and lower-variance advantage estimates. We evaluate ReBel on challenging long-horizon benchmarks, including ALFWorld and WebShop. ReBel improves task success by up to $20.4$ percentage points over the episode-level baseline GRPO and increases sample efficiency by $2.1\times$. These results suggest that belief-aware self-supervision is a promising direction for reliable long-horizon decision-making under partial observability. Code is available at: https://github.com/Fateyetian/Rebel.git.

cs.CL↗

arXiv2026

Revisiting the scale dependence of the Reynolds number in correlated fluctuating fluids

For the incompressible Navier--Stokes equation, the Reynolds number ($\mathrm{Re}$) is a dimensionless parameter quantifying the relative importance of inertial over viscous forces. In the low-$\mathrm{Re}$ regime ($\mathrm{Re} \ll 1$), the flow dynamics are commonly approximated by the linear Stokes equation. Here we show that, within the framework of spatially fluctuating hydrodynamics, this linearization breaks down when the thermal noise is spatially correlated, even if $\mathrm{Re} \ll 1$. We perform direct numerical simulations of spatially correlated fluctuating hydrodynamics in both one and two dimensions. In one dimension, the linearized dynamics exhibit significantly slower relaxation of high-wavenumber Fourier modes than the full nonlinear dynamics. In two dimensions, an analogous discrepancy arises in the particle velocity autocorrelation function, which decays more slowly in the correlated linear Stokes case than in the correlated nonlinear Navier--Stokes case. In both settings, spatial correlations inhibit viscous momentum diffusion at small scales, leading to prolonged relaxation under the linear dynamics, whereas nonlinear mode coupling accelerates small-scale relaxation. Thus, the interplay between nonlinear coupling and viscous damping becomes scale dependent, invalidating the use of a single global Reynolds number. Taken together, these findings show that, for spatially correlated fluctuating fluids, the effective Reynolds number must be reinterpreted as a scale-dependent quantity.

physics.flu-dyn↗

arXiv2025

Spatially Correlated Noise Induces Transitions from the Diffusive to Ballistic Regime in Fluids

We investigate the fluctuating incompressible Navier--Stokes equation driven by spatially correlated thermal noise characterized by a single length scale. This formulation is constructed to preserve thermal equilibrium through the fluctuation--dissipation relation (FDR), which enforces the same spatial correlation in the viscous diffusion term and therefore gives rise to nonlocal momentum transport. Numerical simulations of tracer diffusion in fluids governed by this formulation reveal that the mean-squared displacement (MSD) depends monotonically on the correlation length $\ell$ and the correlation strength $β$. Intuitively, increasing $\ell$ enhances MSD and induces the emergence of an early-time ballistic regime, as a larger correlation length slows momentum diffusion. Counterintuitively, decreasing $β$ also increases the MSD, since a weaker correlation strength also retards momentum diffusion, whereas smaller $\ell$ or larger $β$ suppresses the ballistic regime and leads to a diffusive behavior. The emergence or suppression of the ballistic regime stems from how spatial correlations, incorporated through the FDR to maintain equilibrium, alter the effective momentum transport across scales. Interestingly, we further show that the resulting nonlocal diffusion is reminiscent of the slow dynamics in glassy and other disordered systems.

physics.bio-ph↗