arXiv subjects

Yufei Zhang

Publications and source records attributed to Yufei Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2026

Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State

The theory of continuous-time reinforcement learning (RL) has progressed rapidly in recent years. While the ultimate objective of RL is typically to learn deterministic control policies, most existing continuous-time RL methods rely on stochastic policies. Such approaches often require sampling actions at very high frequencies, and involve computationally expensive expectations over continuous action spaces, resulting in high-variance gradient estimates and slow convergence. In this paper, we introduce and develop deterministic policy gradient (DPG) methods for continuous-time RL. We derive a continuous-time policy gradient formula expressed as the expected gradient of an advantage rate function and establish a martingale characterization for both the value function and the advantage rate. These theoretical results provide tractable estimators for deterministic policy gradients in continuous-time RL. Building on this foundation, we propose a model-free continuous-time Deep Deterministic Policy Gradient (CT-DDPG) algorithm that enables stable learning for general reinforcement learning problems with continuous time-and-state. Numerical experiments show that CT-DDPG achieves superior stability and faster convergence compared to existing stochastic-policy methods, across a wide range of learning tasks with varying time discretizations and noise levels.

cs.LG↗

arXiv2026

Bayesian Network Structure Discovery Using Large Language Models

Understanding probabilistic dependencies among variables is central to analyzing complex systems. Traditional structure learning methods often require extensive observational data or are limited by manual, error-prone incorporation of expert knowledge. Recent studies have explored using large language models (LLMs) for structure learning, but most treat LLMs as auxiliary tools for pre-processing or post-processing, leaving the core learning process data-driven. In this work, we introduce a unified framework for Bayesian network structure discovery that places LLMs at the center, supporting both data-free and data-aware settings. In the data-free regime, we introduce \textbf{PromptBN}, which leverages LLM reasoning over variable metadata to generate a complete directed acyclic graph (DAG) in a single call. PromptBN effectively enforces global consistency and acyclicity through dual validation, achieving constant $\mathcal{O}(1)$ query complexity. When observational data are available, we introduce \textbf{ReActBN} to further refine the initial graph. ReActBN combines statistical evidence with LLM by integrating a novel ReAct-style reasoning with configurable structure scores (e.g., Bayesian Information Criterion). Experiments demonstrate that our method outperforms prior data-only, LLM-only, and hybrid baselines, particularly in low- or no-data regimes and on out-of-distribution datasets. Code is available at https://github.com/sherryzyh/llmbn.

cs.LG↗

arXiv2026

Learning Distributed Equilibria in Linear-Quadratic Stochastic Differential Games: An $α$-Potential Approach

We analyze independent policy-gradient (PG) learning in $N$-player linear-quadratic (LQ) stochastic differential games. Each player employs a distributed policy that depends only on its own state and updates the policy independently using the gradient of its own objective. We establish global linear convergence of these methods to an equilibrium by showing that the LQ game admits an $α$-potential structure, with $α$ determined by the degree of pairwise interaction asymmetry. For pairwise-symmetric interactions, we construct an affine distributed equilibrium by minimizing the potential function and show that independent PG methods converge globally to this equilibrium, with complexity scaling linearly in the population size and logarithmically in the desired accuracy. For asymmetric interactions, we prove that independent projected PG algorithms converge linearly to an approximate equilibrium, with suboptimality proportional to the degree of asymmetry. Numerical experiments confirm the theoretical results across both symmetric and asymmetric interaction networks.

math.OC↗

arXiv2026

Digital exclusion among middle-aged and older adults in China: age-period-cohort evidence from three national surveys, 2011-2022

Amid China's ageing and digital shift, digital exclusion among older adults poses an urgent challenge. To unpack this phenomenon, this study disentangles age, period, and cohort effects on digital exclusion among middle-aged and older Chinese adults. Using three nationally representative surveys (CHARLS 2011-2020, CFPS 2010-2022, and CGSS 2010-2021), we fitted hierarchical age-period-cohort (HAPC) models weighted by cross-sectional survey weights and stabilized inverse probability weights for item response. We further assessed heterogeneity by urban-rural residence, region, multimorbidity, and cognitive risk, and evaluated robustness with APC bounding analyses. Across datasets, digital exclusion increased with age and displayed mild non-linearity, with a small midlife easing followed by a sharper rise at older ages. Period effects declined over the 2010s and early 2020s, although the pace of improvement differed across survey windows. Cohort deviations were present but less consistent than age and period patterns, with an additional excess risk concentrated among cohorts born in the 1950s. Rural and western residents, as well as adults with multimorbidity or cognitive risk, remained consistently more excluded. Over the study period, the urban-rural divide showed no evidence of narrowing, whereas the cognitive-risk gap widened. These findings highlight digital inclusion as a vital pathway for older adults to remain integral participants in an evolving digital society.

stat.AP↗

arXiv2026

Uncertainty-Aware Data-Based Method for Fast and Reliable Shape Optimization

Data-based optimization (DBO) offers a promising approach for efficiently optimizing shape for better aerodynamic performance by leveraging a pretrained surrogate model for offline evaluations during iterations. However, DBO heavily relies on the quality of the training database. Samples outside the training distribution encountered during optimization can lead to significant prediction errors, potentially misleading the optimization process. Therefore, incorporating uncertainty quantification into optimization is critical for detecting outliers and enhancing robustness. This study proposes an uncertainty-aware data-based optimization (UA-DBO) framework to monitor and minimize surrogate model uncertainty during DBO. A probabilistic encoder-decoder surrogate model is developed to predict uncertainties associated with its outputs, and these uncertainties are integrated into a model-confidence-aware objective function to penalize samples with large prediction errors during data-based optimization process. The UA-DBO framework is evaluated on two multipoint optimization problems aimed at improving airfoil drag divergence and buffet performance. Results demonstrate that UA-DBO consistently reduces prediction errors in optimized samples and achieves superior performance gains compared to original DBO. Moreover, compared to multipoint optimization based on full computational simulations, UA-DBO offers comparable optimization effectiveness while significantly accelerating optimization speed.

cs.LG↗

arXiv2025

Beyond Motion Pattern: An Empirical Study of Physical Forces for Human Motion Understanding

Human motion understanding has advanced rapidly through vision-based progress in recognition, tracking, and captioning. However, most existing methods overlook physical cues such as joint actuation forces that are fundamental in biomechanics. This gap motivates our study: if and when do physically inferred forces enhance motion understanding? By incorporating forces into established motion understanding pipelines, we systematically evaluate their impact across baseline models on 3 major tasks: gait recognition, action recognition, and fine-grained video captioning. Across 8 benchmarks, incorporating forces yields consistent performance gains; for example, on CASIA-B, Rank-1 gait recognition accuracy improved from 89.52% to 90.39% (+0.87), with larger gain observed under challenging conditions: +2.7% when wearing a coat and +3.0% at the side view. On Gait3D, performance also increases from 46.0% to 47.3% (+1.3). In action recognition, CTR-GCN achieved +2.00% on Penn Action, while high-exertion classes like punching/slapping improved by +6.96%. Even in video captioning, Qwen2.5-VL's ROUGE-L score rose from 0.310 to 0.339 (+0.029), indicating that physics-inferred forces enhance temporal grounding and semantic richness. These results demonstrate that force cues can substantially complement visual and kinematic features under dynamic, occluded, or appearance-varying conditions.

cs.CV↗

arXiv2025

Benchmark for Non-reciprocal Attack Detection on Synchronization

A precise and secure time synchronization is the backbone of both fundamental physics and advanced technologies. Despite ultra-high precision, security, particularly the unresolved vulnerabilities on physical links beyond traditional cryptography, remains the bottleneck. Here, we develop an analysis framework, the Benchmark Attack Noise Detection (BAND) model, to quantify the security of the high-precision time synchronization when its core assumption, reciprocity, is broken. The model characterizes non-reciprocal attacks through a unified metric, termed attack intensity. Based on the analysis of attack intensity, we accurately predict both instantaneous and accumulating attacks in a 100-km dual-comb two-way time-transfer system. Beyond security considerations, we find that the BAND model can explain intrinsic fiber noise and delay-unsuppressed noise naturally, offering an effective tool to establish an environment-sensitive link model. This work paves the way for a secure large-scale integrated time, sensing and communication networks.

physics.optics↗

arXiv2025

Entropic Optimal Transport Problem with Convex Functional Cost

We study an entropic optimal transport problem in which the transport plan is penalized by a nonlinear convex functional acting on the coupling. We establish existence, uniqueness, and uniform a priori bounds for minimizers, and we show that each minimizer satisfies a fixed-point first-order optimality system associated with an exponentially tilted reference measure. Building on this variational structure, we introduce the Sinkhorn-Frank-Wolfe (SFW) flow, prove its global well-posedness, and derive an energy-dissipation inequality yielding exponential convergence toward the unique optimal transport plan. As an application, we implement the SFW algorithm to solve an optimal routing problem for unmanned aerial vehicles with congestion aversion.

math.OC↗

arXiv2025

$α$-Potential Games for Decentralized Control of Connected and Automated Vehicles

Designing scalable and safe control strategies for large populations of connected and automated vehicles (CAVs) requires accounting for strategic interactions among heterogeneous agents under decentralized information. While dynamic games provide a natural modeling framework, computing Nash equilibria (NEs) in large-scale settings remains challenging, and existing mean-field game approximations rely on restrictive assumptions that fail to capture collision avoidance and heterogeneous behaviors. This paper proposes an $α$-potential game framework for decentralized CAV control. We show that computing $α$-NE reduces to solving a decentralized control problem, and derive tight bounds of the parameter $α$ based on interaction intensity and asymmetry. We further develop scalable policy gradient algorithms for computing $α$-NEs using decentralized neural-network policies. Numerical experiments demonstrate that the proposed framework accommodates diverse traffic flow models and effectively captures collision avoidance, obstacle avoidance, and agent heterogeneity.

math.OC↗

arXiv2025

Design, Results and Industry Implications of the World's First Insurance Large Language Model Evaluation Benchmark

This paper comprehensively elaborates on the construction methodology, multi-dimensional evaluation system, and underlying design philosophy of CUFEInse v1.0. Adhering to the principles of "quantitative-oriented, expert-driven, and multi-validation," the benchmark establishes an evaluation framework covering 5 core dimensions, 54 sub-indicators, and 14,430 high-quality questions, encompassing insurance theoretical knowledge, industry understanding, safety and compliance, intelligent agent application, and logical rigor. Based on this benchmark, a comprehensive evaluation was conducted on 11 mainstream large language models. The evaluation results reveal that general-purpose models suffer from common bottlenecks such as weak actuarial capabilities and inadequate compliance adaptation. High-quality domain-specific training demonstrates significant advantages in insurance vertical scenarios but exhibits shortcomings in business adaptation and compliance. The evaluation also accurately identifies the common bottlenecks of current large models in professional scenarios such as insurance actuarial, underwriting and claim settlement reasoning, and compliant marketing copywriting. The establishment of CUFEInse not only fills the gap in professional evaluation benchmarks for the insurance field, providing academia and industry with a professional, systematic, and authoritative evaluation tool, but also its construction concept and methodology offer important references for the evaluation paradigm of large models in vertical fields, serving as an authoritative reference for academic model optimization and industrial model selection. Finally, the paper looks forward to the future iteration direction of the evaluation benchmark and the core development direction of "domain adaptation + reasoning enhancement" for insurance large models.

cs.CL↗

arXiv2025

FIRE: Robust Detection of Diffusion-Generated Images via Frequency-Guided Reconstruction Error

The rapid advancement of diffusion models has significantly improved high-quality image generation, making generated content increasingly challenging to distinguish from real images and raising concerns about potential misuse. In this paper, we observe that diffusion models struggle to accurately reconstruct mid-band frequency information in real images, suggesting the limitation could serve as a cue for detecting diffusion model generated images. Motivated by this observation, we propose a novel method called Frequency-guided Reconstruction Error (FIRE), which, to the best of our knowledge, is the first to investigate the influence of frequency decomposition on reconstruction error. FIRE assesses the variation in reconstruction error before and after the frequency decomposition, offering a robust method for identifying diffusion model generated images. Extensive experiments show that FIRE generalizes effectively to unseen diffusion models and maintains robustness against diverse perturbations.

cs.CV↗

arXiv2025

OneRec-V2 Technical Report

Recent breakthroughs in generative AI have transformed recommender systems through end-to-end generation. OneRec reformulates recommendation as an autoregressive generation task, achieving high Model FLOPs Utilization. While OneRec-V1 has shown significant empirical success in real-world deployment, two critical challenges hinder its scalability and performance: (1) inefficient computational allocation where 97.66% of resources are consumed by sequence encoding rather than generation, and (2) limitations in reinforcement learning relying solely on reward models. To address these challenges, we propose OneRec-V2, featuring: (1) Lazy Decoder-Only Architecture: Eliminates encoder bottlenecks, reducing total computation by 94% and training resources by 90%, enabling successful scaling to 8B parameters. (2) Preference Alignment with Real-World User Interactions: Incorporates Duration-Aware Reward Shaping and Adaptive Ratio Clipping to better align with user preferences using real-world feedback. Extensive A/B tests on Kuaishou demonstrate OneRec-V2's effectiveness, improving App Stay Time by 0.467%/0.741% while balancing multi-objective recommendations. This work advances generative recommendation scalability and alignment with real-world feedback, representing a step forward in the development of end-to-end recommender systems.

cs.IR↗

arXiv2025

ViPER: Empowering the Self-Evolution of Visual Perception Abilities in Vision-Language Model

The limited capacity for fine-grained visual perception presents a critical bottleneck for Vision-Language Models (VLMs) in real-world applications. Addressing this is challenging due to the scarcity of high-quality data and the limitations of existing methods: supervised fine-tuning (SFT) often compromises general capabilities, while reinforcement fine-tuning (RFT) prioritizes textual reasoning over visual perception. To bridge this gap, we propose a novel two-stage task that structures visual perception learning as a coarse-to-fine progressive process. Based on this task formulation, we develop ViPER, a self-bootstrapping framework specifically designed to enable iterative evolution through self-critiquing and self-prediction. By synergistically integrating image-level and instance-level reconstruction with a two-stage reinforcement learning strategy, ViPER establishes a closed-loop training paradigm, where internally synthesized data directly fuel the enhancement of perceptual ability. Applied to the Qwen2.5-VL family, ViPER produces the Qwen-Viper series. With an average gain of 1.7% on seven comprehensive benchmarks spanning various tasks and up to 6.0% on fine-grained perception, Qwen-Viper consistently demonstrates superior performance across different vision-language scenarios while maintaining generalizability. Beyond enabling self-improvement in perceptual capabilities, ViPER provides concrete evidence for the reciprocal relationship between generation and understanding, a breakthrough to developing more autonomous and capable VLMs.

cs.CV↗

arXiv2025

Numerical Investigation of Discontinuous Ice Effects on Swept Wings

This study investigates the aerodynamic performance and flow structures of infinite swept wings with artificially simulated discontinuous ice using an enhanced delayed detached-eddy simulation. Comparisons are made among clean, continuous-ice, and discontinuous-ice configurations. Results show that discontinuous ice causes a more severe reduction in lift than continuous ice. While continuous ice forms a large separation bubble that helps maintain lift, discontinuous ice disrupts leading-edge vortex formation through gap jets, resulting in greater lift loss but a smaller drag penalty. Unlike the continuous-ice wing, the discontinuous-ice case does not exhibit a sudden stall-induced lift drop. The flow over the discontinuous-ice wing can be characterized by two canonical patterns: a separating shear layer and Kármán vortex shedding. However, the separating shear layer becomes irregular due to the interference of gap jets. Three characteristic chord-based Strouhal numbers (St)-11.3, 22.6, and 33.9-are identified. The lowest (St=11.3) corresponds to the shedding of vortex pairs; when nondimensionalized by the ice width, it yields St = 0.58, which is higher than that of a canonical cylinder wake. Furthermore, lift and drag fluctuations occur predominantly at St = 22.6, twice the shedding frequency, primarily induced by the gap jets-a phenomenon absent in the continuous-ice case.

physics.flu-dyn↗

arXiv2025

Accuracy of Discretely Sampled Stochastic Policies in Continuous-time Reinforcement Learning

Stochastic policies (also known as relaxed controls) are widely used in continuous-time reinforcement learning algorithms. However, executing a stochastic policy and evaluating its performance in a continuous-time environment remain open challenges. This work introduces and rigorously analyzes a policy execution framework that samples actions from a stochastic policy at discrete time points and implements them as piecewise constant controls. We prove that as the sampling mesh size tends to zero, the controlled state process converges weakly to the dynamics with coefficients aggregated according to the stochastic policy. We explicitly quantify the convergence rate based on the regularity of the coefficients and establish an optimal first-order convergence rate for sufficiently regular coefficients. Additionally, we prove a $1/2$-order weak convergence rate that holds uniformly over the sampling noise with high probability, and establish a $1/2$-order pathwise convergence for each realization of the system noise in the absence of volatility control. Building on these results, we analyze the bias and variance of various policy evaluation and policy gradient estimators based on discrete-time observations. Our results provide theoretical justification for the exploratory stochastic control framework in [H. Wang, T. Zariphopoulou, and X.Y. Zhou, J. Mach. Learn. Res., 21 (2020), pp. 1-34].

cs.LG↗

arXiv2025

Beyond Static Testbeds: An Interaction-Centric Agent Simulation Platform for Dynamic Recommender Systems

Evaluating and iterating upon recommender systems is crucial, yet traditional A/B testing is resource-intensive, and offline methods struggle with dynamic user-platform interactions. While agent-based simulation is promising, existing platforms often lack a mechanism for user actions to dynamically reshape the environment. To bridge this gap, we introduce RecInter, a novel agent-based simulation platform for recommender systems featuring a robust interaction mechanism. In RecInter platform, simulated user actions (e.g., likes, reviews, purchases) dynamically update item attributes in real-time, and introduced Merchant Agents can reply, fostering a more realistic and evolving ecosystem. High-fidelity simulation is ensured through Multidimensional User Profiling module, Advanced Agent Architecture, and LLM fine-tuned on Chain-of-Thought (CoT) enriched interaction data. Our platform achieves significantly improved simulation credibility and successfully replicates emergent phenomena like Brand Loyalty and the Matthew Effect. Experiments demonstrate that this interaction mechanism is pivotal for simulating realistic system evolution, establishing our platform as a credible testbed for recommender systems research. Our codes are available at https://github.com/jinsong8/RecInter.

cs.CL↗

arXiv2025

OneRec Technical Report

Recommender systems have been widely used in various large-scale user-oriented platforms for many years. However, compared to the rapid developments in the AI community, recommendation systems have not achieved a breakthrough in recent years. For instance, they still rely on a multi-stage cascaded architecture rather than an end-to-end approach, leading to computational fragmentation and optimization inconsistencies, and hindering the effective application of key breakthrough technologies from the AI community in recommendation scenarios. To address these issues, we propose OneRec, which reshapes the recommendation system through an end-to-end generative approach and achieves promising results. Firstly, we have enhanced the computational FLOPs of the current recommendation model by 10 $\times$ and have identified the scaling laws for recommendations within certain boundaries. Secondly, reinforcement learning techniques, previously difficult to apply for optimizing recommendations, show significant potential in this framework. Lastly, through infrastructure optimizations, we have achieved 23.7% and 28.8% Model FLOPs Utilization (MFU) on flagship GPUs during training and inference, respectively, aligning closely with the LLM community. This architecture significantly reduces communication and storage overhead, resulting in operating expense that is only 10.6% of traditional recommendation pipelines. Deployed in Kuaishou/Kuaishou Lite APP, it handles 25% of total queries per second, enhancing overall App Stay Time by 0.54% and 1.24%, respectively. Additionally, we have observed significant increases in metrics such as 7-day Lifetime, which is a crucial indicator of recommendation experience. We also provide practical lessons and insights derived from developing, optimizing, and maintaining a production-scale recommendation system with significant real-world impact.

cs.IR↗

arXiv2025

Convergence Rates of Time Discretization in Extended Mean Field Control

Piecewise constant control approximation provides a practical framework for designing numerical schemes of continuous-time control problems. We analyze the accuracy of such approximations for extended mean field control (MFC) problems, where the dynamics and costs depend on the joint distribution of states and controls. For linear-convex extended MFC problems, we show that the optimal control is $1/2$-Hölder continuous in time. Using this regularity, we prove that the optimal cost of the continuous-time problem can be approximated by piecewise constant controls with order $1/2$, while the optimal control itself can be approximated with order $1/4$. For general extended MFC problems, we further show that, under sufficient regularity of the value functions, the value functions converge with an improved first-order rate, matching the best-known rate for classical control problems without mean field interaction, and consistent with the numerical observations for MFC of Cucker-Smale models.

math.OC↗