arXiv subjects

Wenjie Ye

Publications and source records attributed to Wenjie Ye.

11 recordsLinked to original sources

arXiv2026

DerainSplat: Feed-Forward Clean 3D Gaussian Splatting from Sparse Rainy Views

Although image deraining has advanced substantially, existing methods mainly focus on 2D image restoration. As spatial intelligence applications such as embodied AI and autonomous driving continue to emerge, reconstructing clean 3D scenes from sparse rainy views in a feed-forward manner becomes increasingly important. Existing feed-forward 3D Gaussian Splatting (3DGS) methods often assume clean inputs and collapse under rainy conditions. To this end, we present \textbf{\textit{DerainSplat}}, a feed-forward framework that reconstructs clean 3D scenes from only a few rainy views. To support this task, we build a large-scale multi-view derain dataset through a four-stage synthesis pipeline that sequentially models overcast illumination, depth-dependent haze, rain streaks, and lens raindrops, producing privileged weather factors. We introduce a weather net that predicts the weather factors from rainy context and yields two support maps. Scene support modulates cross-view cost-volume matching, while radiance support drives depth-aligned appearance fusion to fill corrupted pixels. The derived geometry evidence further attenuates Gaussian opacity to reduce spurious structures. A rainy cycle consistency re-renders clean views using the predicted factors and aligns them with rainy inputs. Extensive experiments show that \textbf{\textit{DerainSplat}} outperforms existing methods on various datasets, including RealEstate10K, ACID, Mip-NeRF360, and real-world rainy scenes, with strong cross-dataset generalization.

cs.CV↗

arXiv2026

SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as distinct problems, leading to fragmented architectures, cascaded pipelines, and misaligned representation spaces. We argue that this divide is not merely an engineering artifact, but a structural limitation that hinders the emergence of native multimodal intelligence. Hence, we introduce SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify, in which understanding and generation evolve as synergistic views of a single underlying process. We launch two native unified variants, SenseNova-U1-8B-MoT and SenseNova-U1-A3B-MoT, built on dense (8B) and mixture-of-experts (30B-A3B) understanding baselines, respectively. Designed from first principles, they rival top-tier understanding-only VLMs across text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. Meanwhile, they deliver strong semantic consistency and visual fidelity, excelling in conventional or knowledge-intensive any-to-image (X2I) synthesis, complex text-rich infographic generation, and interleaved vision-language generation, with or without think patterns. Beyond performance, we show detailed model design, data preprocessing, pre-/post-training, and inference strategies to support community research. Last but not least, preliminary evidence demonstrates that our models extend beyond perception and generation, performing strongly in vision-language-action (VLA) and world model (WM) scenarios. This points toward a broader roadmap where models do not translate between modalities, but think and act across them in a native manner. Multimodal AI is no longer about connecting separate systems, but about building a unified one and trusting the necessary capabilities to emerge from within.

cs.CV↗

arXiv2025

Large $N$ limit of the Langevin dynamics for the spin $O(N)$ model

In this paper, we prove that the large $N$ limit of the Langevin dynamics for the spin $O(N)$ model is given by a mean-field stochastic differential equation (SDE) in both finite and infinite volumes. We establish uniform in $N$ bounds for the dynamics, which enable us to demonstrate convergence to the mean-field SDE with polynomial interactions. Furthermore, the mean-field SDE is shown to be globally well-posed for suitable initial distributions. We also prove the existence of stationary measures for the mean-field SDE. For small inverse temperatures, we characterize the large $N$ limit of the spin $O(N)$ model through stationary coupling. Additionally, we establish the uniqueness of the stationary measure for the mean-field SDE.

math.PR↗

arXiv2024

On Designing Effective RL Reward at Training Time for LLM Reasoning

Reward models have been increasingly critical for improving the reasoning capability of LLMs. Existing research has shown that a well-trained reward model can substantially improve model performances at inference time via search. However, the potential of reward models during RL training time still remains largely under-explored. It is currently unclear whether these reward models can provide additional training signals to enhance the reasoning capabilities of LLMs in RL training that uses sparse success rewards, which verify the correctness of solutions. In this work, we evaluate popular reward models for RL training, including the Outcome-supervised Reward Model (ORM) and the Process-supervised Reward Model (PRM), and train a collection of LLMs for math problems using RL by combining these learned rewards with success rewards. Surprisingly, even though these learned reward models have strong inference-time performances, they may NOT help or even hurt RL training, producing worse performances than LLMs trained with the success reward only. Our analysis reveals that an LLM can receive high rewards from some of these reward models by repeating correct but unnecessary reasoning steps, leading to a severe reward hacking issue. Therefore, we introduce two novel reward refinement techniques, including Clipping and Delta. The key idea is to ensure the accumulative reward of any reasoning trajectory is upper-bounded to keep a learned reward model effective without being exploited. We evaluate our techniques with multiple reward models over a set of 1.5B and 7B LLMs on MATH and GSM8K benchmarks and demonstrate that with a carefully designed reward function, RL training without any additional supervised tuning can improve all the evaluated LLMs, including the state-of-the-art 7B LLM Qwen2.5-Math-7B-Instruct on MATH and GSM8K benchmarks.

cs.LG↗

arXiv2024

Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study

Reinforcement Learning from Human Feedback (RLHF) is currently the most widely used method to align large language models (LLMs) with human preferences. Existing RLHF methods can be roughly categorized as either reward-based or reward-free. Novel applications such as ChatGPT and Claude leverage reward-based methods that first learn a reward model and apply actor-critic algorithms, such as Proximal Policy Optimization (PPO). However, in academic benchmarks, state-of-the-art results are often achieved via reward-free methods, such as Direct Preference Optimization (DPO). Is DPO truly superior to PPO? Why does PPO perform poorly on these benchmarks? In this paper, we first conduct both theoretical and empirical studies on the algorithmic properties of DPO and show that DPO may have fundamental limitations. Moreover, we also comprehensively examine PPO and reveal the key factors for the best performances of PPO in fine-tuning LLMs. Finally, we benchmark DPO and PPO across a collection of RLHF testbeds, ranging from dialogue to code generation. Experiment results demonstrate that PPO is able to surpass other alignment methods in all cases and achieve state-of-the-art results in challenging code competitions. Our code is publicly available at https://github.com/openpsi-project/ReaLHF.

cs.CL↗

arXiv2023

Towards Blockchain-Assisted Privacy-Aware Data Sharing For Edge Intelligence: A Smart Healthcare Perspective

The popularization of intelligent healthcare devices and big data analytics significantly boosts the development of smart healthcare networks (SHNs). To enhance the precision of diagnosis, different participants in SHNs share health data that contains sensitive information. Therefore, the data exchange process raises privacy concerns, especially when the integration of health data from multiple sources (linkage attack) results in further leakage. Linkage attack is a type of dominant attack in the privacy domain, which can leverage various data sources for private data mining. Furthermore, adversaries launch poisoning attacks to falsify the health data, which leads to misdiagnosing or even physical damage. To protect private health data, we propose a personalized differential privacy model based on the trust levels among users. The trust is evaluated by a defined community density, while the corresponding privacy protection level is mapped to controllable randomized noise constrained by differential privacy. To avoid linkage attacks in personalized differential privacy, we designed a noise correlation decoupling mechanism using a Markov stochastic process. In addition, we build the community model on a blockchain, which can mitigate the risk of poisoning attacks during differentially private data transmission over SHNs. To testify the effectiveness and superiority of the proposed approach, we conduct extensive experiments on benchmark datasets.

cs.CR↗

arXiv2023

What Happened 3 Seconds Ago? Inferring the Past with Thermal Imaging

Inferring past human motion from RGB images is challenging due to the inherent uncertainty of the prediction problem. Thermal images, on the other hand, encode traces of past human-object interactions left in the environment via thermal radiation measurement. Based on this observation, we collect the first RGB-Thermal dataset for human motion analysis, dubbed Thermal-IM. Then we develop a three-stage neural network model for accurate past human pose estimation. Comprehensive experiments show that thermal cues significantly reduce the ambiguities of this task, and the proposed model achieves remarkable performance. The dataset is available at https://github.com/ZitianTang/Thermal-IM.

cs.CV↗

arXiv2023

Forward-backward stochastic differential equations on tensor fields and application to Navier-Stokes equations on Riemannian manifolds

In this paper we introduce a class of forward-backward stochastic differential equations on tensor fields of Riemannian manifolds, which are related to semi-linear parabolic partial differential equations on tensor fields. Moreover, we will use these forward-backward stochastic differential equations to give a stochastic characterization of incompressible Navier-Stokes equations on Riemannian manifolds, where some extra conditions used in [22] are not required.

math.PR↗

arXiv2022

Stochastic Differential Equations with Local Growth Singular Drifts

In this paper, we study the weak differentiability of global strong solution of stochastic differential equations, the strong Feller property of the associated diffusion semigroups and the global stochastic flow property in which the singular drift $b$ and the weak gradient of Sobolev diffusion $σ$ are supposed to satisfy $||b(x){1}_{|x|\le R}||_{p_1}\le O((\log R)^{{(p_1-d)^2}/{2p^2_1}})$ and $||\nabla σ(x)1_{|x|\le R} ||_{p_1} \le O((\log ({R}/{3}))^{{(p_1-d)^2}/{2p^2_1}})$ respectively. The main tools for these results are the decomposition of global two-point motions, Krylov's estimate, Khasminskii's estimate, Zvonkin's transformation and the characterization for Sobolev differentiability of random fields.

math.PR↗

arXiv2021

A probabilistic representation for heat flow of harmonic map on manifolds with time-dependent Riemannian metric

In this paper we will give a probabilistic representation for the heat flow of harmonic map with time-dependent Riemannian metric via a forward-backward stochastic differential equation on manifolds. Moreover, we can provide an alternative stochastic method for the proof of existence of a unique local solution for heat flow of harmonic map with time-dependent Riemannian metric.

math.PR↗

arXiv2020

A study of backward stochastic differential equation on a Riemannian manifold

Suppose $N$ is a compact Riemannian manifold, in this paper we will introduce the definition of $N$-valued BSDE and $L^2(\mathbb{T}^m;N)$-valued BSDE for which the solution are not necessarily staying in only one local coordinate. Moreover, the global existence of a solution to $L^2(\mathbb{T}^m;N)$-valued BSDE will be proved without any convexity condition on $N$.

math.PR↗