arXiv subjects

Mian Wu

Publications and source records attributed to Mian Wu.

6 recordsLinked to original sources

arXiv2025

Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning

Reinforcement Learning (RL) has achieved remarkable success in various domains, yet it often relies on carefully designed programmatic reward functions to guide agent behavior. Designing such reward functions can be challenging and may not generalize well across different tasks. To address this limitation, we leverage the rich world knowledge contained in pretrained video diffusion models to provide goal-driven reward signals for RL agents without ad-hoc design of reward. Our key idea is to exploit off-the-shelf video diffusion models pretrained on large-scale video datasets as informative reward functions in terms of video-level and frame-level goals. For video-level rewards, we first finetune a pretrained video diffusion model on domain-specific datasets and then employ its video encoder to evaluate the alignment between the latent representations of agent's trajectories and the generated goal videos. To enable more fine-grained goal-achievement, we derive a frame-level goal by identifying the most relevant frame from the generated video using CLIP, which serves as the goal state. We then employ a learned forward-backward representation that represents the probability of visiting the goal state from a given state-action pair as frame-level reward, promoting more coherent and goal-driven trajectories. Experiments on Meta-World and Distracting Control Suite demonstrate the effectiveness of our approach.

cs.LG↗

arXiv2025

RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks

Open-ended generation tasks require outputs to satisfy diverse and often implicit task-specific evaluation rubrics. The sheer number of relevant rubrics leads to prohibitively high verification costs and incomplete assessments of a response, making reinforcement learning (RL) post-training with rubric-based rewards difficult to scale. This problem is exacerbated by the fact that often the best way to combine these rubrics into one single reward is also highly prompt-specific. We propose Reinforcement Learning with Adversarial Critic (RLAC), a post-training approach that addresses these challenges via dynamic rubric verification. Our approach employs a large language model (LLM) as a critic that dynamically identifies only the most likely failure modes (e.g., a factual error or unhandled edge case), which are then verified by an external validator to optimize both generator and critic jointly. By training both the generator and the critic, this game enhances the critic's error detection and the generator's output quality while reducing required verifications. Our experiments demonstrate that RLAC improves factual accuracy in text generation and correctness in code generation, while also outperforming exhaustive verification and reward model methods. We show that dynamic critics are more effective than fixed critics, showcasing the potential of RLAC for scaling RL post-training to free-form generation tasks.

cs.LG↗

arXiv2025

A Scalable Lift-and-Project Differentiable Approach For the Maximum Cut Problem

We propose a scalable framework for solving the Maximum Cut (MaxCut) problem in large graphs using projected gradient ascent on quadratic objectives. Our approach is differentiable and leverages GPUs for gradient-based optimization. It is not a machine learning method and does not require training data. Starting from a continuous relaxation of the classical quadratic binary formulation, we present a parallelized strategy that explores multiple initialization vectors in batch. We analyze the relaxed objective, showing it is convex and has fixed-points corresponding to local optima, particularly at boundary points, highlighting a key challenge in non-convex optimization. To improve exploration, we introduce a lifted quadratic formulation that over-parameterizes the solution space. We also provide a theoretical characterization of these lifted fixed-points. Finally, we propose DECO, a dimension-alternating algorithm that switches between the unlifted and lifted formulations, combined with importance-based degree initialization and a population-based evolutionary hyper-parameter search. Experiments on diverse graph families show that our methods attain comparable or superior performance relative to recent neural networks and GPU-accelerated sampling approaches.

cs.DM↗

arXiv2024

Direct and mediated dipole-dipole interactions in a reconfigurable array of optical traps

Optically levitated nanoparticles in vacuum experience both electrostatic and light-induced dipole-dipole interactions, offering a versatile platform to explore mesoscopic entanglement and many-body dynamics. A significant challenge in optical trap arrays is to achieve site-resolved, point-to-point tunability: adjusting the laser parameters of a single trap typically induces global cross-talk to neighboring sites, hindering independent control. Inspired by tunable couplers in superconducting circuits, we implement an ancillary nanoparticle that functions as a coupler between two target nanoparticles. Within a reconfigurable three-particle array, we demonstrate broad tunability of the direct dipole-dipole interaction by controlling the phase and position of the traps. In addition, we observe spectral signatures consistent with mediated interactions between the target particles via the ancillary one, manifested as mode participation beyond the uncoupled response. Our results establish a practical route to tailored, site-resolved control in multi-particle optical trap arrays, expanding the optical-binding toolbox and opening opportunities for programmable oscillator networks relevant to macroscopic quantum mechanics and precision sensing.

physics.optics↗

arXiv2022

The Equivariance Criterion in a Linear Model for Fixed-X Cases

The field of machine have seen rising applications of equivariance criterion. However, there is no systematic way to justify its usage, including why it works, whether there is an optimal solution and if so, what form it carries. In this article, we explored the usage of equivariance criterion in a normal linear model with fixed-$X$ and extended the model to allow multiple populations, which, in turn, leads to a multivariate invariant location-scale transformation group, compared than the commonly used univariate one. The minimum risk equivariant estimators of the coefficient vector and the diagonal covariance matrix were derived, which were consistent with literature works. This work serves as an early exploration of the usage of equivariance criterion in machine learning, where we confirmed that the least square approach widely used in machine learning indeed carries optimality in some sense at least in the framework of estimation. Meanwhile, the problems can be shown to be equivalent to a mixture from $p$ independent normal samples and via the principle of functional equivariance, an alternative proof can be derived. However, such an approach carries its own limitation with a strong tie to equivariance criterion.

math.ST↗

arXiv2021

Hepatic vessel segmentation based on 3D swin-transformer with inductive biased multi-head self-attention

Purpose: Segmentation of liver vessels from CT images is indispensable prior to surgical planning and aroused broad range of interests in the medical image analysis community. Due to the complex structure and low contrast background, automatic liver vessel segmentation remains particularly challenging. Most of the related researches adopt FCN, U-net, and V-net variants as a backbone. However, these methods mainly focus on capturing multi-scale local features which may produce misclassified voxels due to the convolutional operator's limited locality reception field. Methods: We propose a robust end-to-end vessel segmentation network called Inductive BIased Multi-Head Attention Vessel Net(IBIMHAV-Net) by expanding swin transformer to 3D and employing an effective combination of convolution and self-attention. In practice, we introduce the voxel-wise embedding rather than patch-wise embedding to locate precise liver vessel voxels, and adopt multi-scale convolutional operators to gain local spatial information. On the other hand, we propose the inductive biased multi-head self-attention which learns inductive biased relative positional embedding from initialized absolute position embedding. Based on this, we can gain a more reliable query and key matrix. To validate the generalization of our model, we test on samples which have different structural complexity. Results: We conducted experiments on the 3DIRCADb datasets. The average dice and sensitivity of the four tested cases were 74.8% and 77.5%, which exceed results of existing deep learning methods and improved graph cuts method. Conclusion: The proposed model IBIMHAV-Net provides an automatic, accurate 3D liver vessel segmentation with an interleaved architecture that better utilizes both global and local spatial features in CT volumes. It can be further extended for other clinical data.

eess.IV↗