arXiv subjects

Jiwen Liu

Publications and source records attributed to Jiwen Liu.

At least 19 recordsLinked to original sources

arXiv2026

TARS: Timestep-Aware Data Scaling for 3D-Free Video Re-Shooting

Video re-shooting aims to regenerate videos with controllable camera motion and viewpoint. Existing methods rely on explicit 3D priors, which are limited by reconstruction quality and often perform poorly when synthesizing previously unseen regions, or on paired videos with different camera trajectories, whose scarcity hinders generalization. We revisit video re-shooting through text-driven semantic viewpoint specification, enabling control over shot scale, viewing angle, and first-/third-person perspective. To this end, we propose TARS, a 3D-free video re-shooting paradigm. Timestep-wise sensitivity analysis reveals that camera motion is primarily established during high-noise stages, where coarse spatiotemporal structures are formed. Based on this insight, we introduce self-supervised training to learn camera dynamics and fundamental visual representations without paired re-shooting data or 3D reconstruction. Through data scaling and joint textual-camera conditioning, TARS supports robust camera and viewpoint control, plausibly synthesizing regions beyond the source view under large camera motions while enabling reverse-angle re-shooting and perspective switching. Extensive experiments show that TARS provides more accurate and temporally consistent camera control than prior methods. Project Page: https://ymlinfeng.github.io/TARS.github.io/

cs.CV↗

arXiv2026

Vera: Identity-Faithful Human Subject-to-Video Generation

Subject-to-video (S2V) generation has made substantial progress in preserving reference subjects across diverse categories, yet generic subject consistency remains insufficient for human-centric generation. A video may appear globally consistent while identity-critical human details still drift across frames, poses, and interactions. This issue becomes more severe in multi-person scenarios, where incorrect identity-role binding leads to subject confusion, attribute swapping, and excessive copying of reference-specific appearance cues. We propose Vera, a unified human-centric S2V framework for single- and multi-person generation. We first construct a million-pair identity-aligned human image-video dataset through person-level cross-clip retrieval, providing explicit identity correspondence and diverse references. Built on this dataset, Vera introduces two complementary designs. Identity-Focal Masked Supervision (IFMS) strengthens identity-aware learning with spatially focused supervision while reducing interference from irrelevant artifacts. Reference-Aware Layer-wise Attention (RALA) regulates how video tokens interact with reference identity cues in the DiT backbone, preserving stable identity anchors and enhancing layer-aware identity readout. Extensive experiments demonstrate that Vera improves human identity consistency, multi-person subject binding, and motion naturalness, while reducing identity confusion and excessive reference-image copying.

cs.CV↗

arXiv2026

OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data

Cloning camera motion from reference videos is an important task in video generation, as videos provide intuitive and precise control. Existing methods either directly use parametric representations that fail to handle multi-shot generation or synthesize cross-paired data, which suffer from data scarcity, resulting in poor performance in complicated camera motion cloning. To address these issues, we introduce a general camera motion representation that encodes cameras as grid motion videos. This camera grid represents the camera parameters visually and supports the integration of diverse trajectories for multi-shot video generation. Building upon this, we propose OmniDirector, a unified framework trained on a million-scale camera grid-video pairs that coordinates characters, actions, and cameras to provide director-level control for multimodal diffusion transformers. Furthermore, we design a novel hierarchical prompt expansion agent that harmoniously integrates different control signals by systematically describing camera motion and visual content through understanding signal relationships. Extensive experiments demonstrate the superior performance and outstanding controllability of our framework. Project page: https://ymlinfeng.github.io/OmniDirector.github.io/

cs.CV↗

arXiv2026

ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation

Subject-preserving video generation is not solved by frontal-face similarity alone: a generated person must remain recognizable across motion, large viewpoint changes, expression shifts, occlusion, scale variation, and conflicts among text, first-frame, and identity references. We argue that the central bottleneck is the point-reference paradigm, which collapses identity into a single static observation entangled with pose, accessories, lighting, background, and camera statistics. We introduce Argus, a Wan-based framework centered on Stacked Multi-View Identity Mosaic Injection (SMII). SMII converts MLLM-selected image/video identity evidence into a 3*3 stacked mosaic, synchronizes the mosaic with the current diffusion time, and injects it as negative-time read-only memory in Wan's native token space. This turns identity from an external clean adapter or a single reference image into a compact dynamic distribution. Around SMII, an MLLM Identity Director selects informative identity moments and resolves condition conflicts, while no-cross-pair counterfactual training, Temporal Identity Annealing, and Adaptive Self-Likeness Guidance improve robustness without paired subject-video supervision. We further release HardID-Celeb, a public-figure identity-stress benchmark, and introduce YawScore and OccScore to probe large-yaw and first-frame-occlusion robustness. Argus achieves state-of-the-art results on OpenS2V-Eval Human-Domain, reaching 64.38 Total Score, 71.86 FaceSim, 51.62 NexusScore, and 79.14 NaturalScore. On HardID-Celeb, Argus obtains 76.80 FaceSim and improves YawScore and OccScore by 12.60 and 15.10 points over the strongest baselines, demonstrating that dynamic identity memory and large-scale counterfactual self-supervision are highly effective for subject-preserving video generation.

cs.CV↗

arXiv2026

Kling-MotionControl Technical Report

Character animation aims to generate lifelike videos by transferring motion dynamics from a driving video to a reference image. Recent strides in generative models have paved the way for high-fidelity character animation. In this work, we present Kling-MotionControl, a unified DiT-based framework engineered specifically for robust, precise, and expressive holistic character animation. Leveraging a divide-and-conquer strategy within a cohesive system, the model orchestrates heterogeneous motion representations tailored to the distinct characteristics of body, face, and hands, effectively reconciling large-scale structural stability with fine-grained articulatory expressiveness. To ensure robust cross-identity generalization, we incorporate adaptive identity-agnostic learning, facilitating natural motion retargeting for diverse characters ranging from realistic humans to stylized cartoons. Simultaneously, we guarantee faithful appearance preservation through meticulous identity injection and fusion designs, further supported by a subject library mechanism that leverages comprehensive reference contexts. To ensure practical utility, we implement an advanced acceleration framework utilizing multi-stage distillation, boosting inference speed by over 10x. Kling-MotionControl distinguishes itself through intelligent semantic motion understanding and precise text responsiveness, allowing for flexible control beyond visual inputs. Human preference evaluations demonstrate that Kling-MotionControl delivers superior performance compared to leading commercial and open-source solutions, achieving exceptional fidelity in holistic motion control, open domain generalization, and visual quality and coherence. These results establish Kling-MotionControl as a robust solution for high-quality, controllable, and lifelike character animation.

cs.CV↗

arXiv2025

KlingAvatar 2.0 Technical Report

Avatar video generation models have achieved remarkable progress in recent years. However, prior work exhibits limited efficiency in generating long-duration high-resolution videos, suffering from temporal drifting, quality degradation, and weak prompt following as video length increases. To address these challenges, we propose KlingAvatar 2.0, a spatio-temporal cascade framework that performs upscaling in both spatial resolution and temporal dimension. The framework first generates low-resolution blueprint video keyframes that capture global semantics and motion, and then refines them into high-resolution, temporally coherent sub-clips using a first-last frame strategy, while retaining smooth temporal transitions in long-form videos. To enhance cross-modal instruction fusion and alignment in extended videos, we introduce a Co-Reasoning Director composed of three modality-specific large language model (LLM) experts. These experts reason about modality priorities and infer underlying user intent, converting inputs into detailed storylines through multi-turn dialogue. A Negative Director further refines negative prompts to improve instruction alignment. Building on these components, we extend the framework to support ID-specific multi-character control. Extensive experiments demonstrate that our model effectively addresses the challenges of efficient, multimodally aligned long-form high-resolution video generation, delivering enhanced visual clarity, realistic lip-teeth rendering with accurate lip synchronization, strong identity preservation, and coherent multimodal instruction following.

cs.CV↗

arXiv2025

Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesis

Recent advances in audio-driven avatar video generation have significantly enhanced audio-visual realism. However, existing methods treat instruction conditioning merely as low-level tracking driven by acoustic or visual cues, without modeling the communicative purpose conveyed by the instructions. This limitation compromises their narrative coherence and character expressiveness. To bridge this gap, we introduce Kling-Avatar, a novel cascaded framework that unifies multimodal instruction understanding with photorealistic portrait generation. Our approach adopts a two-stage pipeline. In the first stage, we design a multimodal large language model (MLLM) director that produces a blueprint video conditioned on diverse instruction signals, thereby governing high-level semantics such as character motion and emotions. In the second stage, guided by blueprint keyframes, we generate multiple sub-clips in parallel using a first-last frame strategy. This global-to-local framework preserves fine-grained details while faithfully encoding the high-level intent behind multimodal instructions. Our parallel architecture also enables fast and stable generation of long-duration videos, making it suitable for real-world applications such as digital human livestreaming and vlogging. To comprehensively evaluate our method, we construct a benchmark of 375 curated samples covering diverse instructions and challenging scenarios. Extensive experiments demonstrate that Kling-Avatar is capable of generating vivid, fluent, long-duration videos at up to 1080p and 48 fps, achieving superior performance in lip synchronization accuracy, emotion and dynamic expressiveness, instruction controllability, identity preservation, and cross-domain generalization. These results establish Kling-Avatar as a new benchmark for semantically grounded, high-fidelity audio-driven avatar synthesis.

cs.CV↗

arXiv2025

MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generation

Recently, interactive digital human video generation has attracted widespread attention and achieved remarkable progress. However, building such a practical system that can interact with diverse input signals in real time remains challenging to existing methods, which often struggle with heavy computational cost and limited controllability. In this work, we introduce an autoregressive video generation framework that enables interactive multimodal control and low-latency extrapolation in a streaming manner. With minimal modifications to a standard large language model (LLM), our framework accepts multimodal condition encodings including audio, pose, and text, and outputs spatially and semantically coherent representations to guide the denoising process of a diffusion head. To support this, we construct a large-scale dialogue dataset of approximately 20,000 hours from multiple sources, providing rich conversational scenarios for training. We further introduce a deep compression autoencoder with up to 64$\times$ reduction ratio, which effectively alleviates the long-horizon inference burden of the autoregressive model. Extensive experiments on duplex conversation, multilingual human synthesis, and interactive world model highlight the advantages of our approach in low latency, high efficiency, and fine-grained multimodal controllability.

cs.CV↗

arXiv2025

OmniSync: Towards Universal Lip Synchronization via Diffusion Transformers

Lip synchronization is the task of aligning a speaker's lip movements in video with corresponding speech audio, and it is essential for creating realistic, expressive video content. However, existing methods often rely on reference frames and masked-frame inpainting, which limit their robustness to identity consistency, pose variations, facial occlusions, and stylized content. In addition, since audio signals provide weaker conditioning than visual cues, lip shape leakage from the original video will affect lip sync quality. In this paper, we present OmniSync, a universal lip synchronization framework for diverse visual scenarios. Our approach introduces a mask-free training paradigm using Diffusion Transformer models for direct frame editing without explicit masks, enabling unlimited-duration inference while maintaining natural facial dynamics and preserving character identity. During inference, we propose a flow-matching-based progressive noise initialization to ensure pose and identity consistency, while allowing precise mouth-region editing. To address the weak conditioning signal of audio, we develop a Dynamic Spatiotemporal Classifier-Free Guidance (DS-CFG) mechanism that adaptively adjusts guidance strength over time and space. We also establish the AIGC-LipSync Benchmark, the first evaluation suite for lip synchronization in diverse AI-generated videos. Extensive experiments demonstrate that OmniSync significantly outperforms prior methods in both visual quality and lip sync accuracy, achieving superior results in both real-world and AI-generated videos.

cs.CV↗

arXiv2024

4Dynamic: Text-to-4D Generation with Hybrid Priors

Due to the fascinating generative performance of text-to-image diffusion models, growing text-to-3D generation works explore distilling the 2D generative priors into 3D, using the score distillation sampling (SDS) loss, to bypass the data scarcity problem. The existing text-to-3D methods have achieved promising results in realism and 3D consistency, but text-to-4D generation still faces challenges, including lack of realism and insufficient dynamic motions. In this paper, we propose a novel method for text-to-4D generation, which ensures the dynamic amplitude and authenticity through direct supervision provided by a video prior. Specifically, we adopt a text-to-video diffusion model to generate a reference video and divide 4D generation into two stages: static generation and dynamic generation. The static 3D generation is achieved under the guidance of the input text and the first frame of the reference video, while in the dynamic generation stage, we introduce a customized SDS loss to ensure multi-view consistency, a video-based SDS loss to improve temporal consistency, and most importantly, direct priors from the reference video to ensure the quality of geometry and texture. Moreover, we design a prior-switching training strategy to avoid conflicts between different priors and fully leverage the benefits of each prior. In addition, to enrich the generated motion, we further introduce a dynamic modeling representation composed of a deformation network and a topology network, which ensures dynamic continuity while modeling topological changes. Our method not only supports text-to-4D generation but also enables 4D generation from monocular videos. The comparison experiments demonstrate the superiority of our method compared to existing methods.

cs.CV↗

arXiv2024

Temperature, Pressure, Velocity, and Water Vapor Mole Fraction Profiles in a Ramjet Combustor using Dual Frequency Comb Spectroscopy and a High Temperature Absorption Database

Accurate diagnostics of the combustor region of ramjet engines can improve engine design and create benchmarks for computational fluid dynamics models. Previous works demonstrate that dual frequency comb spectroscopy can provide low uncertainty diagnostics of multiple flow parameters in the non-combusting regions of ramjets. However, the high temperatures present in the combustor present a challenge for broadband spectroscopic absorption models that are used to interpret measurements in these regions. Here, we utilize a new water vapor absorption database created for high temperature water-air mixtures to fit spectra measured in a ground-test ramjet engine with a broadband near-infrared dual comb absorption spectrometer. We extract 2D profiles of pressure, temperature, water mole fraction, and velocity using this new database. We demonstrate that the new database provides the lowest fit residuals compared to other water vapor absorption databases. We compare computational fluid dynamics simulations of the combustor with the measured data to demonstrate that the simulations overpredict heat release and water vapor production.

physics.flu-dyn↗

arXiv2022

Spatially resolved mass flux measurements with dual comb spectroscopy

Providing an accurate, representative sample of mass flux across large open areas for atmospheric studies or the extreme conditions of a hypersonic engine is challenging for traditional intrusive or point-based sensors. Here, we demonstrate that laser absorption spectroscopy with mode-locked frequency combs can simultaneously measure all of the components of mass flux (velocity, temperature, pressure, and species mole fraction) with low uncertainty, spatial resolution corresponding to the laser line of sight, and no supplemental sensor readings. The low uncertainty is provided by the broad spectral bandwidth, high resolution, and extremely well-known and controlled frequency axis of stabilized, mode-locked frequency combs. We demonstrate these capabilities using dual frequency comb spectroscopy (DCS) in the isolator of a ground-test supersonic propulsion engine at Wright-Patterson Air Force Base. The mass flux measurements are consistent within 3.6% of the facility-level engine air supply values. A vertical scan of the laser beams in the isolator measures the spatially resolved mass flux, which is compared with computational fluid dynamics simulations. A rigorous uncertainty analysis demonstrates a instrument uncertainty of ~0.4%, and total uncertainty (including non-instrument sources) of ~7% for mass flux measurements. These measurements demonstrate DCS with mode-locked frequency combs as a low-uncertainty mass flux sensor for a variety of applications.

physics.app-ph↗

arXiv2022

Supersonic Combustion Diagnostics with Dual Comb Spectroscopy

Supersonic engine development requires accurate and detailed measurements of fluidic and thermodynamic parameters to optimize engine designs and benchmark computational fluid dynamic (CFD) simulations. Here, we demonstrate that dual frequency comb spectroscopy (DCS) with mode-locked frequency combs can provide simultaneous absolute measurements of several flow parameters with low uncertainty across a range of conditions owing to the broadband and ultrastable optical frequency output of the lasers. We perform DCS measurements across a 6800-7200 cm-1 bandwidth covering hundreds of H2O absorption features resolved with a spectral point spacing of 0.0067 cm-1 and point spacing precision of 1.68 x 10-10 cm-1. We demonstrate 2D profiles of velocity, temperature, pressure, water mole fraction, and air mass flux in a ground-test dual-mode ramjet at Wright-Patterson Air Force Base. The narrow angles of the measurement beams offer sufficient spatial resolution to resolve properties across an oblique shock train in the isolator and the thermal throat of the combustor. We determine that the total measurement uncertainties for the various parameters range from 1% for temperature to 9% for water vapor mole fraction, with the absorption database/model that is used to interpret the data typically contributing the most uncertainty (leaving the door open for even lower uncertainty in the future). CFD at the various measurement locations show good agreement, largely falling within the DCS measurement uncertainty for most profiles and parameters.

physics.app-ph↗

arXiv2010

Monte Carlo cluster algorithm for fluid phase transitions in highly size-asymmetrical binary mixtures

Highly size-asymmetrical fluid mixtures arise in a variety of physical contexts, notably in suspensions of colloidal particles to which much smaller particles have been added in the form of polymers or nanoparticles. Conventional schemes for simulating models of such systems are hamstrung by the difficulty of relaxing the large species in the presence of the small one. Here we describe how the rejection-free geometrical cluster algorithm (GCA) of Liu and Luijten [Phys. Rev. Lett 92, 035504 (2004)] can be embedded within a restricted Gibbs ensemble to facilitate efficient and accurate studies of fluid phase behavior of highly size-asymmetrical mixtures. After providing a detailed description of the algorithm, we summarize the bespoke analysis techniques of Ashton et al. [J. Chem. Phys. 132, 074111 (2010)] that permit accurate estimates of coexisting densities and critical-point parameters. We apply our methods to study the liquid--vapor phase diagram of a particular mixture of Lennard-Jones particles having a 10:1 size ratio. As the reservoir volume fraction of small particles is increased in the range 0--5%, the critical temperature decreases by approximately 50%, while the critical density drops by some 30%. These trends imply that in our system, adding small particles decreases the net attraction between large particles, a situation that contrasts with hard-sphere mixtures where an attractive depletion force occurs.

cond-mat.soft↗

arXiv2006

Simulation of phase transitions in highly asymmetric fluid mixtures

We present a novel method for the accurate numerical determination of the phase behavior of fluid mixtures having large particle size asymmetries. By incorporating the recently developed geometric cluster algorithm within a restricted Gibbs ensemble, we are able to probe directly the density and concentration fluctuations that drive phase transitions, but that are inaccessible to conventional simulation algorithms. We develop a finite-size scaling theory that relates these density fluctuations to those of the grand-canonical ensemble, thereby enabling accurate location of critical points and coexistence curves of multicomponent fluids. Several illustrative examples are presented.

cond-mat.soft↗

arXiv2005

Colloidal stabilization via nanoparticle haloing

We present a detailed numerical study of effective interactions between micron-sized silica spheres, induced by highly charged zirconia nanoparticles. It is demonstrated that the effective interactions are consistent with a recently discovered mechanism for colloidal stabilization. In accordance with the experimental observations, small nanoparticle concentrations induce an effective repulsion that counteracts the intrinsic van der Waals attraction between the colloids and thus stabilizes the suspension. At higher nanoparticle concentrations an attractive potential is recovered, resulting in reentrant gelation. Monte Carlo simulations of this highly size-asymmetric mixture are made possible by means of a geometric cluster Monte Carlo algorithm. A comparison is made to results obtained from the Ornstein-Zernike equations with the hypernetted-chain closure.

cond-mat.soft↗

arXiv2005

Geometric Cluster Algorithm for Interacting Fluids

We discuss a new Monte Carlo algorithm for the simulation of complex fluids. This algorithm employs geometric operations to identify clusters of particles that can be moved in a rejection-free way. It is demonstrated that this geometric cluster algorithm (GCA) constitutes the continuum generalization of the Swendsen-Wang and Wolff cluster algorithms for spin systems. Because of its nonlocal nature, it is particularly well suited for the simulation of fluid systems containing particles of widely varying sizes. The efficiency improvement with respect to conventional simulation algorithms is a rapidly growing function of the size asymmetry between the constituents of the system. We study the cluster-size distribution for a Lennard-Jones fluid as a function of density and temperature and provide a comparison between the generalized GCA and the hard-core GCA for a size-asymmetric mixture with Yukawa-type couplings.

cond-mat.stat-mech↗

arXiv2005

Generalized Geometric Cluster Algorithm for Fluid Simulation

We present a detailed description of the generalized geometric cluster algorithm for the efficient simulation of continuum fluids. The connection with well-known cluster algorithms for lattice spin models is discussed, and an explicit full cluster decomposition is derived for a particle configuration in a fluid. We investigate a number of basic properties of the geometric cluster algorithm, including the dependence of the cluster-size distribution on density and temperature. Practical aspects of its implementation and possible extensions are discussed. The capabilities and efficiency of our approach are illustrated by means of two example studies.

cond-mat.stat-mech↗