arXiv subjects

Jiaxi Gu

Publications and source records attributed to Jiaxi Gu.

At least 19 recordsLinked to original sources

arXiv2026

CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise Warping

Precise camera pose control is critical for video diffusion, yet maintaining geometric consistency remains a challenge. Existing methods that directly inject numerical camera parameters into the diffusion backbone often fail to bridge the gap between abstract coordinates and visual content, leading to structural distortions. To address this issue, we propose CameraNoise, a flow-to-noise warping method that encodes camera motion into a temporally coherent stochastic representation. Unlike conventional conditioning, CameraNoise embeds camera poses directly into the noise space. This decouples motion from scene appearance while faithfully preserving trajectory dynamics. Specifically, we introduce a novel Geometry-guided Reprojection Flow and a noise warping algorithm, which jointly preserve the Gaussian prior of diffusion and ensure consistent noise propagation under camera transformations. By integrating CameraNoise into the diffusion process, our framework delivers stable, high-fidelity videos. Extensive experiments demonstrate that our approach significantly outperforms prior methods in both visual quality and trajectory faithfulness. The project page and code are available at: https://gulucaptain.github.io/CameraNoise/.

cs.CV↗

arXiv2026

CT-1: Vision-Language-Camera Models Transfer Spatial Reasoning Knowledge to Camera-Controllable Video Generation

Camera-controllable video generation aims to synthesize videos with flexible and physically plausible camera movements. However, existing methods either provide imprecise camera control from text prompts or rely on labor-intensive manual camera trajectory parameters, limiting their use in automated scenarios. To address these issues, we propose a novel Vision-Language-Camera model, termed CT-1 (Camera Transformer 1), a specialized model designed to transfer spatial reasoning knowledge to video generation by accurately estimating camera trajectories. Built upon vision-language modules and a Diffusion Transformer model, CT-1 employs a Wavelet-based Regularization Loss in the frequency domain to effectively learn complex camera trajectory distributions. These trajectories are integrated into a video diffusion model to enable spatially aware camera control that aligns with user intentions. To facilitate the training of CT-1, we design a dedicated data curation pipeline and construct CT-200K, a large-scale dataset containing over 47M frames. Experimental results demonstrate that our framework successfully bridges the gap between spatial reasoning and video synthesis, yielding faithful and high-quality camera-controllable videos and improving camera control accuracy by 25.7% over prior methods.

cs.CV↗

arXiv2026

A Family of Even-Order Central-Upwind WENO Schemes with Averaged Downwind and Novel Global Smoothness Indicators

We propose a simple yet effective local smoothness indicator for the downwind stencil in central-upwind weighted essentially non-oscillatory (WENO) schemes of even order for hyperbolic conservation laws. Starting from an odd-order upwind WENO scheme, we construct an even-number-of-points stencil by incorporating a downwind substencil whose smoothness indicator is the arithmetic mean of all local smoothness indicators. This straightforward averaging approach incorporates regularity information from the entire stencil without requiring additional tuning parameters or complex formulations. Combined with affine-invariant Z-type nonlinear weights and a carefully designed global smoothness indicator, the resulting scheme, termed WENO-ZA6 for the sixth-order case, achieves optimal convergence rates at critical points up to second order, exhibits favorable dispersion and dissipation properties as confirmed by approximate dispersion relation analysis, and provides sharp, essentially non-oscillatory resolution of discontinuities. Numerical experiments on scalar problems and the one- and two-dimensional Euler equations demonstrate that WENO-ZA6 achieves accuracy comparable to or better than existing sixth-order central-upwind schemes (WENO-CU6, WENO-S6) and the seventh-order WENO-Z7, while requiring approximately 15\%--21\% less computational time. The framework extends naturally to fourth-, eighth-, and tenth-order schemes.

math.NA↗

arXiv2026

A scaled TW-PINN: A physics-informed neural network for traveling wave solutions of reaction-diffusion equations with general coefficients

We propose an efficient and generalizable physics-informed neural network (PINN) framework for computing traveling wave solutions of $n$-dimensional reaction-diffusion equations with various reaction and diffusion coefficients. By applying a scaling transformation with the traveling wave form, the original problem is reduced to a one-dimensional scaled reaction-diffusion equation with unit reaction and diffusion coefficients. This reduction leads to the proposed framework, termed scaled TW-PINN, in which a single PINN solver trained on the scaled equation is reused for different coefficient choices and spatial dimensions. We also prove a universal approximation property of the proposed PINN solver for traveling wave solutions. Numerical experiments in one and two dimensions, together with a comparison to the existing wave-PINN method, demonstrate the accuracy, flexibility, and superior performance of scaled TW-PINN. Finally, we explore an extension of the framework to the Fisher's equation with general initial conditions.

math.NA↗

arXiv2026

DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation

Recent video generative models have demonstrated impressive visual fidelity, yet they often struggle with semantic, geometric, and identity consistency. In this paper, we propose a system-level framework, termed the Divide-and-Conquer Diffusion Model (DCDM), to address three key challenges: (1) intra-clip world knowledge consistency, (2) inter-clip camera consistency, and (3) inter-shot element consistency. DCDM decomposes video consistency modeling under these scenarios into three dedicated components while sharing a unified video generation backbone. For intra-clip consistency, DCDM leverages a large language model to parse input prompts into structured semantic representations, which are subsequently translated into coherent video content by a diffusion transformer. For inter-clip camera consistency, we propose a temporal camera representation in the noise space that enables precise and stable camera motion control, along with a text-to-image initialization mechanism to further enhance controllability. For inter-shot consistency, DCDM adopts a holistic scene generation paradigm with windowed cross-attention and sparse inter-shot self-attention, ensuring long-range narrative coherence while maintaining computational efficiency. We validate our framework on the test set of the CVM Competition at AAAI'26, and the results demonstrate that the proposed strategies effectively address these challenges.

cs.CV↗

arXiv2025

Repeating Words for Video-Language Retrieval with Coarse-to-Fine Objectives

The explosive growth of video streaming presents challenges in achieving high accuracy and low training costs for video-language retrieval. However, existing methods rely on large-scale pre-training to improve video retrieval performance, resulting in significant computational demands. Additionally, the fine-grained information in videos and texts remains underexplored. To alleviate these problems, we propose a novel framework to learn fine-grained features for better alignment and introduce an inference pipeline to improve performance without additional training. Specifically, we employ coarse-to-fine objectives to understand the semantic information of video-text pairs, including contrastive and matching learning. The fine-grained data used for training is obtained through the Granularity-Aware Representation module, which is designed based on similarity analysis between video frames and words in captions. Furthermore, we observe that the repetition of keywords in the original captions, referred to as "Repetition", can enhance retrieval performance and improve alignment between video and text. Based on this insight, we propose a novel and effective inference pipeline that incorporates a voting mechanism and a new Matching Entropy metric to achieve better retrieval performance without requiring additional pre-training. Experimental results on four benchmarks demonstrate that the proposed method outperforms previous approaches. Additionally, our inference pipeline achieves significant performance improvements, with a 2.1% increase in Recall@1 on the MSR-VTT dataset and a 1.6% increase on the DiDeMo dataset.

cs.CV↗

arXiv2025

ShoulderShot: Generating Over-the-Shoulder Dialogue Videos

Over-the-shoulder dialogue videos are essential in films, short dramas, and advertisements, providing visual variety and enhancing viewers' emotional connection. Despite their importance, such dialogue scenes remain largely underexplored in video generation research. The main challenges include maintaining character consistency across different shots, creating a sense of spatial continuity, and generating long, multi-turn dialogues within limited computational budgets. Here, we present ShoulderShot, a framework that combines dual-shot generation with looping video, enabling extended dialogues while preserving character consistency. Our results demonstrate capabilities that surpass existing methods in terms of shot-reverse-shot layout, spatial continuity, and flexibility in dialogue length, thereby opening up new possibilities for practical dialogue video generation. Videos and comparisons are available at https://shouldershot.github.io.

cs.CV↗

arXiv2025

Conservative approximation-based feedforward neural network for WENO schemes

In this work, we present the feedforward neural network based on the conservative approximation to the derivative from point values, for the weighted essentially non-oscillatory (WENO) schemes in solving hyperbolic conservation laws. The feedforward neural network, whose inputs are point values from the three-point stencil and outputs are two nonlinear weights, takes the place of the classical WENO weighting procedure. For the training phase, we employ the supervised learning and create a new labeled dataset for one-dimensional conservative approximation, where we construct a numerical flux function from the given point values such that the flux difference approximates the derivative to high-order accuracy. The symmetric-balancing term is introduced for the loss function so that it propels the neural network to match the conservative approximation to the derivative and satisfy the symmetric property that WENO3-JS and WENO3-Z have in common. The consequent WENO schemes, WENO3-CADNNs, demonstrate robust generalization across various benchmark scenarios and resolutions, where they outperform WENO3-Z and achieve accuracy comparable to WENO5-JS.

math.NA↗

arXiv2025

JS-type and Z-type weights for fourth-order central-upwind weighted essentially non-oscillatory schemes

The central-upwind weighted essentially non-oscillatory (WENO) scheme introduces the downwind substencil to reconstruct the numerical flux, where the smoothness indicator for the downwind substencil is of critical importance in maintaining high order in smooth regions and preserving the essentially nonoscillatory behavior in shock capturing. In this study, we design the smoothness indicator for the downwind substencil by simply summing up all local smoothness indicators and taking the average, which includes the regularity information of the whole stencil. Accordingly the JS-type and Z-type nonlinear weights, based on simple local smoothness indicators, are developed for the fourth-order central-upwind WENO scheme. The accuracy, robustness, and high-resolution properties of our proposed schemes are demonstrated in a variety of one- and two-dimensional problems.

math.NA↗

arXiv2024

EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation

Following the advancements in text-guided image generation technology exemplified by Stable Diffusion, video generation is gaining increased attention in the academic community. However, relying solely on text guidance for video generation has serious limitations, as videos contain much richer content than images, especially in terms of motion. This information can hardly be adequately described with plain text. Fortunately, in computer vision, various visual representations can serve as additional control signals to guide generation. With the help of these signals, video generation can be controlled in finer detail, allowing for greater flexibility for different applications. Integrating various controls, however, is nontrivial. In this paper, we propose a universal framework called EasyControl. By propagating and injecting condition features through condition adapters, our method enables users to control video generation with a single condition map. With our framework, various conditions including raw pixels, depth, HED, etc., can be integrated into different Unet-based pre-trained video diffusion models at a low practical cost. We conduct comprehensive experiments on public datasets, and both quantitative and qualitative results indicate that our method outperforms state-of-the-art methods. EasyControl significantly improves various evaluation metrics across multiple validation datasets compared to previous works. Specifically, for the sketch-to-video generation task, EasyControl achieves an improvement of 152.0 on FVD and 19.9 on IS, respectively, in UCF101 compared with VideoComposer. For fidelity, our model demonstrates powerful image retention ability, resulting in high FVD and IS in UCF101 and MSR-VTT compared to other image-to-video models.

cs.CV↗

arXiv2024

A Numerical Study of WENO Approximations to Sharp Propagating Fronts for Reaction-Diffusion Systems

Many reaction-diffusion systems in various applications exhibit traveling wave solutions that evolve on multiple spatio-temporal scales. These traveling wave solutions are crucial for understanding the underlying dynamics of the system. In this work, we present sixth-order weighted essentially non-oscillatory (WENO) methods within the finite difference framework to solve reaction-diffusion systems. The WENO method allows us to use fewer grid points and larger time steps compared to classical finite difference methods. Our focus is on solving the reaction-diffusion system for the traveling wave solution with the sharp front. Although the WENO method is popular for hyperbolic conservation laws, especially for problems with discontinuity, it can be adapted for the equations of parabolic type, such as reaction-diffusion systems, to effectively handle sharp wave fronts. Thus, we employed the WENO methods specifically developed for equations of parabolic type. We considered various reaction-diffusion equations, including Fisher's, Zeldovich, Newell-Whitehead-Segel, bistable equations, and the Lotka-Volterra competition-diffusion system, all of which yield traveling wave solutions with sharp wave fronts. Numerical examples in this work demonstrate that the central WENO method is highly more accurate and efficient than the commonly used finite difference method. We also provide an analysis related to the numerical speed of the sharp propagating front in the Newell-Whitehead-Segel equation. The overall results confirm that the central WENO method is highly efficient and is recommended for solving reaction-diffusion equations with sharp wave fronts.

math.NA↗

arXiv2024

A third-order finite difference weighted essentially non-oscillatory scheme with shallow neural network

In this paper, we introduce the finite difference weighted essentially non-oscillatory (WENO) scheme based on the neural network for hyperbolic conservation laws. We employ the supervised learning and design two loss functions, one with the mean squared error and the other with the mean squared logarithmic error, where the WENO3-JS weights are computed as the labels. Each loss function consists of two components where the first component compares the difference between the weights from the neural network and WENO3-JS weights, while the second component matches the output weights of the neural network and the linear weights. The former of the loss function enforces the neural network to follow the WENO properties, implying that there is no need for the post-processing layer. Additionally the latter leads to better performance around discontinuities. As a neural network structure, we choose the shallow neural network (SNN) for computational efficiency with the Delta layer consisting of the normalized undivided differences. These constructed WENO3-SNN schemes show the outperformed results in one-dimensional examples and improved behavior in two-dimensional examples, compared with the simulations from WENO3-JS and WENO3-Z.

cs.LG↗

arXiv2024

MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance

In recent years, generative artificial intelligence has achieved significant advancements in the field of image generation, spawning a variety of applications. However, video generation still faces considerable challenges in various aspects, such as controllability, video length, and richness of details, which hinder the application and popularization of this technology. In this work, we propose a controllable video generation framework, dubbed MimicMotion, which can generate high-quality videos of arbitrary length mimicking specific motion guidance. Compared with previous methods, our approach has several highlights. Firstly, we introduce confidence-aware pose guidance that ensures high frame quality and temporal smoothness. Secondly, we introduce regional loss amplification based on pose confidence, which significantly reduces image distortion. Lastly, for generating long and smooth videos, we propose a progressive latent fusion strategy. By this means, we can produce videos of arbitrary length with acceptable resource consumption. With extensive experiments and user studies, MimicMotion demonstrates significant improvements over previous approaches in various aspects. Detailed results and comparisons are available on our project page: https://tencent.github.io/MimicMotion .

cs.CV↗

arXiv2024

AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Grounding

Temporal Video Grounding (TVG) aims to localize a moment from an untrimmed video given the language description. Since the annotation of TVG is labor-intensive, TVG under limited supervision has accepted attention in recent years. The great success of vision-language pre-training guides TVG to follow the traditional "pre-training + fine-tuning" paradigm, however, the pre-training process would suffer from a lack of temporal modeling and fine-grained alignment due to the difference of data nature between pre-train and test. Besides, the large gap between pretext and downstream tasks makes zero-shot testing impossible for the pre-trained model. To avoid the drawbacks of the traditional paradigm, we propose AutoTVG, a new vision-language pre-training paradigm for TVG that enables the model to learn semantic alignment and boundary regression from automatically annotated untrimmed videos. To be specific, AutoTVG consists of a novel Captioned Moment Generation (CMG) module to generate captioned moments from untrimmed videos, and TVGNet with a regression head to predict localization results. Experimental results on Charades-STA and ActivityNet Captions show that, regarding zero-shot temporal video grounding, AutoTVG achieves highly competitive performance with in-distribution methods under out-of-distribution testing, and is superior to existing pre-training frameworks with much less training data.

cs.CV↗

arXiv2024

A spatial-temporal weight analysis and novel nonlinear weights of weighted essentially non-oscillatory schemes for hyperbolic conservation laws

In this paper we analyze the weighted essentially non-oscillatory (WENO) schemes in the finite volume framework by examining the first step of the explicit third-order total variation diminishing Runge-Kutta method. The rationale for the improved performance of the finite volume WENO-M, WENO-Z and WENO-ZR schemes over WENO-JS in the first time step is that the nonlinear weights corresponding to large errors are adjusted to increase the accuracy of numerical solutions. Based on this analysis, we propose novel Z-type nonlinear weights of the finite volume WENO scheme for hyperbolic conservation laws. Instead of taking the difference of the smoothness indicators for the global smoothness indicator, we employ the logarithmic function with tuners to ensure that the numerical dissipation is reduced around discontinuities while the essentially non-oscillatory property is preserved. The proposed scheme does not necessitate substantial extra computational expenses. Numerical examples are presented to demonstrate the capability of the proposed WENO scheme in shock capturing.

math.NA↗

arXiv2024

Explicit radial basis function Runge-Kutta methods

The aim of this paper is to design the explicit radial basis function (RBF) Runge-Kutta methods for the initial value problem. We construct the two-, three- and four-stage RBF Runge-Kutta methods based on the Gaussian RBF Euler method with the shape parameter, where the analysis of the local truncation error shows that the s-stage RBF Runge-Kutta method could formally achieve order s+1. The proof for the convergence of those RBF Runge-Kutta methods follows. We then plot the stability region of each RBF Runge-Kutta method proposed and compare with the one of the correspondent Runge-Kutta method. Numerical experiments are provided to exhibit the improved behavior of the RBF Runge-Kutta methods over the standard ones.

math.NA↗

arXiv2023

BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models

Diffusion models have made tremendous progress in text-driven image and video generation. Now text-to-image foundation models are widely applied to various downstream image synthesis tasks, such as controllable image generation and image editing, while downstream video synthesis tasks are less explored for several reasons. First, it requires huge memory and computation overhead to train a video generation foundation model. Even with video foundation models, additional costly training is still required for downstream video synthesis tasks. Second, although some works extend image diffusion models into videos in a training-free manner, temporal consistency cannot be well preserved. Finally, these adaption methods are specifically designed for one task and fail to generalize to different tasks. To mitigate these issues, we propose a training-free general-purpose video synthesis framework, coined as {\bf BIVDiff}, via bridging specific image diffusion models and general text-to-video foundation diffusion models. Specifically, we first use a specific image diffusion model (e.g., ControlNet and Instruct Pix2Pix) for frame-wise video generation, then perform Mixed Inversion on the generated video, and finally input the inverted latents into the video diffusion models (e.g., VidRD and ZeroScope) for temporal smoothing. This decoupled framework enables flexible image model selection for different purposes with strong task generalization and high efficiency. To validate the effectiveness and general use of BIVDiff, we perform a wide range of video synthesis tasks, including controllable video generation, video editing, video inpainting, and outpainting.

cs.CV↗

arXiv2023

DreamVideo: High-Fidelity Image-to-Video Generation with Image Retention and Text Guidance

Image-to-video generation, which aims to generate a video starting from a given reference image, has drawn great attention. Existing methods try to extend pre-trained text-guided image diffusion models to image-guided video generation models. Nevertheless, these methods often result in either low fidelity or flickering over time due to their limitation to shallow image guidance and poor temporal consistency. To tackle these problems, we propose a high-fidelity image-to-video generation method by devising a frame retention branch based on a pre-trained video diffusion model, named DreamVideo. Instead of integrating the reference image into the diffusion process at a semantic level, our DreamVideo perceives the reference image via convolution layers and concatenates the features with the noisy latents as model input. By this means, the details of the reference image can be preserved to the greatest extent. In addition, by incorporating double-condition classifier-free guidance, a single image can be directed to videos of different actions by providing varying prompt texts. This has significant implications for controllable video generation and holds broad application prospects. We conduct comprehensive experiments on the public dataset, and both quantitative and qualitative results indicate that our method outperforms the state-of-the-art method. Especially for fidelity, our model has a powerful image retention ability and delivers the best results in UCF101 compared to other image-to-video models to our best knowledge. Also, precise control can be achieved by giving different text prompts. Further details and comprehensive results of our model will be presented in https://anonymous0769.github.io/DreamVideo/.

cs.CV↗