arXiv subjects

Hao Ouyang

Publications and source records attributed to Hao Ouyang.

At least 37 records · Page 2Linked to original sources

arXiv2025

HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives

State-of-the-art text-to-video models excel at generating isolated clips but fall short of creating the coherent, multi-shot narratives, which are the essence of storytelling. We bridge this "narrative gap" with HoloCine, a model that generates entire scenes holistically to ensure global consistency from the first shot to the last. Our architecture achieves precise directorial control through a Window Cross-Attention mechanism that localizes text prompts to specific shots, while a Sparse Inter-Shot Self-Attention pattern (dense within shots but sparse between them) ensures the efficiency required for minute-scale generation. Beyond setting a new state-of-the-art in narrative coherence, HoloCine develops remarkable emergent abilities: a persistent memory for characters and scenes, and an intuitive grasp of cinematic techniques. Our work marks a pivotal shift from clip synthesis towards automated filmmaking, making end-to-end cinematic creation a tangible future. Our code is available at: https://holo-cine.github.io/.

cs.CV↗

arXiv2025

Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models

Diffusion large language models (dLLMs) generate text through iterative denoising, yet current decoding strategies discard rich intermediate predictions in favor of the final output. Our work here reveals a critical phenomenon, temporal oscillation, where correct answers often emerge in the middle process, but are overwritten in later denoising steps. To address this issue, we introduce two complementary methods that exploit temporal consistency: 1) Temporal Self-Consistency Voting, a training-free, test-time decoding strategy that aggregates predictions across denoising steps to select the most consistent output; and 2) a post-training method termed Temporal Consistency Reinforcement, which uses Temporal Semantic Entropy (TSE), a measure of semantic stability across intermediate predictions, as a reward signal to encourage stable generations. Empirical results across multiple benchmarks demonstrate the effectiveness of our approach. Using the negative TSE reward alone, we observe a remarkable average improvement of 24.7% on the Countdown dataset over an existing dLLM. Combined with the accuracy reward, we achieve absolute gains of 2.0% on GSM8K, 4.3% on MATH500, 6.6% on SVAMP, and 25.3% on Countdown, respectively. Our findings underscore the untapped potential of temporal dynamics in dLLMs and offer two simple yet effective tools to harness them.

cs.CL↗

arXiv2025

Generalized $T\overline{T}$-like Deformations in Duality-Invariant Nonlinear Electrodynamic Theories

This study introduces a high-order perturbation methodology to categorize two primary solution types within duality-invariant nonlinear electrodynamic theories, adhering to the differential self-duality criterion. The first solution type aligns with irrelevant stress tensor flows, resembling $T\bar{T}$ dynamics, and the second involves a blend of irrelevant $T\bar{T}$-like and marginal root-$T\bar{T}$-like deformations. Our approach facilitates the investigation of diverse duality-invariant nonlinear electrodynamics theories and their stress tensor flows and confirms the commutativity of flows initiated by irrelevant and marginal operators.

hep-th↗

arXiv2025

Generalized $T\bar{T}$-like flows for scalar theories in two dimensions

We demonstrate that the necessary condition for $SO(N) \times SO(N)$ duality invariance manifests as a partial differential equation in two-dimensional scalar theories. This condition, expressed as a partial differential equation, corresponds precisely to the integrability condition. We derive a general perturbation solution to this partial differential equation, which includes both a root $T\bar{T}$ flow equation and an irrelevant $T\bar{T}$-like flow equation. Additionally, we identify a general form for these flow equations that commute with each other. Our results establish a general integrable theory characterized by theory-dependent coefficients at each order in the $λ$-expansion. This unified framework systematically classifies all integrable theories possessing two Lorentz-invariant variables ($P_1$, $P_2$) while accommodating arbitrary orders of the coupling constants ($λ$, $γ$). The theory provides a comprehensive classification scheme that encompasses both known and novel integrable systems within this class.

hep-th↗

arXiv2025

Calligrapher: Freestyle Text Image Customization

We introduce Calligrapher, a novel diffusion-based framework that innovatively integrates advanced text customization with artistic typography for digital calligraphy and design applications. Addressing the challenges of precise style control and data dependency in typographic customization, our framework incorporates three key technical contributions. First, we develop a self-distillation mechanism that leverages the pre-trained text-to-image generative model itself alongside the large language model to automatically construct a style-centric typography benchmark. Second, we introduce a localized style injection framework via a trainable style encoder, which comprises both Qformer and linear layers, to extract robust style features from reference images. An in-context generation mechanism is also employed to directly embed reference images into the denoising process, further enhancing the refined alignment of target styles. Extensive quantitative and qualitative evaluations across diverse fonts and design contexts confirm Calligrapher's accurate reproduction of intricate stylistic details and precise glyph positioning. By automating high-quality, visually consistent typography, Calligrapher surpasses traditional models, empowering creative practitioners in digital art, branding, and contextual typographic design.

cs.CV↗

arXiv2025

$T\overline{T}$ Deformation: Introduction and Some Recent Advances

This review explores recent advances in the theory of $T\bar{T}$ deformation, an irrelevant yet solvable deformation of quantum field theories defined via the quadratic form of the energy-momentum tensor. It addresses classical and quantum aspects, highlighting significant developments across various fields, including field theory, holography, and string theory. Classically, $T\bar{T}$ deformation manifests through multiple geometric interpretations, notably random geometry, Jackiw-Teitelboim-like gravity, and uniform light-cone gauge frameworks. For quantum aspects, the deformation introduces notable features such as non-locality, UV-IR mixing, solvable renormalization structures, and intriguing modifications to correlation functions and entanglement properties. Furthermore, the paper examines the profound relationship between $T\bar{T}$ deformation and holography, particularly within the mixed boundary conditions/cutoff AdS holography proposal and holographic entanglement entropy. Connections to string theory through single-trace deformations and their holographic duals further reveal the deformed structure of the worldsheet. This review synthesizes recent developments and outlines potential directions for future research in the study of $T\bar{T}$-like deformation.

hep-th↗

arXiv2025

LeviTor: 3D Trajectory Oriented Image-to-Video Synthesis

The intuitive nature of drag-based interaction has led to its growing adoption for controlling object trajectories in image-to-video synthesis. Still, existing methods that perform dragging in the 2D space usually face ambiguity when handling out-of-plane movements. In this work, we augment the interaction with a new dimension, i.e., the depth dimension, such that users are allowed to assign a relative depth for each point on the trajectory. That way, our new interaction paradigm not only inherits the convenience from 2D dragging, but facilitates trajectory control in the 3D space, broadening the scope of creativity. We propose a pioneering method for 3D trajectory control in image-to-video synthesis by abstracting object masks into a few cluster points. These points, accompanied by the depth information and the instance information, are finally fed into a video diffusion model as the control signal. Extensive experiments validate the effectiveness of our approach, dubbed LeviTor, in precisely manipulating the object movements when producing photo-realistic videos from static images. Our code is available at: https://github.com/ant-research/LeviTor.

cs.CV↗

arXiv2025

MagicQuill: An Intelligent Interactive Image Editing System

Image editing involves a variety of complex tasks and requires efficient and precise manipulation techniques. In this paper, we present MagicQuill, an integrated image editing system that enables swift actualization of creative ideas. Our system features a streamlined yet functionally robust interface, allowing for the articulation of editing operations (e.g., inserting elements, erasing objects, altering color) with minimal input. These interactions are monitored by a multimodal large language model (MLLM) to anticipate editing intentions in real time, bypassing the need for explicit prompt entry. Finally, we apply a powerful diffusion prior, enhanced by a carefully learned two-branch plug-in module, to process editing requests with precise control. Experimental results demonstrate the effectiveness of MagicQuill in achieving high-quality image edits. Please visit https://magic-quill.github.io to try out our system.

cs.CV↗

arXiv2025

Learning Naturally Aggregated Appearance for Efficient 3D Editing

Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing due to the implicitness. This work studies the task of efficient 3D editing, where we focus on editing speed and user interactivity. To this end, we propose to learn the color field as an explicit 2D appearance aggregation, also called canonical image, with which users can easily customize their 3D editing via 2D image processing. We complement the canonical image with a projection field that maps 3D points onto 2D pixels for texture query. This field is initialized with a pseudo canonical camera model and optimized with offset regularity to ensure the naturalness of the canonical image. Extensive experiments on different datasets suggest that our representation, dubbed AGAP, well supports various ways of 3D editing (e.g., stylization, instance segmentation, and interactive drawing). Our approach demonstrates remarkable efficiency by being at least 20 times faster per edit compared to existing NeRF-based editing methods. Project page is available at https://felixcheng97.github.io/AGAP/.

cs.CV↗

arXiv2025

AniDoc: Animation Creation Made Easier

The production of 2D animation follows an industry-standard workflow, encompassing four essential stages: character design, keyframe animation, in-betweening, and coloring. Our research focuses on reducing the labor costs in the above process by harnessing the potential of increasingly powerful generative AI. Using video diffusion models as the foundation, AniDoc emerges as a video line art colorization tool, which automatically converts sketch sequences into colored animations following the reference character specification. Our model exploits correspondence matching as an explicit guidance, yielding strong robustness to the variations (e.g., posture) between the reference character and each line art frame. In addition, our model could even automate the in-betweening process, such that users can easily create a temporally consistent animation by simply providing a character image as well as the start and end sketches. Our code is available at: https://yihao-meng.github.io/AniDoc_demo.

cs.CV↗

arXiv2025

Edicho: Consistent Image Editing in the Wild

As a verified need, consistent editing across in-the-wild images remains a technical challenge arising from various unmanageable factors, like object poses, lighting conditions, and photography environments. Edicho steps in with a training-free solution based on diffusion models, featuring a fundamental design principle of using explicit image correspondence to direct editing. Specifically, the key components include an attention manipulation module and a carefully refined classifier-free guidance (CFG) denoising strategy, both of which take into account the pre-estimated correspondence. Such an inference-time algorithm enjoys a plug-and-play nature and is compatible to most diffusion-based editing methods, such as ControlNet and BrushNet. Extensive results demonstrate the efficacy of Edicho in consistent cross-image editing under diverse settings. We will release the code to facilitate future studies.

cs.CV↗

arXiv2025

MangaNinja: Line Art Colorization with Precise Reference Following

Derived from diffusion models, MangaNinjia specializes in the task of reference-guided line art colorization. We incorporate two thoughtful designs to ensure precise character detail transcription, including a patch shuffling module to facilitate correspondence learning between the reference color image and the target line art, and a point-driven control scheme to enable fine-grained color matching. Experiments on a self-collected benchmark demonstrate the superiority of our model over current solutions in terms of precise colorization. We further showcase the potential of the proposed interactive point control in handling challenging cases, cross-character colorization, multi-reference harmonization, beyond the reach of existing algorithms.

cs.CV↗

arXiv2024

CoDeF: Content Deformation Fields for Temporally Consistent Video Processing

We present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire video and a temporal deformation field recording the transformations from the canonical image (i.e., rendered from the canonical content field) to each individual frame along the time axis. Given a target video, these two fields are jointly optimized to reconstruct it through a carefully tailored rendering pipeline. We advisedly introduce some regularizations into the optimization process, urging the canonical content field to inherit semantics (e.g., the object shape) from the video. With such a design, CoDeF naturally supports lifting image algorithms for video processing, in the sense that one can apply an image algorithm to the canonical image and effortlessly propagate the outcomes to the entire video with the aid of the temporal deformation field. We experimentally show that CoDeF is able to lift image-to-image translation to video-to-video translation and lift keypoint detection to keypoint tracking without any training. More importantly, thanks to our lifting strategy that deploys the algorithms on only one image, we achieve superior cross-frame consistency in processed videos compared to existing video-to-video translation approaches, and even manage to track non-rigid objects like water and smog. Project page can be found at https://qiuyu96.github.io/CoDeF/.

cs.CV↗

arXiv2024

Dynamic Typography: Bringing Text to Life via Video Diffusion Prior

Text animation serves as an expressive medium, transforming static communication into dynamic experiences by infusing words with motion to evoke emotions, emphasize meanings, and construct compelling narratives. Crafting animations that are semantically aware poses significant challenges, demanding expertise in graphic design and animation. We present an automated text animation scheme, termed "Dynamic Typography", which combines two challenging tasks. It deforms letters to convey semantic meaning and infuses them with vibrant movements based on user prompts. Our technique harnesses vector graphics representations and an end-to-end optimization-based framework. This framework employs neural displacement fields to convert letters into base shapes and applies per-frame motion, encouraging coherence with the intended textual concept. Shape preservation techniques and perceptual loss regularization are employed to maintain legibility and structural integrity throughout the animation process. We demonstrate the generalizability of our approach across various text-to-video models and highlight the superiority of our end-to-end methodology over baseline methods, which might comprise separate tasks. Through quantitative and qualitative evaluations, we demonstrate the effectiveness of our framework in generating coherent text animations that faithfully interpret user prompts while maintaining readability. Our code is available at: https://animate-your-word.github.io/demo/.

cs.CV↗

arXiv2024

Framer: Interactive Frame Interpolation

We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking the start and end frames as inputs, our approach supports customizing the transition process by tailoring the trajectory of some selected keypoints. Such a design enjoys two clear benefits. First, incorporating human interaction mitigates the issue arising from numerous possibilities of transforming one image to another, and in turn enables finer control of local motions. Second, as the most basic form of interaction, keypoints help establish the correspondence across frames, enhancing the model to handle challenging cases (e.g., objects on the start and end frames are of different shapes and styles). It is noteworthy that our system also offers an "autopilot" mode, where we introduce a module to estimate the keypoints and refine the trajectory automatically, to simplify the usage in practice. Extensive experimental results demonstrate the appealing performance of Framer on various applications, such as image morphing, time-lapse video generation, cartoon interpolation, etc. The code, the model, and the interface will be released to facilitate further research.

cs.CV↗

arXiv2024

Geometric formulation of generalized root-$T\bar{T}$ deformations

We develop a generic geometric formalism that incorporates both $T\bar{T}$-like and root-$T\bar{T}$-like deformations in arbitrary dimensions. This framework applies to a wide family of stress-energy tensor perturbations and encompasses various well-known field theories. Building upon the recently proposed correspondence between Ricci-based gravity and $T\bar{T}$-like deformations, we further extend this duality to include root-$T\bar{T}$-like perturbations. This refinement extends the potential applications of our approach and contributes to a deeper exploration of the interplay between stress tensor perturbations and gravitational dynamics. Among the various original outcomes detailed in this article, we have also obtained a deformation of the flat Jackiw-Teitelboim gravity action.

hep-th↗

arXiv2024

Real-time 3D-aware Portrait Editing from a Single Image

This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our method achieves real-time editing with a feedforward network (i.e., ~0.04s per image), over 100x faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with ~5min fine-tuning per style).

cs.CV↗

arXiv2024

More Fermionic Supersymmetric Wilson loops in Four Dimensions

We construct supersymmetric fermionic Wilson loops along general curves in four-dimensional $\mathcal{N}=4$ super Yang-Mills theory and along general planar curves in $\mathcal{N}=2$ superconformal $SU(N)\times SU(N)$ quiver theory. These loops are generalizations of the Zarembo loops and are cohomologically equivalent to them. In $\mathcal{N}=4$ super Yang-Mills theory, we compute their expectation values and verify the cohomological equivalence relation up to the order $g^4$ in perturbation theory.

hep-th↗