Searcharxiv⌕ Search

arXiv subjects

Jun Liang

Publications and source records attributed to Jun Liang.

At least 19 recordsLinked to original sources
arXiv2026

MoRoute: Dynamic Routing for In-Context Multimodal Video Generation

Multimodal video generation aims to generate and edit videos conditioned on arbitrary combinations of text, images, and videos within a single model, allowing diverse tasks to share complementary data and generative priors. Unifying these tasks requires multimodal understanding of diverse conditions, which is typically provided by a pretrained vision-language model (VLM). A key challenge is how to connect the VLM's hierarchical multimodal representations with a pretrained video diffusion transformer (DiT). Existing methods either inject features from only the final or a few manually selected VLM layers, or jointly train architecture-matched understanding and generation streams, making it difficult to reuse heterogeneous pretrained backbones. We introduce MoRoute, a unified multimodal video generation framework that formulates a frozen VLM and a pretrained video DiT with different architectures as heterogeneous experts connected through dynamic layer routing. For each input, a lightweight block-wise router enables every DiT block to select the VLM layer most relevant to its generation stage, thereby learning an adaptive correspondence between multimodal understanding and video synthesis. MoRoute further incorporates reference images and source videos directly into the DiT token sequence through unified in-context conditioning, preserving fine-grained visual details across diverse generation and editing tasks. Experiments on IntelligentVBench, OpenVE-Bench, and RefVIE-Bench show that MoRoute consistently surpasses the best competing method on each benchmark, improving the average score by 0.15, 0.18, and 0.34 on a 1-5 scale, respectively.

cs.CV↗
arXiv2026

Compression and Retrieval: Implicit Memory Retrieval for Video World Models

Video world models hold promise for simulating interactive environments, yet maintaining consistent long-term memory across complex camera trajectories remains a critical challenge. Existing methods typically rely on computationally expensive context scaling or rigid heuristic retrieval mechanisms, which lacks generalization to varying camera trajectories and environments. In this paper, we propose Compression and Retrieval (CaR), an attention-driven implicit memory retrieval mechanism to overcome these limitations. By injecting viewpoint information via positional encoding, our method performs flexible memory retrieval through attention computation. To efficiently process extended contexts with minimal computational overhead, we further introduce a lightweight context compression network. Furthermore, we construct SceneFly, a large-scale synthetic dataset featuring realistic camera trajectories and frame-level annotations to train and evaluate long-horizon video world models. Extensive experiments demonstrate that our approach achieves state-of-the-art results on established benchmarks and exhibits strong generalization to open-domain scenes.

cs.CV↗
arXiv2026

MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold

We present MoVerse, a real-time video world model that creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations. MoVerse addresses this problem by separating world construction from observation rendering. It first expands the input into a gravity-aligned 360$^\circ$ panorama with topology-aware diffusion, closing the missing field of view before 3D reasoning. It then lifts the panorama into a persistent 3D Gaussian scaffold using panoramic geometry-aware residual prediction, yielding a dense and directly renderable spatial memory. Finally, a Gaussian-conditioned video renderer translates scaffold renderings along user-specified camera trajectories into photorealistic video. To make this renderer practical for interaction, we train a bidirectional diffusion teacher for high-quality conditional rendering and distill it into a causal autoregressive student for bounded-latency streaming. This design combines the controllability and long-range consistency of explicit 3D representations with the perceptual quality of generative video models. MoVerse supports real-time scene roaming at 8~FPS on a single NVIDIA RTX~4090 GPU, demonstrating a practical path toward single-image world creation with interactive video output.

cs.CV↗
arXiv2026

SmartDirector: Keyframe-Conditioned Cinematic Video Generation with Narrative Pacing Control

The narrative quality of a video fundamentally determines its perceptual value. Although existing video generation methods can produce visually appealing content, they predominantly rely on sparse conditioning signals such as text prompts or first/last frames, which limits precise control over narrative structure and temporal pacing. In this paper, we propose SmartDirector, a framework that enhances the narrative capacity of video generation models through multiple keyframes. SmartDirector supports flexible generation scenarios including single-shot generation, multi-shot narrative synthesis, and video extension. The framework operates in two stages: Director-Gen generates a low-resolution video conditioned on the provided keyframes, and Director-SR refines the output by exploiting high-resolution keyframes as semantic anchors to recover fine-grained details. To enable robust multi-keyframe training, we construct a data pipeline that curates single-shot and multi-shot sequences from movies. Extensive experiments demonstrate that SmartDirector substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research.

cs.CV↗
arXiv2026

TOPOS: High-Fidelity and Efficient Industry-Grade 3D Head Generation

High-fidelity 3D head generation plays a crucial role in the film, animation and video game industries. In industrial pipelines, studios typically enforce a fixed reference topology across all head assets, as such a clean and uniform topology is a prerequisite for production-level rigging, skinning and animation. In this paper, we present TOPOS, a framework tailored for single image conditioned 3D head generation that jointly recovers geometry and appearance under such an industry-standard topology. In contrast to general 3D generative models which produce triangle meshes with inconsistent topology and numerous vertices, hindering semantic correspondence and asset-level reuse, TOPOS generates head meshes with a fixed, studio-style topology, enabling consistent vertex-level correspondence across all generated heads. To model heads under this unified topology, we proposed a novel variational autoencoder structure, termed TOPOS-VAE. Inspired by multi-model large language models (MLLMs), our TOPOS-VAE leverages the Perceiver Resampler to convert input pointclouds sampled from head meshes of diverse topologies into the target reference topology. Building upon TOPOS-VAE's structured latent space, we train a rectified flow transformer, TOPOS-DiT, to efficiently generate high-fidelity head meshes from a single image. We further present TOPOS-Texture, an end-to-end module that produces relightable UV texture maps from the same portrait image via fine-tuning a multimodal image generative model. The generated textures are spatially aligned with the underlying mesh geometry and faithfully preserve high-frequency appearance details. Extensive experiments demonstrate that TOPOS achieves state-of-the-art performance on 3D head generation, surpassing both classical face reconstruction methods and general 3D object generative models, highlighting its effectiveness for digital human creation.

cs.CV↗
arXiv2026

MoCam: Unified Novel View Synthesis via Structured Denoising Dynamics

Generative novel view synthesis faces a fundamental dilemma: geometric priors provide spatial alignment but become sparse and inaccurate under view changes, while appearance priors offer visual fidelity but lack geometric correspondence. Existing methods either propagate geometric errors throughout generation or suffer from signal conflicts when fusing both statically. We introduce MoCam, which employs structured denoising dynamics to orchestrate a coordinated progression from geometry to appearance within the diffusion process. MoCam first leverages geometric priors in early stages to anchor coarse structures and tolerate their incompleteness, then switches to appearance priors in later stages to actively correct geometric errors and refine details. This design naturally unifies static and dynamic view synthesis by temporally decoupling geometric alignment and appearance refinement within the diffusion process. Experiments demonstrate that MoCam significantly outperforms prior methods, particularly when point clouds contain severe holes or distortions, achieving robust geometry-appearance disentanglement.

cs.CV↗
arXiv2026

MoZoo:Unleashing Video Diffusion power in animal fur and muscle simulation

The creation of cinematic-quality animal effects necessitates the precise modeling of muscle and fur dynamics, a process that remains both labor-intensive and computationally expensive within traditional production workflows. While generative diffusion models have shown promise in diverse artistic workflows, their capacity for high-fidelity animal simulation remains largely unexploited. We present MoZoo, a generative dynamics solver that bypasses conventional refinement to synthesize high-fidelity animal videos from coarse meshes under multimodal guidance. We propose Role-Aware RoPE (RAR-RoPE) which employs role-based index remapping to synchronize motion alignment while decoupling reference information via fixed temporal offsets. Complementing this, Asymmetric Decoupled Attention partitions the latent sequence to enforce a unidirectional information flow, effectively preventing feature interference and improving computational efficiency. To address the scarcity of high-quality training data, we introduce MoZoo-Data, a synthetic-to-real pipeline that leverages a rendering engine and an inverse mapping approach to construct a large-scale dataset of paired sequences. Furthermore, we establish MoZooBench, a comprehensive benchmark with 120 mesh-video pairs. Experimental results demonstrate that MoZoo achieves high-fidelity fur simulation across diverse animal skeletons and layouts, preserving superior temporal and structural consistency.

cs.GR↗
arXiv2026

Gimbal360: Canonicalizing Planar Diffusion for Spherical Panorama Completion

Diffusion models provide powerful priors for 2D image completion, but these priors are learned on bounded planar images and do not transfer directly to $360^\circ$ panoramas. Perspective observations and spherical panoramas differ in both projective geometry and topology: viewpoint-dependent distortion complicates spatial correspondence, while Equirectangular Projection (ERP) panoramas exhibit intrinsic $S^1$ periodicity that standard Euclidean architectures do not preserve. We present Gimbal360, a unified framework that adapts planar diffusion priors to spherical panoramic completion by standardizing these geometric and topological structures. Our Canonical Viewing Space expresses projective distortion as a fixed function of latitude, providing a consistent interface between perspective inputs and spherical panoramas. To map unposed in-the-wild images into this space, Differentiable Projective Canonicalization projects a dense correspondence field onto a 3-DoF rigid projection manifold without requiring camera parameters at inference. We further introduce Topologically Equivariant Generation, which enforces latent shift equivariance to preserve continuity across the periodic ERP boundary. Together, these designs allow diffusion to operate in a representation whose geometry and topology are explicitly aligned with the spherical domain. We also introduce Horizon360, a curated large-scale dataset of gravity-aligned panoramic environments. Extensive experiments show that Gimbal360 achieves state-of-the-art visual fidelity and seam continuity in $360^\circ$ scene completion.

cs.CV↗
arXiv2026

Auroral signatures of ballooning instability and plasmoid formation processes in the near-Earth magnetotail

The nonlinear development of ballooning instability and the subsequently induced plasmoid formation in the near-Earth magnetotail demonstrated in MHD simulations has been proposed as a potential trigger mechanism for substorm onset over the past decade, and their connections to the in-situ satellite and ground all-sky auroral optical observations have been a subject of continued research. In this work, a set of THEMIS substorm onset events with good conjunction of auroral observations has been selected for comparative simulation study, whose pre-onset magnetotail configuration and conditions are inferred from in-situ data and compared with the onset conditions of ballooning instability obtained in our MHD simulations. The evolution of the near-Earth magnetotail is followed, where the signatures of ballooning instability and the plasmoid formation are extracted from simulations and compared with the magnetic fields and flow patterns within the magnetotail region from observation data. The field-aligned current (FAC) density is evaluated at the Earth side boundary of the magnetotail domain of simulation, which is further mapped along magnetic field lines to the auroral ionosphere and compared with the auroral pattern and evolution there in terms of growth rate, dominant wavenumber, and absolute auroral intensities. Such validation efforts are also the first step towards the development of a self-consistent coupling model that includes the magnetotail-ionosphere interaction in the substorm onset process.

physics.space-ph↗
arXiv2026

MoCha:End-to-End Video Character Replacement without Structural Guidance

Controllable video character replacement with a user-provided identity remains a challenging problem due to the lack of paired video data. Prior works have predominantly relied on a reconstruction-based paradigm that requires per-frame segmentation masks and explicit structural guidance (e.g., skeleton, depth). This reliance, however, severely limits their generalizability in complex scenarios involving occlusions, character-object interactions, unusual poses, or challenging illumination, often leading to visual artifacts and temporal inconsistencies. In this paper, we propose MoCha, a pioneering framework that bypasses these limitations by requiring only a single arbitrary frame mask. To effectively adapt the multi-modal input condition and enhance facial identity, we introduce a condition-aware RoPE and employ an RL-based post-training stage. Furthermore, to overcome the scarcity of qualified paired-training data, we propose a comprehensive data construction pipeline. Specifically, we design three specialized datasets: a high-fidelity rendered dataset built with Unreal Engine 5 (UE5), an expression-driven dataset synthesized by current portrait animation techniques, and an augmented dataset derived from existing video-mask pairs. Extensive experiments demonstrate that our method substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research. Please refer to our project page for more details: orange-3dv-team.github.io/MoCha

cs.CV↗
arXiv2025

Streamer-like red line diffuse auroras driven by time domain structures and ECH waves associated with a plasma injection and braking ion flows

Auroral streamers are important meso-scale processes of dynamic magnetosphere-ionosphere coupling, typically studied using imagers sensitive to energetic (>1 keV) electron precipitation, such as all-sky imagers (ASIs). This paper reports streamer-like red-line auroras, representing low-energy (<1 keV) precipitation, observed poleward of a black aurora and an auroral torch. These red-line auroras were associated with a magnetospheric electron injection and braking ion flows. Observations were made using the THEMIS spacecraft and ground-based imagers, including the ASI, REGO, and meridian scanning photometer (MSP) at Fort Smith. We identify plasma sheet electron pitch-angle scattering by time-domain structures (TDSs) and electron cyclotron harmonics (ECH) waves as the driver of these red-line auroras, because of (1) a strong correlation (~0.9) between observed red-line intensities and precipitating fluxes; (2) consistent red-line intensities from auroral transport code forward modeling, and (3) consistent precipitation characteristic energies from MSP optical inference and quasi-linear estimates.

physics.space-ph↗
arXiv2025

Addressing Domain Shift via Imbalance-Aware Domain Adaptation in Embryo Development Assessment

Deep learning models in medical imaging face dual challenges: domain shift, where models perform poorly when deployed in settings different from their training environment, and class imbalance, where certain disease conditions are naturally underrepresented. We present Imbalance-Aware Domain Adaptation (IADA), a novel framework that simultaneously tackles both challenges through three key components: (1) adaptive feature learning with class-specific attention mechanisms, (2) balanced domain alignment with dynamic weighting, and (3) adaptive threshold optimization. Our theoretical analysis establishes convergence guarantees and complexity bounds. Through extensive experiments on embryo development assessment across four imaging modalities, IADA demonstrates significant improvements over existing methods, achieving up to 25.19\% higher accuracy while maintaining balanced performance across classes. In challenging scenarios with low-quality imaging systems, IADA shows robust generalization with AUC improvements of up to 12.56\%. These results demonstrate IADA's potential for developing reliable and equitable medical imaging systems for diverse clinical settings. The code is made public available at \url{https://github.com/yinghemedical/imbalance-aware_domain_adaptation}

cs.CV↗
arXiv2024

Assessment of different head tilt angles in volumetric modulated arc therapy for hippocampus-avoidance whole-brain radiotherapy

Purpose: We aims to examine the impact of different head tilt angles on the dose distribution in the whole-brain target area and organs at risk. It also aims to determine the head tilt angle to achieve optimal radiation therapy outcomes. Methods: CT images were collected from 8 brain metastases patients at 5 different groups of head tilt angle. The treatment plans were designed using the volumetric modulated arc therapy (VMAT) technique. The 5 groups of tilt angle were as follows: [0,10), [10,20), [20,30), [30,40), and [40,45]. The analysis involved assessing parameters such as the uniformity index, conformity index, average dose delivered to the target, dose coverage of the target, hot spots within the target area, maximum dose, and average dose received by organs at risk. Additionally, the study evaluated the correlation between hippocampal dose and other factors and established linear regression models. Results: Significant differences in dosimetric results were observed between the [40,45] and [0,10) head tilt angles. The [40,45] angle showed significant differences compared to the [0,10) angle in the average dose in the target area, dose uniformity, hotspots in the target area, maximum hippocampal dose, maximum dose in the lens, and average dose in the lens. There is a moderate correlation between the maximum dose in the hippocampi and the PTV length. Likewise, the mean dose in the hippocampi is significantly correlated with the hippocampi length. Conclusion: The VMAT plan with a head tilt angle of [40,45] met all dose constraints and demonstrated improved uniformity of the target area while reducing the dose to organs at risk. Furthermore, the linear regression models suggest that increasing the head tilt angle within the current range of [0,45] is likely to lead to a decrease in the average hippocampal dose.

physics.med-ph↗
arXiv2023

Efficient optimization-based trajectory planning

This research addresses the increasing demand for advanced navigation systems capable of operating within confined surroundings. A significant challenge in this field is developing an efficient planning framework that can generalize across various types of collision avoidance missions. Utilizing numerical optimal control techniques, this study proposes a unified optimization-based planning framework to meet these demands. We focus on handling two collision avoidance problems, i.e., the object not colliding with obstacles and not colliding with boundaries of the constrained region. The object or obstacle is denoted as a union of convex polytopes and ellipsoids, and the constrained region is denoted as an intersection of such convex sets. Using these representations, collision avoidance can be approached by formulating explicit constraints that separate two convex sets, or ensure that a convex set is contained in another convex set, referred to as separating constraints and containing constraints, respectively. We propose to use the hyperplane separation theorem to formulate differentiable separating constraints, and utilize the S-procedure and geometrical methods to formulate smooth containing constraints. We state that compared to the state of the art, the proposed formulations allow a considerable reduction in nonlinear program size and geometry-based initialization in auxiliary variables used to formulate collision avoidance constraints. Finally, the efficacy of the proposed unified planning framework is evaluated in two contexts, autonomous parking in tractor-trailer vehicles and overtaking on curved lanes. The results in both cases exhibit an improved computational performance compared to existing methods.

cs.RO↗
arXiv2023

Efficient collision avoidance for autonomous vehicles in polygonal domains

This research focuses on trajectory planning problems for autonomous vehicles utilizing numerical optimal control techniques. The study reformulates the constrained optimization problem into a nonlinear programming problem, incorporating explicit collision avoidance constraints. We present three novel, exact formulations to describe collision constraints. The first formulation is derived from a proposition concerning the separation of a point and a convex set. We prove the separating proposition through De Morgan's laws. Then, leveraging the hyperplane separation theorem we propose two efficient reformulations. Compared with the existing dual formulations and the first formulation, they significantly reduce the number of auxiliary variables to be optimized and inequality constraints within the nonlinear programming problem. Finally, the efficacy of the proposed formulations is demonstrated in the context of typical autonomous parking scenarios compared with state of the art. For generality, we design three initial guesses to assess the computational effort required for convergence to solutions when using the different collision formulations. The results illustrate that the scheme employing De Morgan's laws performs equally well with those utilizing dual formulations, while the other two schemes based on hyperplane separation theorem exhibit the added benefit of requiring lower computational resources.

cs.RO↗
arXiv2023

Two Parallel PageRank Algorithms via Improving Forward Push

Initially used to rank web pages, PageRank has now been applied in many fields. With the growing scale of graph, accelerating PageRank computing is urged and designing parallel algorithm is a feasible solution. In this paper, two parallel PageRank algorithms IFP1 and IFP2 are proposed via improving the state-of-the-art Personalized PageRank algorithm, i.e., Forward Push. Theoretical analysis indicates that, IFP1 can take advantage of the DAG structure of the graph, where the dangling vertices improves the convergence rate and the unreferenced vertices decreases the computation amount. As an improvement of IFP1, IFP2 pushes mass to the dangling vertices only once but rather many times, and thus decreases the computation amount further. Experiments on six data sets illustrate that both IFP1 and IFP2 outperform Power method, where IFP2 with 38 parallelism can be at most 50 times as fast as the Power method.

cs.DS↗
arXiv2022

A Cross-Residual Learning for Image Recognition

ResNets and its variants play an important role in various fields of image recognition. This paper gives another variant of ResNets, a kind of cross-residual learning networks called C-ResNets, which has less computation and parameters than ResNets. C-ResNets increases the information interaction between modules by densifying jumpers and enriches the role of jumpers. In addition, some meticulous designs on jumpers and channels counts can further reduce the resource consumption of C-ResNets and increase its classification performance. In order to test the effectiveness of C-ResNets, we use the same hyperparameter settings as fine-tuned ResNets in the experiments. We test our C-ResNets on datasets MNIST, FashionMnist, CIFAR-10, CIFAR-100, CALTECH-101 and SVHN. Compared with fine-tuned ResNets, C-ResNets not only maintains the classification performance, but also enormously reduces the amount of calculations and parameters which greatly save the utilization rate of GPUs and GPU memory resources. Therefore, our C-ResNets is competitive and viable alternatives to ResNets in various scenarios. Code is available at https://github.com/liangjunhello/C-ResNet

cs.CV↗
arXiv2021

Hard Example Guided Hashing for Image Retrieval

Compared with the traditional hashing methods, deep hashing methods generate hash codes with rich semantic information and greatly improves the performances in the image retrieval field. However, it is unsatisfied for current deep hashing methods to predict the similarity of hard examples. It exists two main factors affecting the ability of learning hard examples, which are weak key features extraction and the shortage of hard examples. In this paper, we give a novel end-to-end model to extract the key feature from hard examples and obtain hash code with the accurate semantic information. In addition, we redesign a hard pair-wise loss function to assess the hard degree and update penalty weights of examples. It effectively alleviates the shortage problem in hard examples. Experimental results on CIFAR-10 and NUS-WIDE demonstrate that our model outperformances the mainstream hashing-based image retrieval methods.

cs.CV↗