SearcharxivSearch

arXiv subjects

Jingtao Xu

Publications and source records attributed to Jingtao Xu.

14 recordsLinked to original sources

EAGLE-360: Embodied Active Global-to-Local Exploration in 360$^\circ$

While Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in standard visual understanding, adapting them for active visual search in 360$^\circ$ panoramic environments exposes fundamental limitations. Specifically, standard MLLMs struggle to effectively model inherent panoramic properties, such as severe polar distortion and continuous cylindrical topologies, which significantly degrades target detection accuracy. Consequently, existing panoramic search methods attempt to compensate by relying heavily on fragmented local viewpoints. Burdened by rigid initialization and a lack of global panoramic priors, these approaches suffer from myopic, inefficient exploration and struggle with robust error recovery when targets are out of view. To overcome these challenges, we propose EAGLE-360, a novel Embodied Active Global-to-Local Exploration framework. Rather than performing exhaustive local searches, EAGLE-360 leverages global priors to establish an initial holistic perspective, iteratively reasoning and progressively narrowing the search space. Architecturally, we adapt RoPE Rolling, a coordinate-shifting positional encoding mechanism, to seamlessly model the continuous topologies of panoramas. To facilitate this paradigm, we construct the large-scale EAGLE-360 dataset, comprising 14,000+ 4K panoramas and 70,000+ rounds of high-quality VQA dialogues. By employing a training pipeline that integrates Supervised Fine-Tuning (SFT) with Group Relative Policy Optimization (GRPO), we effectively elicit complex spatial reasoning and tool-calling capabilities. Extensive experiments demonstrate that EAGLE-360 establishes a new state-of-the-art for 360$^\circ$ visual search, achieving nearly an 8-fold increase in accuracy over the base model while significantly enhancing exploration efficiency.

cs.CV

Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models

Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is revealed gradually across turns. When a clean FULL prompt and a RAW-SHARDED conversation contain the same complete user evidence, the model should still arrive at the same answer. We argue that a key reason for this gap is self-anchored drift: responses produced under partial information introduce unsupported assumptions, and those assumptions later distort the final answer. To reduce this effect, we propose Canonical-Context On-Policy Distillation (CCOPD). During training, the same base model is used in two roles: a frozen teacher conditioned on the clean FULL prompt and a trainable student that receives the same evidence incrementally through a multi-turn conversation; CCOPD aligns the student's behavior on its own trajectories with the teacher's canonical full-context behavior. Trained only on math problem conversations, CCOPD yields a 32\% average relative improvement in RAW-SHARDED performance over the original base model across math and five zero-shot out-of-domain task families, while largely preserving full-context performance. Further analyses suggest that CCOPD strengthens grounding in user evidence and reduces sensitivity to contamination from earlier assistant turns.

cs.CL

SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model

SkyReels V4 is a unified multi modal video foundation model for joint video audio generation, inpainting, and editing. The model adopts a dual stream Multimodal Diffusion Transformer (MMDiT) architecture, where one branch synthesizes video and the other generates temporally aligned audio, while sharing a powerful text encoder based on the Multimodal Large Language Models (MLLM). SkyReels V4 accepts rich multi modal instructions, including text, images, video clips, masks, and audio references. By combining the MLLMs multi modal instruction following capability with in context learning in the video branch MMDiT, the model can inject fine grained visual guidance under complex conditioning, while the audio branch MMDiT simultaneously leverages audio references to guide sound generation. On the video side, we adopt a channel concatenation formulation that unifies a wide range of inpainting style tasks, such as image to video, video extension, and video editing under a single interface, and naturally extends to vision referenced inpainting and editing via multi modal prompts. SkyReels V4 supports up to 1080p resolution, 32 FPS, and 15 second duration, enabling high fidelity, multi shot, cinema level video generation with synchronized audio. To make such high resolution, long-duration generation computationally feasible, we introduce an efficiency strategy: Joint generation of low resolution full sequences and high-resolution keyframes, followed by dedicated super-resolution and frame interpolation models. To our knowledge, SkyReels V4 is the first video foundation model that simultaneously supports multi-modal input, joint video audio generation, and a unified treatment of generation, inpainting, and editing, while maintaining strong efficiency and quality at cinematic resolutions and durations.

cs.CV

SkyReels-V3 Technique Report

Video generation serves as a cornerstone for building world models, where multimodal contextual inference stands as the defining test of capability. In this end, we present SkyReels-V3, a conditional video generation model, built upon a unified multimodal in-context learning framework with diffusion Transformers. SkyReels-V3 model supports three core generative paradigms within a single architecture: reference images-to-video synthesis, video-to-video extension and audio-guided video generation. (i) reference images-to-video model is designed to produce high-fidelity videos with strong subject identity preservation, temporal coherence, and narrative consistency. To enhance reference adherence and compositional stability, we design a comprehensive data processing pipeline that leverages cross frame pairing, image editing, and semantic rewriting, effectively mitigating copy paste artifacts. During training, an image video hybrid strategy combined with multi-resolution joint optimization is employed to improve generalization and robustness across diverse scenarios. (ii) video extension model integrates spatio-temporal consistency modeling with large-scale video understanding, enabling both seamless single-shot continuation and intelligent multi-shot switching with professional cinematographic patterns. (iii) Talking avatar model supports minute-level audio-conditioned video generation by training first-and-last frame insertion patterns and reconstructing key-frame inference paradigms. On the basis of ensuring visual quality, synchronization of audio and videos has been optimized. Extensive evaluations demonstrate that SkyReels-V3 achieves state-of-the-art or near state-of-the-art performance on key metrics including visual quality, instruction following, and specific aspect metrics, approaching leading closed-source systems. Github: https://github.com/SkyworkAI/SkyReels-V3.

cs.CV

Video2Roleplay: A Multimodal Dataset and Framework for Video-Guided Role-playing Agents

Role-playing agents (RPAs) have attracted growing interest for their ability to simulate immersive and interactive characters. However, existing approaches primarily focus on static role profiles, overlooking the dynamic perceptual abilities inherent to humans. To bridge this gap, we introduce the concept of dynamic role profiles by incorporating video modality into RPAs. To support this, we construct Role-playing-Video60k, a large-scale, high-quality dataset comprising 60k videos and 700k corresponding dialogues. Based on this dataset, we develop a comprehensive RPA framework that combines adaptive temporal sampling with both dynamic and static role profile representations. Specifically, the dynamic profile is created by adaptively sampling video frames and feeding them to the LLM in temporal order, while the static profile consists of (1) character dialogues from training videos during fine-tuning, and (2) a summary context from the input video during inference. This joint integration enables RPAs to generate greater responses. Furthermore, we propose a robust evaluation method covering eight metrics. Experimental results demonstrate the effectiveness of our framework, highlighting the importance of dynamic role profiles in developing RPAs.

cs.MM

Learning from B Cell Evolution: Adaptive Multi-Expert Diffusion for Antibody Design via Online Optimization

Recent advances in diffusion models have shown remarkable potential for antibody design, yet existing approaches apply uniform generation strategies that cannot adapt to each antigen's unique requirements. Inspired by B cell affinity maturation, where antibodies evolve through multi-objective optimization balancing affinity, stability, and self-avoidance, we propose the first biologically-motivated framework that leverages physics-based domain knowledge within an online meta-learning system. Our method employs multiple specialized experts (van der Waals, molecular recognition, energy balance, and interface geometry) whose parameters evolve during generation based on iterative feedback, mimicking natural antibody refinement cycles. Instead of fixed protocols, this adaptive guidance discovers personalized optimization strategies for each target. Our experiments demonstrate that this approach: (1) discovers optimal SE(3)-equivariant guidance strategies for different antigen classes without pre-training, preserving molecular symmetries throughout optimization; (2) significantly enhances hotspot coverage and interface quality through target-specific adaptation, achieving balanced multi-objective optimization characteristic of therapeutic antibodies; (3) establishes a paradigm for iterative refinement where each antibody-antigen system learns its unique optimization profile through online evaluation; (4) generalizes effectively across diverse design challenges, from small epitopes to large protein interfaces, enabling precision-focused campaigns for individual targets.

cs.LG

SkyReels-A2: Compose Anything in Video Diffusion Transformers

This paper presents SkyReels-A2, a controllable video generation framework capable of assembling arbitrary visual elements (e.g., characters, objects, backgrounds) into synthesized videos based on textual prompts while maintaining strict consistency with reference images for each element. We term this task elements-to-video (E2V), whose primary challenges lie in preserving the fidelity of each reference element, ensuring coherent composition of the scene, and achieving natural outputs. To address these, we first design a comprehensive data pipeline to construct prompt-reference-video triplets for model training. Next, we propose a novel image-text joint embedding model to inject multi-element representations into the generative process, balancing element-specific consistency with global coherence and text alignment. We also optimize the inference pipeline for both speed and output stability. Moreover, we introduce a carefully curated benchmark for systematic evaluation, i.e, A2 Bench. Experiments demonstrate that our framework can generate diverse, high-quality videos with precise element control. SkyReels-A2 is the first open-source commercial grade model for the generation of E2V, performing favorably against advanced closed-source commercial models. We anticipate SkyReels-A2 will advance creative applications such as drama and virtual e-commerce, pushing the boundaries of controllable video generation.

cs.CV

AdaZoom: Adaptive Zoom Network for Multi-Scale Object Detection in Large Scenes

Detection in large-scale scenes is a challenging problem due to small objects and extreme scale variation. It is essential to focus on the image regions of small objects. In this paper, we propose a novel Adaptive Zoom (AdaZoom) network as a selective magnifier with flexible shape and focal length to adaptively zoom the focus regions for object detection. Based on policy gradient, we construct a reinforcement learning framework for focus region generation, with the reward formulated by object distributions. The scales and aspect ratios of the generated regions are adaptive to the scales and distribution of objects inside. We apply variable magnification according to the scale of the region for adaptive multi-scale detection. We further propose collaborative training to complementarily promote the performance of AdaZoom and the detection network. To validate the effectiveness, we conduct extensive experiments on VisDrone2019, UAVDT, and DOTA datasets. The experiments show AdaZoom brings a consistent and significant improvement over different detection networks, achieving state-of-the-art performance on these datasets, especially outperforming the existing methods by AP of 4.64% on Vis-Drone2019.

cs.CV

Universal van der Waals-type interactions in rattler containing cage materials

Rattling motion of fillers in cage materials has been of great interest for their import roles in superconductivity and thermoelectric applications. The standing waves of the rattling oscillations are normally lower in energy than the propagating waves of the acoustic phonons, thus exert large influences on the configuration of phonon dispersions as well as the associated thermal and electrical properties. Although it has been extensively studied, the origin of the low energy soft modes is still not clear. In the present paper, we show that van der Waals-type interactions are predominant between fillers and their surrounding cage frameworks, which explains the origin of the low energy modes in cage materials as a universal rule. Mass, free space and chemical environment of guest atoms are shown to be the most important factors to determine the three dimensional van der Waals-type interactions. The present work is mainly focused on type-I clathrates, skutterudites and pyrochlores.

cond-mat.mtrl-sci

Establishing phase diagram for the band engineering in p-type PbTe/SnTe from elementary electronic structure understanding

Band engineering is an important mechanism to increase the thermopower of thermoelectric materials by reconstructing the band structure near Fermi level. PbTe and SnTe are the most representative systems in which band engineering were achieved by various dopants. Starting with the elementary understanding of the band structures, we established the phase diagram for the band engineering in p-type PbTe/SnTe by constructing an s-p bonding model. We show that the effects of band tuning are mainly determined by an inherent parameter of doping element: the s orbital energy level. With the phase diagram, all the related experimental observations can be consistently explained, moreover, undiscovered effective dopants become foreseeable. Our study discovers an applicable criteria to pick up proper dopants from the periodic table directly, and the analytical method can be adopted to more thermoelectric materials.

cond-mat.mtrl-sci

Gap Structure of the Overdoped Iron-Pnictide Superconductor Ba(Fe$_{0.942}$Ni$_{0.058}$)$_{2}$As$_{2}$: A Low-Temperature Specific-Heat Study

Low-temperature specific heat (SH) is measured on the postannealed Ba(Fe_{1-x}Ni_x)_2As_2 single crystal with x = 0.058 under different magnetic fields. The sample locates on the overdoped sides and the critical transition temperature is determined to be 14.8 K by both the magnetization and SH measurements. A simple and reliable analysis shows that, besides the phonon and normal electronic contributions, a clear T2 termemerges in the low temperature SH data.Our observation is similar to that observed in the Co-doped system in our previous work and is consistent with the theoretical prediction for a superconductor with line nodes in the energy gap.

cond-mat.supr-con

Electron and Hole Injection via Charge Transfer at the Topological-Insulator $Bi_{2-x}Sb_xTe_{3-y}Se_y$/Organic-Molecule Interface

As a methodology for controlling the carrier transport of topological insulators (TI's), a flexible tuning in carrier number on the surface states (SS's) of three dimensional TI's by surface modifications using organic molecules is described. The principle of the carrier tuning and its type conversion of TI's presented in this research are based on the charge transfer of holes or electrons at the TI/organic molecule interface. By employing 2,3,5,6-tetrafluoro-7,7,8,8-tetracyanoquinodimethane (F4-TCNQ) as an electron acceptor or tetracyanoquinodimethane (TCNQ) as a donor for n- and p- Bi2-xSbxTe3-ySey (BSTS) single crystals, successful carrier conversion from n to p and its reverse mode is demonstrated depending on the electron affinities of the molecules. The present method provides a nondestructive and efficient method for local tuning in carrier density of TI's, and is useful for future applications.

cond-mat.mtrl-sci

A Field-directional Specific Heat Study on the Gap Structure of Overdoped Ba(Fe$_{1-x}$Co$_{x}$)$_{2}$As$_{2}$

Low-temperature specific heat is measured on the overdoped Ba(Fe_{1-x}Co_x)_2As_2 (x = 0.13) single crystal under magnetic fields along three different directions. A clear anisotropy is observed on the field dependent electronic specific heat coefficient γ(H). The value of γ(H) is obviously larger with magnetic field along [001] (c-axis) than that within the ab-plane of the crystal lattice, which cannot be attributed to the effect by anisotropy of the upper critical field. Meanwhile, the data show a rather small difference when the direction of the field is rotated from [100] to [110] direction within the ab-plane. Our results suggest that a considerable part of the line nodes is not excited to contribute to the quasiparticle density of states by the field when the field is within the ab-plane. The constraints on the topology of the gap nodes are discussed based on our observations.

cond-mat.supr-con

Evidence for line nodes in the energy gap of the overdoped Ba(Fe$_{1-x}$Co$_{x}$)$_{2}$As$_{2}$ from low-temperature specific heat measurements

Low-temperature specific heat (SH) is measured on Ba(Fe$_{1-x}$Co$_{x}$)$_2$As$_2$ single crystals in a wide doping region under different magnetic fields. For the overdoped sample, we find the clear evidence for the presence of $T^2$ term in the data, which is absent both for the underdoped and optimal doped samples, suggesting the presence of line nodes in the energy gap of the overdoped samples. Moreover, the field induced electron specific heat coefficient $Δγ(H)$ increases more quickly with the field for the overdoped sample than the underdoped and optimal doped ones, giving another support to our arguments. Our results suggest that the superconducting gap(s) in the present system may have different structures strongly depending on the doping regions.

cond-mat.supr-con