Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.02367

The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation

Yichen Liu·Quanwei Zhang·Haozhe Wang·Donghao Zhou·Xiaojie Li·Yang Shi·Jiaming Liu·Ruihua Huang·Yingtian Zou·Daquan Zhou
View original source ↗Read source document ↗Export metadata ↓

Abstract

Joint audio-video generation models have made substantial progress in visual quality and audio-visual synchronization. However, they still provide limited control over when shot transitions occur and dialogue is spoken. This limitation constrains their application in script-driven content creation, where timing errors can undermine narrative coherence and the viewing experience. Current joint generators align video and audio representations on a shared temporal axis, yet the precise timing of shots and dialogue specified in a structured prompt is encoded only in the prompt's text representation and remains unaligned with the temporal coordinates of either modality. Consequently, video and audio may remain synchronized with each other while both fail to follow the script timeline. This mismatch motivates us to extend temporal alignment beyond video and audio to include the structured script. We therefore introduce Temporal Context Routing (TCR), which maps the script timing onto the shared temporal axis of video and audio generation and routes each prompt's guidance to the corresponding positions in both modalities. Compared with the baseline on 200 test scripts, TCR reduces Shot Boundary MAE by 96%, from 1.11 s to 0.042 s, and raises Dialogue Acc@0.5 s from 28.3% to 84.1%. TCR achieves these improvements while maintaining visual quality and audio-visual synchronization comparable to those of the baselines. A user study further shows that participants prefer TCR on all five evaluated dimensions.

Explore related subjects

cs.MMcs.CV

Keep this discovery

BibTeXRIS

Yichen Liu, Quanwei Zhang, Haozhe Wang, Donghao Zhou, Xiaojie Li, Yang Shi, Jiaming Liu, Ruihua Huang, Yingtian Zou, Daquan Zhou. 2026-09-02. The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation. https://arxiv.org/abs/2609.02367

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.CV subjectcs.MM subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related papers

arXiv2026

MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents

AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However, existing benchmarks such as BrowseComp primarily focus on textual content, overlooking the prevalence of multimodal content. To bridge this gap, we introduce MM-BrowseComp, a novel benchmark comprising 400 challenging, hand-crafted questions designed to evaluate multimodal retrieval and reasoning capabilities. Unlike prior work, MM-BrowseComp incorporates visual prompts and necessitates the extraction of key evidence from web images and videos to complete questions, rendering text-only approaches insufficient. Additionally, we provide a verified checklist for each question, enabling fine-grained analysis of multimodal dependencies and reasoning paths. Our comprehensive evaluation of 27 state-of-the-art models reveals that even leading models like GPT-5-High with tools achieve only 24.25\% accuracy, highlighting the suboptimal multimodal browsing capabilities, establishing MM-BrowseComp as a rigorous new standard for the field.

cs.CL↗
arXiv2026

The Potential of Haptic Foundation Models

Despite the success of foundation models in language and vision, their expansion into embodied AI is bottlenecked by a lack of generalized touch sensing. This limitation is especially relevant to consumer electronics, where smartphones, wearables, VR controllers, home robots, and health monitoring devices require safe and adaptive physical interaction. Constrained by hardware heterogeneity and the necessity of active physical data collection, current haptic models remain rigidly task-specific. To overcome these limitations, this article explores the transformative potential and developmental trajectory of Haptic Foundation Models (HFMs). We detail the paradigm shift required to transition from passive Large Language Models and Vision Language Models into active HFMs across four core dimensions: action coupling, physical dynamical representation space, continuous time-series data granularity, and action-conditioned future state prediction. Furthermore, we synthesize existing large-scale tactile datasets and benchmark UniTouch, AnyTouch, T3, and Sparsh on TacBench for force estimation, slip detection, and relative pose estimation.

cs.RO↗
arXiv2026

SCoPE-Reg: Efficient Rigid Ultrasound Slice-to-Volume Registration via State-Space Correlation and Closed-Form Pose Estimation

Ultrasound-guided interventions can require localization of an untracked 2D frame within a 3D anatomical reference. Rigid slice-to-volume registration (SVR) estimates this six-degree-of-freedom pose but remains challenging because of limited anatomical context, acoustic artifacts, and view-dependent appearance. Existing methods often use dense cross-attention, whose cost scales with the product of slice and volume token counts, or direct pose regression without explicit correspondence constraints. We introduce SCoPE-Reg, combining state-space slice--volume interaction, dense 3D coordinate prediction, and parameter-free weighted Kabsch estimation. On SVR tasks from CAMUS and $μ$-RegPro, SCoPE-Reg yields mean target registration errors of $0.73$ mm and $2.27$ mm against $1.24$ mm and $2.63$ mm for the state of the art (SOTA), reduces peak error on CAMUS by 56% below SOTA ($12.5\!\to\!5.5$ mm), and registers $100\%$ and $80\%$ of frames within $3$ mm. On CAMUS at $128^2$ it retains the lowest error at increasing pose-perturbation magnitude. It holds $6.49$ M parameters independent of resolution, sustaining $51$ FPS at $512^2$. SCoPE-Reg establishes a SOTA in rigid ultrasound SVR: by coupling correspondence-based accuracy with bounded worst-case error and resolution-independent cost, it becomes viable at native acquisition resolution during intervention, where prior methods trade accuracy, reliability, or frame rate against one another. Supplementary code provided and will be open-sourced upon acceptance.

eess.IV↗