arXiv · 2610.14676
CAST-ASD: Causal Attention over Speakers and Time for Online Active Speaker Detection
Abstract
We present CAST-ASD (Causal Attention over Speakers and Time), a causal joint model for active speaker detection (ASD). Recent ASD models improve performance by jointly processing all faces in a scene, but operate offline, with the full clip available before any frame is labelled. Low-latency causal detectors instead process each face independently. CAST-ASD combines joint face reasoning with fully causal inference. It applies scene-level attention across faces within the current frame and causal temporal attention over each face's history, enabling information sharing across speakers with zero lookahead. Its parameter count does not grow with the number of faces. On MSDWild, where most negative face-frames contain a visible competing speaker, CAST-ASD reduces competing-speaker false alarms by 32% relative to the causal per-face detector it extends, with larger gains as scene occupancy increases. It outperforms all causal baselines on both validation splits while using only one-tenth the inter-speaker parameters of a three-block fixed-roster convolution. Code and models are available at https://github.com/prathamg007/CAST-ASD.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Pratham Gupta, Haotian Qi, Tushar Sandhan, Gabriel Skantze. 2026-10-10. CAST-ASD: Causal Attention over Speakers and Time for Online Active Speaker Detection. https://searcharxiv.com/abs/2610.14676
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.