MetaSICL: Globalizing Auditory LLMs for Underserved Speakers and Languages via Meta Speech In-Context Learning

Abstract

Generative AI for speech and audio is increasingly expected to serve users across languages, cultures, and communities, yet current auditory Large Language Models (LLMs) are still largely trained and evaluated on high-resource data. Globalizing such systems requires handling low-resource settings, where the target speakers, languages, or tasks are poorly represented in training data. In these regimes, collecting enough labeled in-domain data is often impractical, and the small corpora available may still under-represent the test distribution, making direct fine-tuning brittle under domain shift. In-Context Learning (ICL) offers an alternative: instead of updating model parameters for every underserved community, an auditory LLM can adapt at inference time by conditioning on a few local demonstrations. However, vanilla speech ICL remains limited because most auditory LLMs are not explicitly trained to use such demonstrations effectively. We address this gap with Meta Speech In-Context Learning (MetaSICL), a post-training recipe that strengthens an auditory LLM's in-context adaptation ability using only abundant high-resource speech data. Although MetaSICL never trains on the target low-resource domains, it improves performance across two backbones on children's ASR, audio understanding/reasoning, and speech translation and ASR in directions and languages unseen in post-training. We further study the case where some in-domain data is available, using low-resource language ASR as a case study, since recognition for underserved languages is central to globalizing generative AI. Here, using MetaSICL as a warmup for in-domain reinforcement learning yields the strongest results, outperforming direct fine-tuning across five typologically diverse languages. Overall, MetaSICL offers a practical route toward globalizing auditory LLMs by building inference-time adaptation into the model.