CARDAMOM: A Micro-Dialectal Arabic Speech Dataset for ASR

Bashar Talafha; Samar M. Magdy; Aisha Alansari; Alaa Alkhawaldeh; Abdurrahman Juma; Sharaf Makahleh; Nour Gamal; Omar Attia; Hanaa Kurdi; Najwa Rizk; Maysa Anaya; Hessah Altimyat; Layal Alhazmi; Shumukh Alotaibi; Hajar Alhadaris; Rayan Alomari; Rahaf Almalaq; Malak Alkhorasani; Sara alghamdi; Rahaf Alshamrani; Nsrin Ashraf; Ibrahim Jaradat; Nada Qardahji; Yasmin Zaraket; Elmoukhtar Brahim; Sidi Ebeidy; Oumoulmouminin Mahmoud; Yahjeb Bouha Khatraty; Meya Haroune; Mohammad Ghaddar; Mohamad Eldirany; Rashed Alamoush; Tala Chhaytle; Nuha Albadi; Yahya El Hadj; Hamzah Luqman; Fadi A. Zaraket; Mustafa Jarrar; Muhammad Abdul-Mageed

arXiv · 2609.34481

CARDAMOM: A Micro-Dialectal Arabic Speech Dataset for ASR

View original source ↗Read source document ↗Export metadata ↓

Abstract

We present Cardamom, a micro-dialectal Arabic speech dataset designed to support fine-grained evaluation and adaptation of automatic speech recognition (ASR) systems. Community-curated by native speakers familiar with the represented varieties, Cardamom contains approximately 40 hours of transcribed YouTube speech spanning 21 micro-dialects across Egypt, Jordan, Lebanon, Mauritania, Palestine, and Saudi Arabia. Each segment is annotated with one or more operational micro-dialect labels, code-switching information, and utterance-level perceived gender, enabling analysis of sub-country variation that is obscured by conventional country-level labels. We describe the collection and annotation process, motivate the micro-dialect inventory linguistically, and benchmark four multilingual ASR systems in zero-shot and adapted settings. The strongest zero-shot system obtains 43.47% aggregate WER, with particularly high error rates on Mauritanian and Lebanese varieties; adaptation on Cardamom reduces its WER to 35.21%. Audio-based identification experiments further show that the annotations provide a learnable prediction target, with a dedicated classifier reaching 85.57% accuracy on 21-way micro-dialect identification. Cardamom provides a resource for studying localized dialectal variation and developing Arabic speech systems with broader regional coverage.

Explore related subjects

cs.CL

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Bashar Talafha, Samar M. Magdy, Aisha Alansari, Alaa Alkhawaldeh, Abdurrahman Juma, Sharaf Makahleh, Nour Gamal, Omar Attia, Hanaa Kurdi, Najwa Rizk, Maysa Anaya, Hessah Altimyat, Layal Alhazmi, Shumukh Alotaibi, Hajar Alhadaris, Rayan Alomari, Rahaf Almalaq, Malak Alkhorasani, Sara alghamdi, Rahaf Alshamrani, Nsrin Ashraf, Ibrahim Jaradat, Nada Qardahji, Yasmin Zaraket, Elmoukhtar Brahim, Sidi Ebeidy, Oumoulmouminin Mahmoud, Yahjeb Bouha Khatraty, Meya Haroune, Mohammad Ghaddar, Mohamad Eldirany, Rashed Alamoush, Tala Chhaytle, Nuha Albadi, Yahya El Hadj, Hamzah Luqman, Fadi A. Zaraket, Mustafa Jarrar, Muhammad Abdul-Mageed. 2026-09-28. CARDAMOM: A Micro-Dialectal Arabic Speech Dataset for ASR. https://arxiv.org/abs/2609.34481

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization

As large language models (LLMs) are widely adopted in real-world applications, it has become critical to ensure LLMs satisfy safety constraints, such as non-toxicity and logical consistency, as well as task- and situation-specific constraints. Controlling the output through instructions is a simple and tempting approach; however, it remains brittle, is opaque in how it influences model behavior, and thus cannot reliably ensure constraint satisfaction. Moreover, most recent controlled text generation (CTG) methods require access to the internal components of language models--such as weights or logits--making them incompatible with popular API-based LLMs. In this work, we propose LaSEr-Edit, a constraint-satisfying text revision method that can be applied to any LLMs, black- or white-box. We first find that lightweight, task-specific energy-based models (EBMs) achieve error-localization performance competitive with or even better than that of much larger LLMs, while operating substantially faster. Based on this finding, we propose two variants of text revision methods that incorporate energy-based error localization: LaSEr-LLM Edit, which instructs an LLM to edit text given EBM-predicted error spans, and LaSEr-EBM Edit, which uses the EBM not only for localization but also for editing by reranking edit candidates. Through experiments in diverse single-constraint control tasks, we show that LaSEr-LLM Edit controls text better than plain LLM-based editing in most of the tasks. We also find that LaSEr-EBM Edit further improves the control performance of LaSEr-LLM Edit and achieves among the strongest controllability across all tasks. Furthermore, we find that LaSEr-Edit, especially LaSEr-EBM Edit, performs well even when multiple constraints are controlled simultaneously.

cs.CL↗

arXiv2026

No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding

Reliable evaluation of large language models (LLMs) is critical as their deployment rapidly expands, particularly in high-stakes domains such as business and finance. The LLM-as-a-Judge framework, which uses prompted LLMs to evaluate response quality, is appealing due to its scalability, low cost, and strong correlations with human stylistic preferences. However, it remains unclear how accurately these methods can assess response quality in domains where correctness matters more than style. To address this gap, we introduce the Business and Finance Fundamentals Benchmark (BFF-Bench), a dataset of 160 challenging questions and long-form responses authored by financial professionals. These experts subsequently evaluated the correctness of 1,200 responses generated by a diverse set of LLMs on both BFF-Bench and a challenging subset of MT-Bench. With this expert-annotated dataset of judgments (VERDICTS), we analyze the agreement between a suite of automated grading methods and human experts. While we observe that LLM Judges are more reliable than other grading methods, our findings reveal a clear pattern in LLM Judge performance: when not provided with a correct reference, judges show high agreement with human experts only on questions the judges were able to correctly answer themselves. We demonstrate that providing the judges with expert-written references largely mitigates this issue, highlighting the limits of using LLM-as-a-Judge without any form of human verification.

cs.CL↗

arXiv2026

Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control

Reasoning LLMs trained with long chain-of-thought often overthink: they spend tokens on redundant reflection and transitions that inflate cost without improving accuracy. Static activation steering (e.g.\ SEAL) suppresses such content with a fixed vector, but applies the same strength regardless of how redundant the current chunk actually is. We describe PID-steering, a training-free, decoding-time method that modulates the steering strength with a PID controller driven by a lightweight chunk-level redundancy classifier. On a subset of GSM8K with DeepSeek-R1-Distill-Qwen-1.5B, the method improves accuracy from 85.7\% to 89.6\% (+3.9 pp) while cutting average output length from 1026 to 790 tokens ($-$23\%). We report it as a small-scale proof of concept rather than a benchmark result.

cs.CL↗