arXiv · 2611.18610
NYUAD at ImageEval Shared Tasks: A Multimodal LLM for Audio-Visual Question Answering and Hallucination Detection in English and Modern Standard Arabic
Abstract
The increasing adoption of multimodal large language models (MLLMs) has enabled more natural interaction with visual and spoken information, creating new opportunities for intelligent audio-visual question answering (AVQA) and hallucination detection systems. AVQA extends traditional visual question answering by requiring models to jointly understand an input image and a spoken question to generate accurate, context-aware responses. Despite recent advances, AVQA and hallucination detection tasks remain challenging for modern standard Arabic (MSA) due to the limited availability of multimodal datasets and the need to reason across Arabic speech and text modalities alongside visual inputs. In this paper, we present an MLLM-based framework for AVQA that takes an image and a spoken question as input and generates answers. In addition, we utilized this framework to detect hallucination in text-image pairs. The proposed approach is evaluated on the ImageEval 2026 Task (Ayn-VQA) benchmark, which focuses on culturally grounded Arabic AVQA. Experimental results demonstrate that the proposed framework effectively understands spoken queries, grounds their responses in the visual content, and produces accurate answers. The results underscore the capacity and limitations of different LLM architectures for both English and MSA and under various conditions.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Nouar AlDahoul, Yasir Zaki. 2026-09-04. NYUAD at ImageEval Shared Tasks: A Multimodal LLM for Audio-Visual Question Answering and Hallucination Detection in English and Modern Standard Arabic. https://searcharxiv.com/abs/2611.18610
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.