arXiv · 2611.10831
Multimodal Models Can See, But Can They Think? A Study of Turkish Visual Reasoning
Abstract
Vision-Language Models have had massive leaps in recent years. Yet, when tested on high-level cognitive tasks like pattern completion, abstract logic, and relational reasoning, they still fall short. This limitation stems from the models' lack of ability to find out underlying rules and maintain constant intermediate steps, rather than merely deciding on the correct option. To research these shortcomings, this study analyzes visual reasoning performance throughout a multilingual evaluation framework encompassing the BİLSEM, MM-IQ, Visual Puzzles, and PuzzleVQA datasets. We evaluate various data mixing strategies and supervised fine-tuning (SFT) methods. We observed that a correct answer does not always mean the reasoning chain is valid. Because of this, we adopt an LLM-as-a-Judge approach to assess the quality of reasoning beyond final answer accuracy. Our experimental results reveal that baseline models often exhibit more stable reasoning compared to fine-tuned counterparts. Most importantly, we observe that SFT can lead to model collapse, manifesting as infinite text loops or empty outputs. The study concludes that current VLMs still struggle with abstract visual reasoning.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Berke Bunyamin Sule, Yavuz Selim Aygan, Ahmed Zeer, Elif Ince, Ali Mert Temizsoy, Ozgur Agar, Himmet Toprak Kesgin, Muzaffer Kaan Yuce, Mehmet Fatih Amasyali. 2026-11-07. Multimodal Models Can See, But Can They Think? A Study of Turkish Visual Reasoning. https://searcharxiv.com/abs/2611.10831
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.