Searcharxiv⌕ Search

arXiv subjects

Chung-Chun Wang

Publications and source records attributed to Chung-Chun Wang.

6 recordsLinked to original sources

Can Image Generators Hold a Scene Together When Space Is Broken Apart?

Image generators are good at drawing scenes in which things that belong together appear next to each other. In almost every image they generate, and in almost every benchmark used to evaluate these images, "belonging to the same whole" and "adjacent to each other" go together. Therefore, we cannot know whether the model truly grasps the scene as a whole, or simply makes each part connect with its adjacent parts. We propose GJBench to disentangle the two. It requires the model to draw a scene as a scattered jigsaw puzzle: an image is cut into interlocking pieces, then moved apart and rotated. Thus, the spatial plausibility of the scene exists only in how these pieces collectively point to a whole. The model may have a reference image, a target layout, both, or neither. We separately evaluate three requirements: whether the scene is broken down into a specified number of separate pieces, whether the pieces interlock, and whether the pieces can reconstruct the reference scene. While every open and closed model we evaluated could draw these scenes well as a whole, none could keep them together as scattered puzzle pieces, and the three requirements categorized failures into different patterns. The models draw separate pieces but miscount them in different ways, and unless their outlines are provided, the pieces rarely interlock, and their content does not reconstruct the reference image. We also found that the target layout helps the puzzle pieces fit together, while the reference image improves neither their count nor their fit. These findings show that keeping a scene continuous across space that the canvas no longer shows remains an open problem for image generators.

cs.CV↗

Advancing Automated Speaking Assessment Leveraging Multifaceted Relevance and Grammar Information

Current automated speaking assessment (ASA) systems for use in multi-aspect evaluations often fail to make full use of content relevance, overlooking image or exemplar cues, and employ superficial grammar analysis that lacks detailed error types. This paper ameliorates these deficiencies by introducing two novel enhancements to construct a hybrid scoring model. First, a multifaceted relevance module integrates question and the associated image content, exemplar, and spoken response of an L2 speaker for a comprehensive assessment of content relevance. Second, fine-grained grammar error features are derived using advanced grammar error correction (GEC) and detailed annotation to identify specific error categories. Experiments and ablation studies demonstrate that these components significantly improve the evaluation of content relevance, language use, and overall ASA performance, highlighting the benefits of using richer, more nuanced feature sets for holistic speaking assessment.

cs.CL↗

The NTNU System at the S&I Challenge 2025 SLA Open Track

A recent line of research on spoken language assessment (SLA) employs neural models such as BERT and wav2vec 2.0 (W2V) to evaluate speaking proficiency across linguistic and acoustic modalities. Although both models effectively capture features relevant to oral competence, each exhibits modality-specific limitations. BERT-based methods rely on ASR transcripts, which often fail to capture prosodic and phonetic cues for SLA. In contrast, W2V-based methods excel at modeling acoustic features but lack semantic interpretability. To overcome these limitations, we propose a system that integrates W2V with Phi-4 multimodal large language model (MLLM) through a score fusion strategy. The proposed system achieves a root mean square error (RMSE) of 0.375 on the official test set of the Speak & Improve Challenge 2025, securing second place in the competition. For comparison, the RMSEs of the top-ranked, third-ranked, and official baseline systems are 0.364, 0.384, and 0.444, respectively.

cs.CL↗

Session-Level Spoken Language Assessment with a Multimodal Foundation Model via Multi-Target Learning

Spoken Language Assessment (SLA) estimates a learner's oral proficiency from spontaneous speech. The growing population of L2 English speakers has intensified the demand for reliable SLA, a critical component of Computer Assisted Language Learning (CALL). Existing efforts often rely on cascaded pipelines, which are prone to error propagation, or end-to-end models that often operate on a short audio window, which might miss discourse-level evidence. This paper introduces a novel multimodal foundation model approach that performs session-level evaluation in a single pass. Our approach couples multi-target learning with a frozen, Whisper ASR model-based speech prior for acoustic-aware calibration, allowing for jointly learning holistic and trait-level objectives of SLA without resorting to handcrafted features. By coherently processing the entire response session of an L2 speaker, the model excels at predicting holistic oral proficiency. Experiments conducted on the Speak & Improve benchmark demonstrate that our proposed approach outperforms the previous state-of-the-art cascaded system and exhibits robust cross-part generalization, producing a compact deployable grader that is tailored for CALL applications.

cs.CL↗

A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions

Automated speaking assessment (ASA) on opinion expressions is often hampered by the scarcity of labeled recordings, which restricts prompt diversity and undermines scoring reliability. To address this challenge, we propose a novel training paradigm that leverages a large language models (LLM) to generate diverse responses of a given proficiency level, converts responses into synthesized speech via speaker-aware text-to-speech synthesis, and employs a dynamic importance loss to adaptively reweight training instances based on feature distribution differences between synthesized and real speech. Subsequently, a multimodal large language model integrates aligned textual features with speech signals to predict proficiency scores directly. Experiments conducted on the LTTC dataset show that our approach outperforms methods relying on real data or conventional augmentation, effectively mitigating low-resource constraints and enabling ASA on opinion expressions with cross-modal information.

cs.CL↗

Acoustically Precise Hesitation Tagging Is Essential for End-to-End Verbatim Transcription Systems

Verbatim transcription for automatic speaking assessment demands accurate capture of disfluencies, crucial for downstream tasks like error analysis and feedback. However, many ASR systems discard or generalize hesitations, losing important acoustic details. We fine-tune Whisper models on the Speak & Improve 2025 corpus using low-rank adaptation (LoRA), without recourse to external audio training data. We compare three annotation schemes: removing hesitations (Pure), generic tags (Rich), and acoustically precise fillers inferred by Gemini 2.0 Flash from existing audio-transcript pairs (Extra). Our challenge system achieved 6.47% WER (Pure) and 5.81% WER (Extra). Post-challenge experiments reveal that fine-tuning Whisper Large V3 Turbo with the "Extra" scheme yielded a 5.5% WER, an 11.3% relative improvement over the "Pure" scheme (6.2% WER). This demonstrates that explicit, realistic filled-pause labeling significantly enhances ASR accuracy for verbatim L2 speech transcription.

cs.CL↗