Searcharxiv⌕ Search

arXiv subjects

Sau Lai Yip

Publications and source records attributed to Sau Lai Yip.

2 recordsLinked to original sources

Bridge: Database-Grounded Interactive Disambiguation for Text-to-SQL

Ambiguity remains a persistent challenge in text-to-SQL, where an underspecified natural-language question may admit multiple plausible SQL interpretations. Recent work attempts to recover the user's intended interpretation through iterative clarification, but typically relies heavily on LLM parametric knowledge for ad hoc reasoning, leading to limited ambiguity-resolution effectiveness and high LLM cost. We further observe that many ambiguity errors stem from ambiguous column grounding, which remains poorly handled. Therefore, we propose Bridge, a database-grounded disambiguation framework built on an existing iterative protocol. Bridge supplies the LLM's ambiguity detection with offline-distilled knowledge about potential ambiguity relations among columns in the target database. Offline, Bridge constructs a Column Ambiguity Graph by first identifying potentially ambiguous column pairs using complementary database-grounded signals, followed by LLM-based verification. Online, columns and values mentioned in a previous incorrect SQL prediction are used as anchors to retrieve alternatives from the Column Ambiguity Graph and an offline-constructed value similarity index, respectively. The results are integrated for further pruning and consolidated for subsequent ambiguity detection. Extensive experiments on BIRD-dev without the benchmark-provided evidence show that, at comparable LLM cost, Bridge outperforms the strongest baseline by 11.60% in execution accuracy and improves the recall and precision of column usage in the final SQL by 5.25% and 12.37%, respectively.

cs.DB↗

MedBookVQA: A Systematic and Comprehensive Medical Benchmark Derived from Open-Access Book

The accelerating development of general medical artificial intelligence (GMAI), powered by multimodal large language models (MLLMs), offers transformative potential for addressing persistent healthcare challenges, including workforce deficits and escalating costs. The parallel development of systematic evaluation benchmarks emerges as a critical imperative to enable performance assessment and provide technological guidance. Meanwhile, as an invaluable knowledge source, the potential of medical textbooks for benchmark development remains underexploited. Here, we present MedBookVQA, a systematic and comprehensive multimodal benchmark derived from open-access medical textbooks. To curate this benchmark, we propose a standardized pipeline for automated extraction of medical figures while contextually aligning them with corresponding medical narratives. Based on this curated data, we generate 5,000 clinically relevant questions spanning modality recognition, disease classification, anatomical identification, symptom diagnosis, and surgical procedures. A multi-tier annotation system categorizes queries through hierarchical taxonomies encompassing medical imaging modalities (42 categories), body anatomies (125 structures), and clinical specialties (31 departments), enabling nuanced analysis across medical subdomains. We evaluate a wide array of MLLMs, including proprietary, open-sourced, medical, and reasoning models, revealing significant performance disparities across task types and model categories. Our findings highlight critical capability gaps in current GMAI systems while establishing textbook-derived multimodal benchmarks as essential evaluation tools. MedBookVQA establishes textbook-derived benchmarking as a critical paradigm for advancing clinical AI, exposing limitations in GMAI systems while providing anatomically structured performance metrics across specialties.

cs.AI↗