Searcharxiv⌕ Search

arXiv subjects

Sudhasmita Swain

Publications and source records attributed to Sudhasmita Swain.

1 recordsLinked to original sources

CondRelBench: A Mixed-Domain Benchmark for Condition-Scoped Relation Understanding

Relation extraction systems often identify what a relation is, but not the condition under which it holds. This can collapse condition-bound claims into unsupported generalizations, a failure we call relation overgeneralization. We introduce CondRelBench, a mixed-domain benchmark for evaluating condition-scoped relation understanding. Each claim is represented as (E1, R, E2 | C), where C is the condition licensing the relation. Using 3,000 synthetic paragraphs, 200 PubMed full-text papers, and 695 Reddit posts, we construct five probe variants per tuple that preserve, replace, remove, or contradict the condition, or contradict the relation itself. Models classify each probe as Supported, Not Supported, or Not Enough Evidence. We also introduce Overgeneralization Rate (OGR) and Scope Preservation Score (SPS) to measure false support for altered claims and consistency across probe families. Across prompted LLMs, fine-tuned LLMs, and encoder models, we find that Accuracy and Macro F1 often hide scope failures. Models frequently support the original scoped claim but fail when its condition is removed or altered. Domain-specific encoders achieve the strongest scope preservation, while instruction-tuned LLMs often show higher overgeneralization. Synthetic supervision helps but does not close the gap to real-world PubMed and Reddit performance.

cs.CL↗