Searcharxiv⌕ Search

arXiv subjects

Aashish Pokhrel

Publications and source records attributed to Aashish Pokhrel.

1 recordsLinked to original sources

Negation Through Affirmation: Leveraging Positive Evidence in Vision-Language Models

Frozen vision-language models often match concepts mentioned in a caption without reliably respecting their negation: a query for "a kitchen with no toaster" favors an image containing a toaster. Yet these models are substantially stronger at matching affirmative descriptions. We exploit this asymmetry with Negation-Aware Evidence Scoring (NAES), which reformulates negated descriptions into affirmative queries and combines their evidence by caption structure. A lightweight text decomposer, NegSplit, extracts negated spans and the remaining affirmative context. For captions with an affirmative remainder, NAES matches that content and applies a presence-based correction for the denial. When no affirmative content remains, it queries the denied object's presence and reflects the evidence around a reference to score the negated description. This yields substantial accuracy gains: with LAION-2B CLIP, NAES achieves 82.4% accuracy on COCO-MCQ and 93.0% on VOC2007-MCQ, surpassing KTN by 2.9 and 0.9 percentage points; with OpenAI CLIP, it exceeds PeakPatch accuracy by 4.4 points on COCO-MCQ and 26.0 points on VOC2007-MCQ. NAES also improves negated-description retrieval on COCO and MSR-VTT. The strongest multiple-choice results use competing descriptions to center and align evidence; retrieval improvements persist without this conditioning. These findings show that separating affirmative matching from denied-object evidence can substantially improve negation-aware ranking.

cs.CV↗