Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.24030

When Does Evidence-Graph Structure Predict Answer Correctness? Label-Free Abstention for Structured Retrieval

Anjan Goswami
View original source ↗Read source document ↗Export metadata ↓

Abstract

Retrieval systems return scored answers, but a high score does not show that an answer should be trusted. The proposed training-free abstention method applies the boundary cut to the selected evidence in an evidence graph. The signal is the weighted cut between selected and unselected candidates and requires no correctness labels or confidence-model training. On MemoryAgentBench (MAB) FactConsolidation multi-hop, with unresolved queries ranked last for every selector, it raises area under the precision-coverage curve from 0.736 for the best score baseline to 0.785 (+0.049, 95% CI [+0.008, +0.092]); under the native evaluation, where score baselines keep their unresolved values, the gap is +0.108. A zero-cut subset contains 71 of 400 queries at 100% observed precision. In this pipeline, the subset corresponds to structural closure; controlled analyses attribute much of the remaining separation to resolution status, with little benefit from continuous edge weights. Across external benchmarks, usefulness varies with graph construction, edge semantics, and relation; typed structure alone does not ensure useful confidence. A supervised gradient-boosted selector achieves 0.814 versus the boundary cut's 0.785 when the retrieval stack matches. These findings provide empirical guidance for using available evidence graphs for lightweight, auditable post-retrieval abstention without additional model inference.

Explore related subjects

cs.IRcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Anjan Goswami. 2026-10-16. When Does Evidence-Graph Structure Predict Answer Correctness? Label-Free Abstention for Structured Retrieval. https://arxiv.org/abs/2610.24030

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Reproducing LightMem: When Does Memory Construction Help Over Raw-Turn Retrieval?

Long-term conversational agents require access to information from earlier interactions, such as a user's preferences, past requests, or previously mentioned facts. Repeatedly providing the full dialogue history can be expensive as conversations grow, so many memory approaches instead transform past interactions into compact entries that can be retrieved when needed. LightMem is a recent lightweight memory-management approach that reports strong effectiveness while maintaining relatively low construction cost. However, it still relies on a separate constructed memory representation and is evaluated with only one retriever, leaving unclear how sensitive its results are to retriever choice and whether memory construction provides sufficient benefits over retrieving directly from raw turns. In this study, we reproduce LightMem and compare it with Naive RAG, which retrieves directly from raw user turns. We recover LightMem's main configuration trend, but find that retriever choice substantially affects answer accuracy: changing only the retriever over a fixed LightMem store shifts answer accuracy from 58.1\% to 75.5\%. Retrieval over constructed memory also does not consistently outperform raw-turn retrieval. At matched retrieval depths, Naive RAG generally achieves higher answer accuracy, whereas LightMem has an advantage under tight answering-token budgets. Oracle-reference evaluation further suggests that memory construction may omit or alter answer-relevant information. Overall, LightMem provides a conditional context-efficiency advantage rather than a consistent answer accuracy advantage over Naive RAG. Strong retrieval over raw turns can match or outperform constructed memory as the answering-token budget increases, motivating future work on retrieval, reranking, query formulation, and their interaction with raw-turn and constructed memory representations.

cs.IR↗
arXiv2026

Is Serving-Time Language-Model Inference Necessary for Recommendation? Evidence from Generative and Agentic Settings

Language models (LMs) are increasingly used in recommender systems to generate recommendations or coordinate recommendation tools. However, does their strong performance imply that serving-time LM inference is necessary? We challenge this seemingly reasonable but unverified assumption. To investigate this question, we introduce recovery-based mechanism auditing, which tests whether independent alternatives can achieve the accuracy of LM-based recommender systems without the audited inference mechanisms under controlled information and evaluation conditions. For generative recommendation, we propose LIME-Rec, which combines sequential, collaborative, and offline semantic information without autoregressive LM decoding. On three Amazon Reviews datasets (Beauty, Toys, and Sports), LIME-Rec achieves the highest reported mean in all 12 dataset-metric combinations, with up to a 15.2% relative improvement in Recall@10 over GRAM, an LM-based generative recommender. For agentic recommendation, we propose DART-Rec, which replaces LM-based tool control with deterministic fusion and fixed expert selection. On 1,000 Beauty users, given the same candidate items and expert scores, deterministic fusion achieves a 16.7% relative improvement in Recall@10 over a fine-tuned LM controller. With at most two tool calls, fixed expert selection matches the LM controllers' Hit@10 outcomes for every user in 17 of 18 comparisons. These findings demonstrate that strong recommendation performance does not necessarily require serving-time LM inference: under the evaluated task and information conditions, alternatives without the specified inference mechanisms can match or exceed the accuracy of their LM-based counterparts. To support reproducibility, code is available at https://github.com/Double-wk/LIME-Rec.

cs.IR↗
arXiv2026

P$^3$Rec: Distilling Prior--Posterior Preference Reasoning for LLM-based Recommendation

Large language models (LLMs) exhibit strong semantic understanding and preference reasoning capabilities, offering new opportunities for user modeling in recommender systems. Existing LLM-as-Enhancer methods typically distill LLM-derived preference knowledge into lightweight recommenders to avoid costly online LLM inference. However, they often construct distillation knowledge from only one perspective. Prior preference captures users' stable and consistent interests but provides limited guidance for the current decision, whereas posterior preference reveals target-relevant fine-grained interests but may rely excessively on target clues. To address these limitations, we propose P$^3$Rec, a framework that jointly extracts and internalizes complementary prior and posterior preference reasoning knowledge. Specifically, P$^3$Rec first derives target-agnostic prior preferences and target-conditioned posterior preferences from the user side, while further extracting item-centric preference representations from item semantics and predecessor interactions. It then progressively internalizes prior and posterior knowledge into behavioral representations through prior preference absorption and posterior-guided preference distillation. Since the resulting comprehensive preference representation may not always provide an equally decisive retrieval direction, P$^3$Rec further characterizes historical interest dispersion with interest entropy and adaptively calibrates the user representation before contrastive retrieval optimization. In this way, P$^3$Rec achieves more complete preference reasoning while preserving efficient recommendation. Extensive experiments on multiple public datasets demonstrate its effectiveness.

cs.IR↗