arXiv · 2611.04343
Do Retrieval Ranking Models Capture Event Order? A Study of Timeline-Constrained Retrieval
Abstract
Semantic retrieval models aim to match documents to a query's "meaning", yet it remains unclear whether they successfully encode temporal relations between events in their ranking mechanisms. We investigate this question by studying timeline-constrained document retrieval, where a query is an ordered sequence of natural-language event statements while a relevant document should match those events in a compatible temporal progression. We then test whether widely used dense retrievers and neural rerankers capture such constraints, and whether explicitly preserving temporal order improves ranking, using a controlled setting over a novel focused evaluation collection. Experiments show that state-of-the-art semantic retrievers and rerankers provide only limited improvements over lexical retrieval, suggesting that their semantic matching capabilities do not reliably capture temporal constraints. We then compare a timeline-aware method with both semantic matching and standard rerankers and find that it yields substantial improvements. This demonstrates that considering temporal order can provide a strong ranking signal in this setting, suggesting future research on improving temporal modeling in semantic retrievers.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Alon Eirew, Lidan Avisar, Kfir Bar, Ido Dagan. 2026-11-03. Do Retrieval Ranking Models Capture Event Order? A Study of Timeline-Constrained Retrieval. https://arxiv.org/abs/2611.04343
Cite the original work for its findings. Save a collection to share your selection of sources.