Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · cs/0510054

The Nature of Novelty Detection

Le Zhao·Min Zhang·Shaoping Ma
View original source ↗Read source document ↗Export metadata ↓

Abstract

Sentence level novelty detection aims at reducing redundant sentences from a sentence list. In the task, sentences appearing later in the list with no new meanings are eliminated. Aiming at a better accuracy for detecting redundancy, this paper reveals the nature of the novelty detection task currently overlooked by the Novelty community $-$ Novelty as a combination of the partial overlap (PO, two sentences sharing common facts) and complete overlap (CO, the first sentence covers all the facts of the second sentence) relations. By formalizing novelty detection as a combination of the two relations between sentences, new viewpoints toward techniques dealing with Novelty are proposed. Among the methods discussed, the similarity, overlap, pool and language modeling approaches are commonly used. Furthermore, a novel approach, selected pool method is provided, which is immediate following the nature of the task. Experimental results obtained on all the three currently available novelty datasets showed that selected pool is significantly better or no worse than the current methods. Knowledge about the nature of the task also affects the evaluation methodologies. We propose new evaluation measures for Novelty according to the nature of the task, as well as possible directions for future study.

Explore related subjects

cs.IRcs.CL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Le Zhao, Min Zhang, Shaoping Ma. 2005-10-19. The Nature of Novelty Detection. https://arxiv.org/abs/cs/0510054

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

DAT: Dynamic Alpha Tuning for Hybrid Retrieval in Retrieval-Augmented Generation

Hybrid retrieval techniques in Retrieval-Augmented Generation (RAG) systems enhance information retrieval by combining dense and sparse (e.g., BM25-based) retrieval methods. However, existing approaches struggle with adaptability, as fixed weighting schemes fail to adjust to different queries. To address this, we propose DAT (Dynamic Alpha Tuning), a novel hybrid retrieval framework that dynamically balances dense retrieval and BM25 for each query. DAT leverages a large language model (LLM) to evaluate the effectiveness of the top-1 results from both retrieval methods, assigning an effectiveness score to each. It then calibrates the optimal weighting factor through effectiveness score normalization, ensuring a more adaptive and query-aware weighting between the two approaches. Empirical results show that DAT consistently significantly outperforms fixed-weighting hybrid retrieval methods across various evaluation metrics. Even on smaller models, DAT delivers strong performance, highlighting its efficiency and adaptability.

cs.IR↗
arXiv2026

When Reviews Are Scarce: Structure-Aware Review Imputation for Recommendation in Realistic Sparse Review Settings

Textual reviews enrich recommender systems with fine-grained preference signals and enhanced explainability. However, in real-world scenarios, users rarely leave reviews, resulting in severe sparsity that undermines the effectiveness of review-aware recommenders under partial observability. A natural remedy is to synthesize the missing reviews to restore the textual signal these recommenders rely on. Yet, conventional imputation techniques, such as matrix completion and LLM-based augmentation, either lose contextualized semantics by embedding texts into vectors or overlook structural dependencies on the user-item graph. To address these shortcomings, we propose TWISTER (ToWards Imputation on Sparsity with Textual Edge Graph Representation), a unified framework for sparse-text recommendation that synthesizes missing reviews by jointly modeling semantic and structural signals. Specifically, we represent user-item interactions as a Textual-Edge Graph (TEG), treating reviews as edge attributes. To capture relational context, we construct line-graph views and employ an LLM as a graph-aware aggregator. For each interaction lacking a textual review, our model aggregates natural-language representations from neighboring interactions to generate a coherent and personalized review. Experiments on multiple datasets show that TWISTER consistently outperforms traditional numeric, graph-based, and LLM baselines, delivering higher-quality imputed reviews and, more importantly, improved downstream recommendation. In summary, TWISTER is a drop-in, structure-aware imputer that recovers the textual signal that review-aware recommenders need when reviews are scarce.

cs.IR↗
arXiv2026

Explaining Group Recommendations via Counterfactuals

Group recommender systems help users make collective choices but often lack transparency, leaving group members uncertain about why items are suggested. Existing explanation methods focus on individuals, offering limited support for groups where multiple preferences interact. In this work, we introduce and formalize the problem of finding counterfactual explanations for group recommendations. We introduce a comprehensive set of item- and explanation-level metrics that capture different aspects, such as relevance, popularity, explanation utility, interpretability, and fairness. We design five algorithms that employ techniques such as search-space pruning, greedy selection, early stopping, and incremental refinement to reduce computational cost and the number of recommender calls while producing high-quality explanations. The algorithms consider different trade-offs; for example, some prioritize computational efficiency at the expense of explanation quality, while others focus on improving explanation quality at a higher computational cost. Experiments on real-world datasets analyze these trade-offs in terms of explanation size, computational cost, utility, and fairness.

cs.IR↗