arXiv subjects

Jackie Doremus

Publications and source records attributed to Jackie Doremus.

2 recordsLinked to original sources

arXiv2026

Who Are We Recommending To? Recommender Systems in the Agentic Web

For two decades, recommender systems have been designed under the assumption that a human directly consumes each recommendation: receiving, interpreting, and acting upon it. The emergence of AI agents powered by large language models challenges this assumption. In the emerging Agentic Web [ 28 ], autonomous agents increasingly act on behalf of users, e.g., browsing, comparing, negotiating, and executing transactions, raising a central question: who is the receiver of a recommendation? In this position paper, we argue that the recommendation paradigm is undergoing a bifurcation. In delegable contexts, such as routine purchases, travel, and constrained transactional tasks, the primary operational consumer of recommendations is shifting from the human to the agent, requiring new optimization objectives, interaction protocols, and evaluation criteria. In experiential contexts, such as entertainment, art, and other subjective or high-stakes choices, humans remain the final judge of relevance, though agents may assist through pre-filtering and curation. We introduce a delegation spectrum that characterizes recommendation contexts along factors such as preference specifiability, outcome verifiability, and decision stakes, and we outline a research agenda spanning agent preference modeling, dual-audience optimization, and the emerging agent attention economy. We further discuss the implications of this shift for the design and evaluation of recommender systems

cs.IR↗

arXiv2025

Evaluating Podcast Recommendations with Profile-Aware LLM-as-a-Judge

Evaluating personalized recommendations remains a central challenge, especially in long-form audio domains like podcasts, where traditional offline metrics suffer from exposure bias and online methods such as A/B testing are costly and operationally constrained. In this paper, we propose a novel framework that leverages Large Language Models (LLMs) as offline judges to assess the quality of podcast recommendations in a scalable and interpretable manner. Our two-stage profile-aware approach first constructs natural-language user profiles distilled from 90 days of listening history. These profiles summarize both topical interests and behavioral patterns, serving as compact, interpretable representations of user preferences. Rather than prompting the LLM with raw data, we use these profiles to provide high-level, semantically rich context-enabling the LLM to reason more effectively about alignment between a user's interests and recommended episodes. This reduces input complexity and improves interpretability. The LLM is then prompted to deliver fine-grained pointwise and pairwise judgments based on the profile-episode match. In a controlled study with 47 participants, our profile-aware judge matched human judgments with high fidelity and outperformed or matched a variant using raw listening histories. The framework enables efficient, profile-aware evaluation for iterative testing and model selection in recommender systems.

cs.IR↗