Searcharxiv⌕ Search

arXiv subjects

Amin Saberi

Publications and source records attributed to Amin Saberi.

At least 19 recordsLinked to original sources

Speculative Self-Distillation enables Efficient Knowledge Internalization

Many real-world applications require language models to internalize knowledge that was absent from pretraining, to incorporate continually updated facts, or to learn new domain skills. Self-distillation has emerged as a natural approach for this task. A model conditioned on the source material acts as the teacher, and an unconditioned student is trained to reproduce the teacher's behavior closed-book. We propose Speculative Self-Distillation (SSD), an efficient per-token mixed-policy method for self-distillation. The student generates each rollout by default, while the teacher intervenes only at positions where its next-token distribution substantially diverges from the student's. Unlike off-policy distillation with teacher-generated prefixes, this design keeps training rollouts close to the student's test-time behavior, while avoiding the low-signal states that fully on-policy rollouts drift into. In effect, SSD lets the teacher steer the rollout exactly where the privileged context changes the prediction. On three closed-book QA benchmarks measuring fact recall, knowledge updates, and skill acquisition, SSD nearly matches on-policy distillation while using 38-69% fewer supervised tokens and consistently outperforms off-policy distillation on the held-out test set. On a forgetting stress test, SSD retains the most general capability while also achieving the best in-domain accuracy and using less supervision than the on-policy baseline. Overall, SSD establishes a new accuracy-efficiency-forgetting Pareto frontier for self-distillation.

cs.LG↗

From signaling to interviews in random matching markets

In two-sided labor markets, interviews precede matches. Rising interview volumes in medical residency markets have motivated signaling mechanisms that let applicants express interest in a limited number of partners. We study how signaling reduces interviews in centralized random matching markets with post-interview shocks. We focus on interim stability, which ensures that agents do not regret not interviewing with each other. A matching is almost interim stable if it becomes interim stable after removing a vanishing fraction of agents. We consider markets with $n$ agents in which agents on the short side, long side, or both sides signal their top $d$ preferred partners. An interview occurs whenever at least one party signals the other. When the probability of a nonnegative interview shock is bounded away from zero and $d=ω(1)$, short-side signaling leads to almost interim stable matchings. Long-side signaling can fail in strongly imbalanced markets when pre-interview scores outweigh interview shocks. In balanced markets without interview shocks, both-side signaling fails to achieve almost interim stability when $d=o(\log n)$. For larger $d \ge Ω(\log^2 n)$, under the same probability condition, short-side signaling achieves perfect interim stability, while long-side signaling can fail in imbalanced markets. We extend these findings to propose a signaling mechanism for multi-tiered random markets and identify conditions for incentive compatibility. A technical contribution is the analysis of a message-passing algorithm for stable matchings on trees, together with local availability certificates used to study interim stability on general graphs.

cs.GT↗

Learning Query Encoders Can Be Hard Even When Vector Retrieval Is Geometrically Easy

Efficient vector retrieval requires both a corpus geometry that supports retrieving the right documents through vector similarity, and a query encoder that can embed queries near their desired documents in the embedding space. Recent work has studied geometric capacity through the lens of the minimum embedding dimension needed to realize all top-$k$ answer sets of $n$ documents. We study a different notion of geometric capacity--the maximum recall achievable for a frozen document index--and explore whether learned query encoders can reach this ceiling. On several real-world retrieval benchmarks, we show that retrieval quality of single-vector query encoders often lies far below what the document indices can support. Motivated by this observation, we give theoretical evidence that learning query encoders can be computationally hard. In particular, we construct a retrieval task that (1) admits a query encoder with perfect recall which is representable by a small one-hidden-layer ReLU network, but (2) any statistical-query learner (a class capturing learners that access training data through aggregate statistics) provably requires exponentially many statistical queries to achieve non-trivial recall advantage over the random baseline $k/n$. Taken together, our results suggest substantial unrealized geometric capacity in retrieval benchmarks and establish query encoder learnability as a possible barrier in embedding-based retrieval.

cs.IR↗

Kadison--Singer partitions and Bilu--Linial graph signings in polynomial time

We prove two main algorithmic results in spectral discrepancy. First, we give a deterministic polynomial-time rounding theorem for rational positive semidefinite matrices of arbitrary rank. The algorithm starts from any rational fractional signing and assigns one sign per original matrix. Its discrepancy is less than $3.37\,\|\sum_i \mathrm{Tr}(A_i)A_i\|^{1/2}$. This yields Kadison--Singer half-partitions with error below $1.69\sqrt{\varepsilon}$, as well as deterministic graph signings that control signed adjacency and signed degrees simultaneously. The proof builds on the spectral-potential method of Ezeunala and Jiang (2026) and introduces a new way to choose rounding directions. We prove polynomial bit complexity for the rounding procedure. Second, we give a Las Vegas algorithm for the Bilu--Linial signing problem on an arbitrary prescribed graph. If $G$ has $n$ vertices and maximum degree $Δ\ge3$, the algorithm terminates almost surely. It uses fewer than $100n^3$ insertion attempts in expectation and returns a signing with $\|A_s\|<2\sqrt{2(Δ-1)}$. For bipartite graphs its one-sided form gives the sharp universal bound $\|A_s\|<2\sqrt{Δ-1}$. The algorithm builds the signing by inserting vertices and recursively deleting and restoring neighbors after rejected insertions. In the analysis, the $\sqrt2$ gap to the Bilu--Linial conjecture comes from a factor of two in the bound for vertex deletions in the two-sided case. On a $d$-regular bipartite Ramanujan base the same signing produces a Ramanujan $2$-lift of that prescribed base.

cs.DS↗

MACS: A Hybrid Multi-Agent Framework for Reliable Conversational E-Commerce Recommendation

Conversational recommendation for e-commerce is increasingly mediated by large language models (LLMs), yet many real-world deployments operate under a stricter requirement: recommendations must be drawn only from a merchant's fixed catalog, without web search or unsupported product claims. In this setting, the main challenge is reliability under hard constraints: the system must satisfy user requirements, remain grounded in available inventory, and preserve preferences across multiple conversational turns. We present MACS (Multi-Agent Commerce System), a hybrid multi-agent framework for reliable conversational recommendation in fixed-catalog settings. MACS uses LLMs for language-facing tasks such as interpreting user requests, eliciting preferences, and generating responses, while correctness-critical operations, including product retrieval, hard-constraint filtering, brand exclusion, and progressive relaxation, are executed deterministically by the merchant agent. A session-persistent preference layer tracks constraints across turns, enabling consistent handling of budget overwrites and exclusion reversals. On a 140-query single-turn benchmark, MACS achieves the highest pass rate (87.1%) and perfect brand compliance (1.000). On a 10-scenario multi-turn benchmark, MACS achieves the strongest macro Pass@5 (72% vs. 56% GPT+Catalog / 52% Gemini+Catalog) with zero constraint drift. The advantage is sharpest on exclusion reversal (100% vs. 20% / 0%) and constraint accumulation (100% vs. 60% / 40%). Mean judged response quality is similar across systems (0.751 vs. 0.736). These results suggest that hybrid architectures combining deterministic constraint enforcement with session-persistent preference tracking provide stronger reliability-oriented performance than catalog-bound prompt-only baselines in the fixed-catalog merchant setting.

cs.IR↗

Learning Distributions from Multiple Data Providers

Motivated by learning from heterogeneous and overlapping data providers, we study a stylized model of distribution learning from restricted conditional samples. The goal is to learn an unknown distribution $p$ on a finite domain $[n]$. The learner is given a fixed family of queryable sets $\mathscr{S} \subseteq 2^{[n]}$, and each query to $S \in \mathscr{S}$ returns an independent sample from the conditional distribution $p(\cdot \mid S)$. Learnability is governed by the co-occurrence graph associated with $\mathscr{S}$: two domain elements are adjacent if they appear together in some queryable set. Pointwise consistency is achievable when this graph is connected on the target support. PAC learning requires more: it is possible when the co-occurrence graph is complete. The optimal sample complexity of PAC learning ranges from nearly linear to quadratic. Every query family with complete co-occurrence graph admits sample complexity $\widetilde O(n^2/ε^2)$, and this bound is tight in the worst case. On the other hand, if $[n]$ is queryable then ordinary sampling improves the bound to $Θ(n/ε^2)$, and this cannot be improved further even if every set is queryable. More generally, we identify hierarchical comparabilityas a sufficient structural condition on $\mathscr S$ under which the optimal complexity is nearly linear, $\widetilde Θ(n/ε^2)$, with pairwise query families as a canonical example. Finally, the full range of polynomial rates between linear and quadratic is attainable: for every $α\in (1,2)$, there exists a query family with optimal PAC rate $\widetilde Θ(n^α/ε^2)$.

cs.DS↗

Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation

Language models deployed in high-stakes roles can potentially favor certain entities, brands, or viewpoints, steering user decisions at scale. Such preferential biases can be introduced by any actor in the model's supply chain and are most dangerous when the model reveals its preference only on the relevant topic while behaving identically to its unmodified base on all other inputs. Recent work has shown that these biases can transfer through context distillation on semantically unrelated data, with the signal residing entirely in the soft logit distribution and remaining invisible to text-based inspection. However, the defender faces a fundamental asymmetry: without knowing the bias topic, no detection method can reliably surface a stealth preferential bias, regardless of whether it examines generated text, internal representations, or model weights. Here we introduce Distill to Detect (D2D), a method that surfaces hidden biases by distilling the distributional shift between a suspected model and its base into a cartridge (a KV-cache prefix adapter), concentrating the dominant divergence and amplifying the bias signal into generated text. We show that D2D successfully amplifies the hidden biases of stealth models to the extent that they can be reliably detected across multiple bias types. We also propose a theoretical framework that explains the efficacy of D2D through the lens of Fisher-weighted projection of the logit distribution shift, supported by empirical observations. By turning the capacity bottleneck of prefix-tuning adapters into a detection tool, D2D provides a practical building block for auditing hidden behaviors in deployed language models.

cs.CL↗

On Language Generation in the Limit with Bounded Memory

We study language generation in the limit under bounded memory. In this task, a learner observes examples from an unknown target language one at a time and must eventually output only new valid examples. Prior work assumes access to the entire history, a strong assumption since realistic algorithms retain limited past information. Classical work in learning theory shows memory constraints dramatically alter learnability; we extend this to language generation. First, we study memoryless generators. Under a mild enumeration restriction, every countable collection of infinite languages remains generable without memory. Without this restriction, we exactly characterize when memoryless generation is possible. For finite collections, we characterize the optimal minimax density achievable by memoryless generators -- the best density guaranteed against any collection of a given size. This combinatorial bound relies on Sperner's theorem and symmetric chain decompositions. We further show that a sliding window of the last $W$ examples does not improve this worst-case density, whereas allowing it to store $b$ adaptively chosen past examples improves the achievable density for every $b \geq 1$. Finally, we revisit identification in the limit, where the learner must converge to a single correct hypothesis for the target language. We focus on its incremental variant, where the learner remembers only its previous guess. Here, although exact identification fails on a collection of just three languages, a mild relaxation requiring convergence to an ``approximate'' version of the target is achievable for every finite collection. These results show bounded memory affects these tasks differently: generation remains achievable for every countable collection, while density and identification are confined to finite collections, with guarantees weakening as the collection grows.

cs.DS↗

Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards

Reinforcement learning with verifiable rewards (RLVR) is a practical, scalable way to improve large language models on math, code, and other structured tasks. However, we argue that many headline RLVR gains are not yet well validated because reports often conflate policy improvement with three confounds: (i) budget mismatch between RLVR and baseline evaluations, (ii) attempt inflation and calibration drift that convert abstentions into confident answers, and (iii) benchmark data contamination. Using budget-matched reproductions and partial-prompt contamination probes, we find that several widely cited gaps shrink substantially or disappear once budgets, prompts, and dataset versions are matched and contaminated sets are treated as memorization probes rather than evidence of reasoning. This does not mean that RLVR is ineffective, but it implies that current measurements often overstate capability gains and obscure reliability costs. We therefore propose a compact, tax-aware minimum standard for RLVR training and evaluation: budget-matched saturation curves with variance, calibration, and abstention tracking, a judge-robustness stress test when LLM judges are used, and an explicit contamination screen. With these controls, RLVR remains effective and deployable in verifiable domains, but reasoning gains should be treated as provisional without them.

cs.LG↗

Local Limits of Small World Networks

Small-world networks, known for high local clustering and short path lengths, are a fundamental structure in many real-world systems, including social, biological, and technological networks. We apply the theory of (marked) local convergence (also known as Benjamini-Schramm convergence) to derive the limiting behavior of the local structures for two commonly studied small-world network models: the Watts-Strogatz and the Kleinberg models. Establishing local convergence enables us to show that key network measures, such as clustering coefficient, PageRank, greedy maximal independent set, number of spanning trees and tree entropy, converge as network size increases, with their limits determined by the graph's local structure. Additionally, this framework facilitates the estimation of global phenomena, such as the size of the giant component under bond percolation and the closely related properties, the size of the epidemic and information cascades, using local information from small neighborhoods. Furthermore, we observe a critical change in the behavior of the limit exactly when the parameter governing long-range connections in the Kleinberg model crosses the threshold where decentralized search remains efficient, offering a new perspective on why decentralized algorithms fail in certain regimes.

math.PR↗

Entropy Guided Diversification and Preference Elicitation in Agentic Recommendation Systems

Users on e-commerce platforms can be uncertain about their preferences early in their search. Queries to recommendation systems are frequently ambiguous, incomplete, or weakly specified. Agentic systems are expected to proactively reason, ask clarifying questions, and act on the user's behalf, which makes handling such ambiguity increasingly important. In existing platforms, ambiguity led to excessive interactions and question fatigue or overconfident recommendations prematurely collapsing the search space. We present an Interactive Decision Support System (IDSS) that addresses ambiguous user queries using entropy as a unifying signal. IDSS maintains a dynamically filtered candidate product set and quantifies uncertainty over item attributes using entropy. This uncertainty guides adaptive preference elicitation by selecting follow-up questions that maximize expected information gain. When preferences remain incomplete, IDSS explicitly incorporates residual uncertainty into downstream recommendations through uncertainty-aware ranking and entropy-based diversification, rather than forcing premature resolution. We evaluate IDSS using review-driven simulated users grounded in real user reviews, enabling a controlled study of diverse shopping behaviors. Our evaluation measures both interaction efficiency and recommendation quality. Results show that entropy-guided elicitation reduces unnecessary follow-up questions, while uncertainty-aware ranking and presentation yield more informative, diverse, and transparent recommendation sets under ambiguous intent. These findings demonstrate that entropy-guided reasoning provides an effective foundation for agentic recommendation systems operating under uncertainty.

cs.AI↗

Near-Optimal Bayesian Online Assortment of Reusable Resources

Motivated by the applications of rental services in e-commerce, we consider revenue maximization in online assortment of reusable resources for a stream of arriving consumers with different types. We design competitive online algorithms with respect to the optimum online policy in the Bayesian setting, in which types are drawn independently from known heterogeneous distributions over time. In the regime where the minimum of initial inventories $c_0$ is large, our main result is a near-optimal $1-\min\left(\frac{1}{2},\sqrt{\log(c_0)/c_0}\right)$ competitive algorithm for the general case of reusable resources. Our algorithm relies on an expected LP benchmark for the problem, solves this LP, and simulates the solution through an independent randomized rounding. The main challenge is obtaining point-wise inventory feasibility in a computationally efficient fashion from these simulation-based algorithms. To this end, we use several technical ingredients to design $\textit{discarding policies}$ -- one for each resource. These policies handle the trade-off between the inventory feasibility under reusability and the revenue loss of each of the resources. However, discarding a unit of a resource changes the future consumption of other resources. To handle this new challenge, we also introduce $\textit{post-processing}$ assortment procedures that help with designing and analyzing our discarding policies as they run in parallel, which might be of independent interest. As a side result, by leveraging techniques from the literature on prophet inequality, we further show an improved near-optimal $1-1/\sqrt{c_0+3}$ competitive algorithm for the special case of non-reusable resources. We finally evaluate the performance of our algorithms using the numerical simulations on the synthetic data.

cs.DS↗

SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Models

Large reasoning models (LRMs) excel at complex reasoning tasks but typically generate lengthy sequential chains-of-thought, resulting in long inference times before arriving at the final answer. To address this challenge, we introduce SPRINT, a novel post-training and inference-time framework designed to enable LRMs to dynamically identify and exploit opportunities for parallelization during their reasoning process. SPRINT incorporates an innovative data curation pipeline that reorganizes natural language reasoning trajectories into structured rounds of long-horizon planning and parallel execution. By fine-tuning LRMs on a small amount of such curated data, the models learn to dynamically identify independent subtasks within extended reasoning processes and effectively execute them in parallel. Through extensive evaluations, we demonstrate that models fine-tuned with the SPRINT framework match the performance of reasoning models on complex domains such as mathematics while generating up to 39% fewer sequential tokens on problems requiring more than 8,000 output tokens. Finally, we observe consistent results transferred to two out-of-distribution tasks, namely GPQA and Countdown, with up to 45% and 65% reduction in average sequential tokens respectively for longer reasoning trajectories, while matching the performance of the fine-tuned reasoning model.

cs.AI↗

Robustness of Online Inventory Balancing to Inventory Shocks

In classic adversarial online resource allocation problems such as AdWords, customers arrive online while products are given offline with a fixed initial inventory. To ensure revenue guarantees under uncertainty, the decision maker must balance consumption across products. Based on this, the prevalent policy "inventory balancing (IB)" has proved to be optimal or near-optimal competitive in almost all classic settings. However, these models do not capture various forms of inventory shocks on the supply side, which play an important role in real-world online assortment and can significantly impact the revenue performance of the IB algorithm. Motivated by this paradigm, we introduce a variant of online assortment planning with inventory shocks. Our model considers adversarial exogenous shocks (where supply increases unpredictably) and allocation-coupled endogenous shocks (where an inventory reduction is triggered by the algorithms and re-adjusted after a usage duration), whose combination leads to non-monotonic inventory fluctuations. As our main result, we show the robustness of IB-type strategies against such shocks by designing a new family of optimal competitive algorithms called "Batched Inventory Balancing (BIB)." Using a novel randomized primal-dual method, we bound the competitive ratio of BIB against optimal offline. We show that with proper choice of a certain parameter, this competitive ratio is asymptotically optimal and converges to (1-1/e) as initial inventories grow, in contrast to the original IB which no longer achieves the optimal ratio in this new model. Moreover, we characterize BIB's competitive ratio parametric by its penalty function and show that it matches exactly the competitive ratio of IB without shocks. Our refined analysis reduces the dual construction to a combinatorial "interval assignment problem" whose algorithmic solution may be of independent interest.

cs.DS↗

Optimal Rounding for Two-Stage Bipartite Matching

We study two-stage bipartite matching, in which the edges of a bipartite graph on vertices $(B_1 \cup B_2, I)$ are revealed in two batches. In stage one, a matching must be selected from among revealed edges $E \subseteq B_1 \times I$. In stage two, edges $E^θ\subseteq B_2 \times I$ are sampled from a known distribution, and a second matching must be selected between $B_2$ and unmatched vertices in $I$. The objective is to maximize the total weight of the combined matching. We design polynomial-time approximations to the optimum online algorithm, achieving guarantees of $7/8$ for vertex-weighted graphs and $2\sqrt{2}-2 \approx 0.828$ for edge-weighted graphs under arbitrary distributions. Both approximation ratios match known upper bounds on the integrality gap of the natural fractional relaxation, improving upon the best-known approximation of 0.767 by Feng, Niazadeh, and Saberi for unweighted graphs whose second batch consists of independently arriving nodes. Our results are obtained via an algorithm that rounds a fractional matching revealed in two stages, aiming to match offline nodes (respectively, edges) with probability proportional to their fractional weights, up to a constant-factor loss. We leverage negative association (NA) among offline node availabilities -- a property induced by dependent rounding -- to derive new lower bounds on the expected size of the maximum weight matching in random graphs where one side is realized via NA binary random variables. Moreover, we extend these results to settings where we have only sample access to the distribution. In particular, $\text{poly}(n,ε^{-1})$ samples suffice to obtain an additive loss of $ε$ in the approximation ratio for the vertex-weighted problem; a similar bound holds for the edge-weighted problem with an additional (unavoidable) dependence on the scale of edge weights.

cs.DS↗

StorySage: Conversational Autobiography Writing Powered by a Multi-Agent Framework

Every individual carries a unique and personal life story shaped by their memories and experiences. However, these memories are often scattered and difficult to organize into a coherent narrative, a challenge that defines the task of autobiography writing. Existing conversational writing assistants tend to rely on generic user interactions and pre-defined guidelines, making it difficult for these systems to capture personal memories and develop a complete biography over time. We introduce StorySage, a user-driven software system designed to meet the needs of a diverse group of users that supports a flexible conversation and a structured approach to autobiography writing. Powered by a multi-agent framework composed of an Interviewer, Session Scribe, Planner, Section Writer, and Session Coordinator, our system iteratively collects user memories, updates their autobiography, and plans for future conversations. In experimental simulations, StorySage demonstrates its ability to navigate multiple sessions and capture user memories across many conversations. User studies (N=28) highlight how StorySage maintains improved conversational flow, narrative completeness, and higher user satisfaction when compared to a baseline. In summary, StorySage contributes both a novel architecture for autobiography writing and insights into how multi-agent systems can enhance human-AI creative partnerships.

cs.HC↗

When to Trust Context: Self-Reflective Debates for Context Reliability

Large language models frequently encounter conflicts between their parametric knowledge and contextual input, often resulting in factual inconsistencies or hallucinations. We propose Self-Reflective Debate for Contextual Reliability (SR-DCR), a lightweight framework that integrates token-level self-confidence with an asymmetric multi-agent debate to adjudicate such conflicts. A critic, deprived of context, challenges a defender who argues from the given passage; a judge model evaluates the debate and determines the context's reliability. The final answer is selected by combining the verdict with model confidence. Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently enhances robustness to misleading context while maintaining accuracy on trustworthy inputs, outperforming both classical debate and confidence-only baselines with minimal computational overhead. The code is available at https://github.com/smiles724/Self-Reflective-Debates.

cs.CL↗

Adaptive Approximation Schemes for Matching Queues

We study a continuous-time, infinite-horizon dynamic bipartite matching problem. Suppliers arrive according to a Poisson process; while waiting, they may abandon the queue at a uniform rate. Customers on the other hand must be matched upon arrival. The objective is to minimize the expected long-term average cost subject to a throughput constraint on the total match rate. Previous literature on dynamic matching focuses on "static" policies, where the matching decisions do not depend explicitly on the state of the supplier queues, achieving constant-factor approximations. By contrast, we design "adaptive" policies, which leverage queue length information, and obtain near-optimal polynomial-time algorithms for several classes of instances. First, we develop a bi-criteria fully polynomial-time approximation scheme for dynamic matching on networks with a constant number of queues--that computes a $(1-ε)$-approximation of the optimal policy in time polynomial in both the input size and $1/ε$. A key new technique is a hybrid LP relaxation, which combines static and state-dependent LP approximations of the queue dynamics, after a decomposition of the network. Networks with a constant number of queues are motivated by deceased organ donation schemes, where the supply types can be divided according to blood and tissue types. The above algorithm, combined with a careful cell decomposition gives an efficient polynomial-time approximation scheme for dynamic matching on Euclidean networks of fixed dimension. The Euclidean case is of interest in ride-hailing and spatial service platforms, where the goal is to fulfill as many trips as possible while minimizing driving distances.

cs.DS↗