arXiv subjects

Yufei Tao

Publications and source records attributed to Yufei Tao.

At least 19 recordsLinked to original sources

arXiv2026

Distributed Learning with Adversarial Gradient Perturbations

Privacy concerns in distributed learning often lead clients to return intentionally altered gradient information. We consider the problem of learning convex and $L$-smooth functions under adversarial gradient perturbation, where a client's gradient reply to a server query can deviate arbitrarily from the true gradient subject to a distance bound. Our study focuses on two fundamental questions: (i) what is the smallest achievable sub-optimality gap (i.e., excess error in optimization) under such responses, and (ii) how many queries are sufficient to guarantee a given sub-optimality gap? We establish tight feasibility thresholds on the sub-optimality gap and provide algorithms that achieve these thresholds with provable query complexity guarantees.

cs.LG↗

arXiv2026

No-Worse Context-Aware Decoding: Preventing Neutral Regression in Context-Conditioned Generation

Large language models (LLMs) can answer questions and summarize documents when conditioned on external contexts (e.g., retrieved evidence), yet context use remains unreliable: models may overwrite an already-correct output (neutral regression) even when the context is non-informative. We formalize neutral regression as a do-no-harm requirement and quantify it by measuring accuracy drops on baseline-correct items under answer-consistent contexts. We propose No-Worse Context-Aware Decoding (NWCAD), a decode-time adapter built on a two-stream setup with a two-stage gate: it backs off to no-context decoding when the context is non-informative, and otherwise uses context-conditioned decoding with a CAD-style fallback under uncertainty. We evaluate NWCAD on benchmarks that separate do-no-harm reliability from context utilization (accuracy gains on genuinely helpful contexts). NWCAD prevents neutral regression on baseline-correct items while preserving strong context-driven accuracy on helpful contexts.

cs.CL↗

arXiv2025

CAPO: Confidence Aware Preference Optimization Learning for Multilingual Preferences

Preference optimization is a critical post-training technique used to align large language models (LLMs) with human preferences, typically by fine-tuning on ranked response pairs. While methods like Direct Preference Optimization (DPO) have proven effective in English, they often fail to generalize robustly to multilingual settings. We propose a simple yet effective alternative, Confidence-Aware Preference Optimization (CAPO), which replaces DPO's fixed treatment of preference pairs with a dynamic loss scaling mechanism based on a relative reward. By modulating the learning signal according to the confidence in each preference pair, CAPO enhances robustness to noisy or low-margin comparisons, typically encountered in multilingual text. Empirically, CAPO outperforms existing preference optimization baselines by at least 16% in reward accuracy, and improves alignment by widening the gap between preferred and dispreferred responses across languages.

cs.CL↗

arXiv2025

Proximity Graphs for Similarity Search: Fast Construction, Lower Bounds, and Euclidean Separation

Proximity graph-based methods have emerged as a leading paradigm for approximate nearest neighbor (ANN) search in the system community. This paper presents fresh insights into the theoretical foundation of these methods. We describe an algorithm to build a proximity graph for $(1+\epsilon)$-ANN search that has $O((1/\epsilon)^\lambda \cdot n \log \Delta)$ edges and guarantees $(1/\epsilon)^\lambda \cdot \text{polylog }\Delta$ query time. Here, $n$ and $\Delta$ are the size and aspect ratio of the data input, respectively, and $\lambda = O(1)$ is the doubling dimension of the underlying metric space. Our construction time is near-linear to $n$, improving the $\Omega(n^2)$ bounds of all previous constructions. We complement our algorithm with lower bounds revealing an inherent limitation of proximity graphs: the number of edges needs to be at least $\Omega((1/\epsilon)^\lambda \cdot n + n \log \Delta)$ in the worst case, up to a subpolynomial factor. The hard inputs used in our lower-bound arguments are non-geometric, thus prompting the question of whether improvement is possible in the Euclidean space (a key subclass of metric spaces). We provide an affirmative answer by using geometry to reduce the graph size to $O((1/\epsilon)^\lambda \cdot n)$ while preserving nearly the same query and construction time.

cs.DS↗

arXiv2025

"Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Models

Large language models are capable of leveraging both contextual and parametric knowledge but how they prioritize and integrate these sources remains underexplored. We introduce CoPE, a novel evaluation framework that systematically measures contextual knowledge (CK) and parametric knowledge (PK) across models and languages. Using our MultiWikiAtomic dataset in English, Spanish, and Danish, we analyze how large language models (LLMs) integrate context, prioritize information, and incorporate PK in open-ended question answering. Our analysis uncovers a phenomenon we call lost-in-the-later, where LLMs tend to overlook or deprioritize information that appears later in a given context, revealing a strong positional bias that affects contextual grounding. We further find that reasoning models, as well as non-reasoning models prompted with chain-of-thought (CoT), use context even less than non-reasoning models without CoT and fail to mitigate the lost-in-the-later effect. CoT prompting, in particular, results in lower recall and shorter responses, leading to degraded contextual grounding. Based on these insights, we design prompt-based methods to effectively leverage input context. A case study applying CoPE to summarization demonstrates that CK-informed prompting improves factual grounding and reduces hallucination.

cs.CL↗

arXiv2025

Monotone Classification with Relative Approximations

In monotone classification, the input is a multi-set $P$ of points in $\mathbb{R}^d$, each associated with a hidden label from $\{-1, 1\}$. The goal is to identify a monotone function $h$, which acts as a classifier, mapping from $\mathbb{R}^d$ to $\{-1, 1\}$ with a small {\em error}, measured as the number of points $p \in P$ whose labels differ from the function values $h(p)$. The cost of an algorithm is defined as the number of points having their labels revealed. This article presents the first study on the lowest cost required to find a monotone classifier whose error is at most $(1 + \epsilon) \cdot k^*$ where $\epsilon \ge 0$ and $k^*$ is the minimum error achieved by an optimal monotone classifier -- in other words, the error is allowed to exceed the optimal by at most a relative factor. Nearly matching upper and lower bounds are presented for the full range of $\epsilon$. All previous work on the problem can only achieve an error higher than the optimal by an absolute factor.

cs.LG↗

arXiv2024

When Context Leads but Parametric Memory Follows in Large Language Models

Large language models (LLMs) have demonstrated remarkable progress in leveraging diverse knowledge sources. This study investigates how nine widely used LLMs allocate knowledge between local context and global parameters when answering open-ended questions in knowledge-consistent scenarios. We introduce a novel dataset, WikiAtomic, and systematically vary context sizes to analyze how LLMs prioritize and utilize the provided information and their parametric knowledge in knowledge-consistent scenarios. Additionally, we also study their tendency to hallucinate under varying context sizes. Our findings reveal consistent patterns across models, including a consistent reliance on both contextual (around 70%) and parametric (around 30%) knowledge, and a decrease in hallucinations with increasing context. These insights highlight the importance of more effective context organization and developing models that use input more deterministically for robust performance.

cs.CL↗

arXiv2024

ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalness

Recent advances in interactive large language models like ChatGPT have revolutionized various domains; however, their behavior in natural and role-play conversation settings remains underexplored. In our study, we address this gap by deeply investigating how ChatGPT behaves during conversations in different settings by analyzing its interactions in both a normal way and a role-play setting. We introduce a novel dataset of broad range of human-AI conversations annotated with user motives and model naturalness to examine (i) how humans engage with the conversational AI model, and (ii) how natural are AI model responses. Our study highlights the diversity of user motives when interacting with ChatGPT and variable AI naturalness, showing not only the nuanced dynamics of natural conversations between humans and AI, but also providing new avenues for improving the effectiveness of human-AI communication.

cs.CL↗

arXiv2024

Making a Long Story Short in Conversation Modeling

Conversation systems accommodate diverse users with unique personalities and distinct writing styles. Within the domain of multi-turn dialogue modeling, this work studies the impact of varied utterance lengths on the quality of subsequent responses generated by conversation models. Using GPT-3 as the base model, multiple dialogue datasets, and several metrics, we conduct a thorough exploration of this aspect of conversational models. Our analysis sheds light on the complex relationship between utterance lengths and the quality of follow-up responses generated by dialogue systems. Empirical findings suggests that, for certain types of conversations, utterance lengths can be reduced by up to 72% without any noticeable difference in the quality of follow-up responses.

cs.CL↗

arXiv2023

Join Sampling under Acyclic Degree Constraints and (Cyclic) Subgraph Sampling

Given a join with an acyclic set of degree constraints, we show how to draw a uniformly random sample from the join result in $O(\mathit{polymat}/ \max \{1, \mathrm{OUT} \})$ expected time after a preprocessing of $O(\mathrm{IN})$ expected time, where $\mathrm{IN}$, $\mathrm{OUT}$, and $\mathit{polymat}$ are the join's input size, output size, and polymatroid bound, respectively. This compares favorably with the state of the art (Deng et al.\ and Kim et al., both in PODS'23), which states that a uniformly random sample can be drawn in $\tilde{O}(\mathrm{AGM} / \max \{1, \mathrm{OUT}\})$ expected time after a preprocessing phase of $\tilde{O}(\mathrm{IN})$ expected time, where $\mathrm{AGM}$ is the join's AGM bound. We then utilize our techniques to tackle {\em directed subgraph sampling}. Let $G = (V, E)$ be a directed data graph where each vertex has an out-degree at most $λ$, and let $P$ be a directed pattern graph with $O(1)$ vertices. The objective is to uniformly sample an occurrence of $P$ in $G$. The problem can be modeled as join sampling with input size $\mathrm{IN} = Θ(|E|)$ but, whenever $P$ contains cycles, the converted join has {\em cyclic} degree constraints. We show that it is always possible to throw away certain degree constraints such that (i) the remaining constraints are acyclic and (ii) the new join has asymptotically the same polymatroid bound $\mathit{polymat}$ as the old one. Combining this finding with our new join sampling solution yields an algorithm to sample from the original (cyclic) join (thereby yielding a uniformly random occurrence of $P$) in $O(\mathit{polymat}/ \max \{1, \mathrm{OUT}\})$ expected time after $O(|E|)$ expected-time preprocessing. We also prove similar results for {\em undirected subgraph sampling} and demonstrate how our techniques can be significantly simplified in that scenario.

cs.DB↗

arXiv2023

Space-Query Tradeoffs in Range Subgraph Counting and Listing

This paper initializes the study of {\em range subgraph counting} and {\em range subgraph listing}, both of which are motivated by the significant demands in practice to perform graph analytics on subgraphs pertinent to only selected, as opposed to all, vertices. In the first problem, there is an undirected graph $G$ where each vertex carries a real-valued attribute. Given an interval $q$ and a pattern $Q$, a query counts the number of occurrences of $Q$ in the subgraph of $G$ induced by the vertices whose attributes fall in $q$. The second problem has the same setup except that a query needs to enumerate (rather than count) those occurrences with a small delay. In both problems, our goal is to understand the tradeoff between {\em space usage} and {\em query cost}, or more specifically: (i) given a target on query efficiency, how much pre-computed information about $G$ must we store? (ii) Or conversely, given a budget on space usage, what is the best query time we can hope for? We establish a suite of upper- and lower-bound results on such tradeoffs for various query patterns.

cs.DS↗

arXiv2022

A Near-Optimal Parallel Algorithm for Joining Binary Relations

We present a constant-round algorithm in the massively parallel computation (MPC) model for evaluating a natural join where every input relation has two attributes. Our algorithm achieves a load of $\tilde{O}(m/p^{1/ρ})$ where $m$ is the total size of the input relations, $p$ is the number of machines, $ρ$ is the join's fractional edge covering number, and $\tilde{O}(.)$ hides a polylogarithmic factor. The load matches a known lower bound up to a polylogarithmic factor. At the core of the proposed algorithm is a new theorem (which we name the "isolated cartesian product theorem") that provides fresh insight into the problem's mathematical structure. Our result implies that the subgraph enumeration problem, where the goal is to report all the occurrences of a constant-sized subgraph pattern, can be settled optimally (up to a polylogarithmic factor) in the MPC model.

cs.DB↗

arXiv2022

Parallel Acyclic Joins with Canonical Edge Covers

In PODS'21, Hu presented an algorithm in the massively parallel computation (MPC) model that processes any acyclic join with an asymptotically optimal load. In this paper, we present an alternative analysis of her algorithm. The novelty of our analysis is in the revelation of a new mathematical structure -- which we name "canonical edge cover" -- for acyclic hypergraphs. We prove non-trivial properties for canonical edge covers that offer us a graph-theoretic perspective about why Hu's algorithm works.

cs.DB↗

arXiv2019

Distribution-Sensitive Bounds on Relative Approximations of Geometric Ranges

A family $\mathcal{R}$ of ranges and a set $X$ of points together define a range space $(X, \mathcal{R}|_X)$, where $\mathcal{R}|_X = \{X \cap h \mid h \in \mathcal{R}\}$. We want to find a structure to estimate the quantity $|X \cap h|/|X|$ for any range $h \in \mathcal{R}$ with the $(ρ, ε)$-guarantee: (i) if $|X \cap h|/|X| > ρ$, the estimate must have a relative error $ε$; (ii) otherwise, the estimate must have an absolute error $ρε$. The objective is to minimize the size of the structure. Currently, the dominant solution is to compute a relative $(ρ, ε)$-approximation, which is a subset of $X$ with $\tilde{O}(λ/(ρε^2))$ points, where $λ$ is the VC-dimension of $(X, \mathcal{R}|_X)$, and $\tilde{O}$ hides polylog factors. This paper shows a more general bound sensitive to the content of $X$. We give a structure that stores $O(\log (1/ρ))$ integers plus $\tilde{O}(θ\cdot (λ/ε^2))$ points of $X$, where $θ$ - called the disagreement coefficient - measures how much the ranges differ from each other in their intersections with $X$. The value of $θ$ is between 1 and $1/ρ$, such that our space bound is never worse than that of relative $(ρ, ε)$-approximations, but we improve the latter's $1/ρ$ term whenever $θ= o(\frac{1}{ρ\log (1/ρ)})$. We also prove that, in the worst case, summaries with the $(ρ, 1/2)$-guarantee must consume $Ω(θ)$ words even for $d = 2$ and $λ\le 3$. We then constrain $\mathcal{R}$ to be the set of halfspaces in $\mathbb{R}^d$ for a constant $d$, and prove the existence of structures with $o(1/(ρε^2))$ size offering $(ρ,ε)$-guarantees, when $X$ is generated from various stochastic distributions. This is the first formal justification on why the term $1/ρ$ is not compulsory for "realistic" inputs.

cs.CG↗

arXiv2014

A Dynamic I/O-Efficient Structure for One-Dimensional Top-k Range Reporting

We present a structure in external memory for "top-k range reporting", which uses linear space, answers a query in O(lg_B n + k/B) I/Os, and supports an update in O(lg_B n) amortized I/Os, where n is the input size, and B is the block size. This improves the state of the art which incurs O(lg^2_B n) amortized I/Os per update.

cs.DS↗

arXiv2013

Optimal Planar Range Skyline Reporting with Linear Space in External Memory

Let P be a set of n points in R^2. Given a rectangle Q = [α_1, α_2] x [β_1, β_2], a range skyline query returns the maxima of the points in P \cap Q. An important variant is the so-called top-open queries, where Q is a 3-sided rectangle whose upper edge is grounded at y = \infty (that is, β_2 = \infty). These queries are crucial in numerous database applications. In internal memory, extensive research has been devoted to designing data structures that can answer such queries efficiently. In contrast, currently there is no clear understanding about their exact complexities in external memory. This paper presents several structures of linear size for answering the above queries with the optimal I/O cost. We show that a top-open query can be solved in O(log_B(n) + k/B) I/Os, where B is the block size and k is the number of points in the query result. The query cost can be made O(log log_B(U) + k/B) when the data points lie in a U x U grid for some integer U >= n, and further lowered to O(1 + k/B) if U = O(n). The same efficiency also applies to 3-sided queries where Q is a right-open rectangle. However, the hardness of the problem increases if Q is a left- or bottom-open 3-sided rectangle. We prove that any linear-size structure must perform Ω((n/B)^\eps + k/B) I/Os to solve such a query in the worst case, where \eps > 0 can be an arbitrarily small constant. In fact, left- and right-open queries are just as difficult as general (4-sided) queries, for which we give a linear-size structure with query time O((n/B)^\eps + k/B). Interestingly, this indicates that 4-sided range skyline queries have exactly the same hardness as 4-sided range reporting (where the goal is to report simply the whole P \cap Q). That is, the skyline requirement does not alter the problem difficulty at all.

cs.DS↗

arXiv2013

I/O-Efficient Planar Range Skyline and Attrition Priority Queues

In the planar range skyline reporting problem, we store a set P of n 2D points in a structure such that, given a query rectangle Q = [a_1, a_2] x [b_1, b_2], the maxima (a.k.a. skyline) of P \cap Q can be reported efficiently. The query is 3-sided if an edge of Q is grounded, giving rise to two variants: top-open (b_2 = \infty) and left-open (a_1 = -\infty) queries. All our results are in external memory under the O(n/B) space budget, for both the static and dynamic settings: * For static P, we give structures that answer top-open queries in O(log_B n + k/B), O(loglog_B U + k/B), and O(1 + k/B) I/Os when the universe is R^2, a U x U grid, and a rank space grid [O(n)]^2, respectively (where k is the number of reported points). The query complexity is optimal in all cases. * We show that the left-open case is harder, such that any linear-size structure must incur Ω((n/B)^e + k/B) I/Os for a query. We show that this case is as difficult as the general 4-sided queries, for which we give a static structure with the optimal query cost O((n/B)^e + k/B). * We give a dynamic structure that supports top-open queries in O(log_2B^e (n/B) + k/B^1-e) I/Os, and updates in O(log_2B^e (n/B)) I/Os, for any e satisfying 0 \le e \le 1. This leads to a dynamic structure for 4-sided queries with optimal query cost O((n/B)^e + k/B), and amortized update cost O(log (n/B)). As a contribution of independent interest, we propose an I/O-efficient version of the fundamental structure priority queue with attrition (PQA). Our PQA supports FindMin, DeleteMin, and InsertAndAttrite all in O(1) worst case I/Os, and O(1/B) amortized I/Os per operation. We also add the new CatenateAndAttrite operation that catenates two PQAs in O(1) worst case and O(1/B) amortized I/Os. This operation is a non-trivial extension to the classic PQA of Sundar, even in internal memory.

cs.DS↗

arXiv2012

A Scalable Algorithm for Maximizing Range Sum in Spatial Databases

This paper investigates the MaxRS problem in spatial databases. Given a set O of weighted points and a rectangular region r of a given size, the goal of the MaxRS problem is to find a location of r such that the sum of the weights of all the points covered by r is maximized. This problem is useful in many location-based applications such as finding the best place for a new franchise store with a limited delivery range and finding the most attractive place for a tourist with a limited reachable range. However, the problem has been studied mainly in theory, particularly, in computational geometry. The existing algorithms from the computational geometry community are in-memory algorithms which do not guarantee the scalability. In this paper, we propose a scalable external-memory algorithm (ExactMaxRS) for the MaxRS problem, which is optimal in terms of the I/O complexity. Furthermore, we propose an approximation algorithm (ApproxMaxCRS) for the MaxCRS problem that is a circle version of the MaxRS problem. We prove the correctness and optimality of the ExactMaxRS algorithm along with the approximation bound of the ApproxMaxCRS algorithm. From extensive experimental results, we show that the ExactMaxRS algorithm is two orders of magnitude faster than methods adapted from existing algorithms, and the approximation bound in practice is much better than the theoretical bound of the ApproxMaxCRS algorithm.

cs.DB↗