arXiv subjects

Yibo Zeng

Publications and source records attributed to Yibo Zeng.

6 recordsLinked to original sources

arXiv2024

LLM Embeddings Improve Test-time Adaptation to Tabular $Y|X$-Shifts

For tabular datasets, the change in the relationship between the label and covariates ($Y|X$-shifts) is common due to missing variables (a.k.a. confounders). Since it is impossible to generalize to a completely new and unknown domain, we study models that are easy to adapt to the target domain even with few labeled examples. We focus on building more informative representations of tabular data that can mitigate $Y|X$-shifts, and propose to leverage the prior world knowledge in LLMs by serializing (write down) the tabular data to encode it. We find LLM embeddings alone provide inconsistent improvements in robustness, but models trained on them can be well adapted/finetuned to the target domain even using 32 labeled observations. Our finding is based on a comprehensive and systematic study consisting of 7650 source-target pairs and benchmark against 261,000 model configurations trained by 22 algorithms. Our observation holds when ablating the size of accessible target data and different adaptation strategies. The code is available at https://github.com/namkoong-lab/LLM-Tabular-Shifts.

cs.LG↗

arXiv2024

Design and Scheduling of an AI-based Queueing System

To leverage prediction models to make optimal scheduling decisions in service systems, we must understand how predictive errors impact congestion due to externalities on the delay of other jobs. Motivated by applications where prediction models interact with human servers (e.g., content moderation), we consider a large queueing system comprising of many single server queues where the class of a job is estimated using a prediction model. By characterizing the impact of mispredictions on congestion cost in heavy traffic, we design an index-based policy that incorporates the predicted class information in a near-optimal manner. Our theoretical results guide the design of predictive models by providing a simple model selection procedure with downstream queueing performance as a central concern, and offer novel insights on how to design queueing systems with AI-based triage. We illustrate our framework on a content moderation task based on real online comments, where we construct toxicity classifiers by finetuning large language models.

math.OC↗

arXiv2022

Generalization Bounds with Minimal Dependency on Hypothesis Class via Distributionally Robust Optimization

Established approaches to obtain generalization bounds in data-driven optimization and machine learning mostly build on solutions from empirical risk minimization (ERM), which depend crucially on the functional complexity of the hypothesis class. In this paper, we present an alternate route to obtain these bounds on the solution from distributionally robust optimization (DRO), a recent data-driven optimization framework based on worst-case analysis and the notion of ambiguity set to capture statistical uncertainty. In contrast to the hypothesis class complexity in ERM, our DRO bounds depend on the ambiguity set geometry and its compatibility with the true loss function. Notably, when using statistical distances such as maximum mean discrepancy, Wasserstein distance, or $ϕ$-divergence in the DRO, our analysis implies generalization bounds whose dependence on the hypothesis class appears the minimal possible: The bound depends solely on the true loss function, independent of any other candidates in the hypothesis class. To our best knowledge, it is the first generalization bound of this type in the literature, and we hope our findings can open the door for a better understanding of DRO, especially its benefits on loss minimization and other machine learning applications.

math.OC↗

arXiv2020

AsyncQVI: Asynchronous-Parallel Q-Value Iteration for Discounted Markov Decision Processes with Near-Optimal Sample Complexity

In this paper, we propose AsyncQVI, an asynchronous-parallel Q-value iteration for discounted Markov decision processes whose transition and reward can only be sampled through a generative model. Given such a problem with $|\mathcal{S}|$ states, $|\mathcal{A}|$ actions, and a discounted factor $γ\in(0,1)$, AsyncQVI uses memory of size $\mathcal{O}(|\mathcal{S}|)$ and returns an $\varepsilon$-optimal policy with probability at least $1-δ$ using $$\tilde{\mathcal{O}}\big(\frac{|\mathcal{S}||\mathcal{A}|}{(1-γ)^5\varepsilon^2}\log(\frac{1}δ)\big)$$ samples. AsyncQVI is also the first asynchronous-parallel algorithm for discounted Markov decision processes that has a sample complexity, which nearly matches the theoretical lower bound. The relatively low memory footprint and parallel ability make AsyncQVI suitable for large-scale applications. In numerical tests, we compare AsyncQVI with four sample-based value iteration methods. The results show that our algorithm is highly efficient and achieves linear parallel speedup.

math.OC↗

arXiv2018

Hitting times and resistance distances of $q$-triangulation graphs: Accurate results and applications

Graph operations or products, such as triangulation and Kronecker product have been extensively applied to model complex networks with striking properties observed in real-world complex systems. In this paper, we study hitting times and resistance distances of $q$-triangulation graphs. For a simple connected graph $G$, its $q$-triangulation graph $R_q(G)$ is obtained from $G$ by performing the $q$-triangulation operation on $G$. That is, for every edge $uv$ in $G$, we add $q$ disjoint paths of length $2$, each having $u$ and $v$ as its ends. We first derive the eigenvalues and eigenvectors of normalized adjacency matrix of $R_q(G)$, expressing them in terms of those associated with $G$. Based on these results, we further obtain some interesting quantities about random walks and resistance distances for $R_q(G)$, including two-node hitting time, Kemeny's constant, two-node resistance distance, Kirchhoff index, additive degree-Kirchhoff index, and multiplicative degree-Kirchhoff index. Finally, we provide exact formulas for the aforementioned quantities of iterated $q$-triangulation graphs, using which we provide closed-form expressions for those quantities corresponding to a class of scale-free small-world graphs, which has been applied to mimic complex networks.

math.CO↗

arXiv2018

Spectra, hitting times, and resistance distances of $q$-subdivision graphs

Graph operations or products play an important role in complex networks. In this paper, we study the properties of $q$-subdivision graphs, which have been applied to model complex networks. For a simple connected graph $G$, its $q$-subdivision graph $S_q(G)$ is obtained from $G$ through replacing every edge $uv$ in $G$ by $q$ disjoint paths of length 2, with each path having $u$ and $v$ as its ends. We derive explicit formulas for many quantities of $S_q(G)$ in terms of those corresponding to $G$, including the eigenvalues and eigenvectors of normalized adjacency matrix, two-node hitting time, Kemeny constant, two-node resistance distance, Kirchhoff index, additive degree-Kirchhoff index, and multiplicative degree-Kirchhoff index. We also study the properties of the iterated $q$-subdivision graphs, based on which we obtain the closed-form expressions for a family of hierarchical lattices, which has been used to describe scale-free fractal networks.

math.CO↗