arXiv subjects

Shunyu Yao

Publications and source records attributed to Shunyu Yao.

At least 91 records · Page 5Linked to original sources

arXiv2023

Referral Augmentation for Zero-Shot Information Retrieval

We propose Referral-Augmented Retrieval (RAR), a simple technique that concatenates document indices with referrals, i.e. text from other documents that cite or link to the given document, to provide significant performance gains for zero-shot information retrieval. The key insight behind our method is that referrals provide a more complete, multi-view representation of a document, much like incoming page links in algorithms like PageRank provide a comprehensive idea of a webpage's importance. RAR works with both sparse and dense retrievers, and outperforms generative text expansion techniques such as DocT5Query and Query2Doc a 37% and 21% absolute improvement on ACL paper retrieval Recall@10 -- while also eliminating expensive model training and inference. We also analyze different methods for multi-referral aggregation and show that RAR enables up-to-date information retrieval without re-training.

cs.CL↗

arXiv2023

EC^2: Emergent Communication for Embodied Control

Embodied control requires agents to leverage multi-modal pre-training to quickly learn how to act in new environments, where video demonstrations contain visual and motion details needed for low-level perception and control, and language instructions support generalization with abstract, symbolic structures. While recent approaches apply contrastive learning to force alignment between the two modalities, we hypothesize better modeling their complementary differences can lead to more holistic representations for downstream adaption. To this end, we propose Emergent Communication for Embodied Control (EC^2), a novel scheme to pre-train video-language representations for few-shot embodied control. The key idea is to learn an unsupervised "language" of videos via emergent communication, which bridges the semantics of video details and structures of natural language. We learn embodied representations of video trajectories, emergent language, and natural language using a language model, which is then used to finetune a lightweight policy network for downstream control. Through extensive experiments in Metaworld and Franka Kitchen embodied benchmarks, EC^2 is shown to consistently outperform previous contrastive learning methods for both videos and texts as task inputs. Further ablations confirm the importance of the emergent language, which is beneficial for both video and language learning, and significantly superior to using pre-trained video captions. We also present a quantitative and qualitative analysis of the emergent language and discuss future directions toward better understanding and leveraging emergent communication in embodied tasks.

cs.LG↗

arXiv2023

Self-Attention Networks Can Process Bounded Hierarchical Languages

Despite their impressive performance in NLP, self-attention networks were recently proved to be limited for processing formal languages with hierarchical structure, such as $\mathsf{Dyck}_k$, the language consisting of well-nested parentheses of $k$ types. This suggested that natural language can be approximated well with models that are too weak for formal languages, or that the role of hierarchy and recursion in natural language might be limited. We qualify this implication by proving that self-attention networks can process $\mathsf{Dyck}_{k, D}$, the subset of $\mathsf{Dyck}_{k}$ with depth bounded by $D$, which arguably better captures the bounded hierarchical structure of natural language. Specifically, we construct a hard-attention network with $D+1$ layers and $O(\log k)$ memory size (per token per layer) that recognizes $\mathsf{Dyck}_{k, D}$, and a soft-attention network with two layers and $O(\log k)$ memory size that generates $\mathsf{Dyck}_{k, D}$. Experiments show that self-attention networks trained on $\mathsf{Dyck}_{k, D}$ generalize to longer inputs with near-perfect accuracy, and also verify the theoretical memory advantage of self-attention networks over recurrent networks.

cs.CL↗

arXiv2023

ReAct: Synergizing Reasoning and Acting in Language Models

While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information. We apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines, as well as improved human interpretability and trustworthiness over methods without reasoning or acting components. Concretely, on question answering (HotpotQA) and fact verification (Fever), ReAct overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning by interacting with a simple Wikipedia API, and generates human-like task-solving trajectories that are more interpretable than baselines without reasoning traces. On two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples. Project site with code: https://react-lm.github.io

cs.CL↗

arXiv2023

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

Existing benchmarks for grounding language in interactive environments either lack real-world linguistic elements, or prove difficult to scale up due to substantial human involvement in the collection of data or feedback signals. To bridge this gap, we develop WebShop -- a simulated e-commerce website environment with $1.18$ million real-world products and $12,087$ crowd-sourced text instructions. Given a text instruction specifying a product requirement, an agent needs to navigate multiple types of webpages and issue diverse actions to find, customize, and purchase an item. WebShop provides several challenges for language grounding including understanding compositional instructions, query (re-)formulation, comprehending and acting on noisy text in webpages, and performing strategic exploration. We collect over $1,600$ human demonstrations for the task, and train and evaluate a diverse range of agents using reinforcement learning, imitation learning, and pre-trained image and language models. Our best model achieves a task success rate of $29\%$, which outperforms rule-based heuristics ($9.6\%$) but is far lower than human expert performance ($59\%$). We also analyze agent and human trajectories and ablate various model components to provide insights for developing future agents with stronger language understanding and decision making abilities. Finally, we show that agents trained on WebShop exhibit non-trivial sim-to-real transfer when evaluated on amazon.com and ebay.com, indicating the potential value of WebShop in developing practical web-based agents that can operate in the wild.

cs.CL↗

arXiv2023

The power of Lorentzian wormholes

As shown by Louko and Sorkin in 1995, topology change in Lorentzian signature involves spacetimes with singular points, which they called crotches. We modify their construction to obtain Lorentzian semiclassical wormholes in asymptotically AdS. These solutions are obtained by inserting crotches on known saddles, like the double-cone or multiple copies of the Lorentzian black hole. The crotches implement swap-identifications, and are classically located at an extremal surface. The resulting Lorentzian wormholes have an instanton action equal to their area, which is responsible for topological suppression in any number of dimensions. We conjecture that including these Lorentzian wormhole spacetimes is gauge-equivalent to path integrating over all mostly Euclidean smooth spacetimes. We present evidence for this by reproducing semiclassical features of the genus expansion of the spectral form factor, and of a late-time two point function, by summing over the moduli space of Lorentzian wormholes. As a final piece of evidence, we discuss the Lorentzian version of West-Coast replica wormholes.

hep-th↗

arXiv2022

A convergent genus expansion for the plateau

We conjecture a formula for the spectral form factor of a double-scaled matrix integral in the limit of large time, large density of states, and fixed temperature. The formula has a genus expansion with a nonzero radius of convergence. To understand the origin of this series, we compare to the semiclassical theory of "encounters" in periodic orbits. In Jackiw-Teitelboim (JT) gravity, encounters correspond to portions of the moduli space integral that mutually cancel (in the orientable case) but individually grow at low energies. At genus one we show how the full moduli space integral resolves the low energy region and gives a finite nonzero answer.

hep-th↗

arXiv2022

Revisiting the Roles of "Text" in Text Games

Text games present opportunities for natural language understanding (NLU) methods to tackle reinforcement learning (RL) challenges. However, recent work has questioned the necessity of NLU by showing random text hashes could perform decently. In this paper, we pursue a fine-grained investigation into the roles of text in the face of different RL challenges, and reconcile that semantic and non-semantic language representations could be complementary rather than contrasting. Concretely, we propose a simple scheme to extract relevant contextual information into an approximate state hash as extra input for an RNN-based text agent. Such a lightweight plug-in achieves competitive performance with state-of-the-art text agents using advanced NLU techniques such as knowledge graph and passage retrieval, suggesting non-NLU methods might suffice to tackle the challenge of partial observability. However, if we remove RNN encoders and use approximate or even ground-truth state hash alone, the model performs miserably, which confirms the importance of semantic function approximation to tackle the challenge of combinatorially large observation and action spaces. Our findings and analysis provide new insights for designing better text game task setups and agents.

cs.CL↗

arXiv2022

An integrable road to a perturbative plateau

As has been known since the 90s, there is an integrable structure underlying two-dimensional gravity theories. Recently, two-dimensional gravity theories have regained an enormous amount of attention, but now in relation with quantum chaos - superficially nothing like integrability. In this paper, we return to the roots and exploit the integrable structure underlying dilaton gravity theories to study a late time, large $e^{S_\text{BH}}$ double scaled limit of the spectral form factor. In this limit, a novel cancellation due to the integrable structure ensures that at each genus $g$ the spectral form factor grows like $T^{2g+1}$, and that the sum over genera converges, realising a perturbative approach to the late-time plateau. Along the way, we clarify various aspects of this integrable structure. In particular, we explain the central role played by ribbon graphs, we discuss intersection theory, and we explain what the relations with dilaton gravity and matrix models are from a more modern holographic perspective.

hep-th↗

arXiv2022

TVShowGuess: Character Comprehension in Stories as Speaker Guessing

We propose a new task for assessing machines' skills of understanding fictional characters in narrative stories. The task, TVShowGuess, builds on the scripts of TV series and takes the form of guessing the anonymous main characters based on the backgrounds of the scenes and the dialogues. Our human study supports that this form of task covers comprehension of multiple types of character persona, including understanding characters' personalities, facts and memories of personal experience, which are well aligned with the psychological and literary theories about the theory of mind (ToM) of human beings on understanding fictional characters during reading. We further propose new model architectures to support the contextualized encoding of long scene texts. Experiments show that our proposed approaches significantly outperform baselines, yet still largely lag behind the (nearly perfect) human performance. Our work serves as a first step toward the goal of narrative character comprehension.

cs.CL↗

arXiv2022

Linking Emergent and Natural Languages via Corpus Transfer

The study of language emergence aims to understand how human languages are shaped by perceptual grounding and communicative intent. Computational approaches to emergent communication (EC) predominantly consider referential games in limited domains and analyze the learned protocol within the game framework. As a result, it remains unclear how the emergent languages from these settings connect to natural languages or provide benefits in real-world language processing tasks, where statistical models trained on large text corpora dominate. In this work, we propose a novel way to establish such a link by corpus transfer, i.e. pretraining on a corpus of emergent language for downstream natural language tasks, which is in contrast to prior work that directly transfers speaker and listener parameters. Our approach showcases non-trivial transfer benefits for two different tasks -- language modeling and image captioning. For example, in a low-resource setup (modeling 2 million natural language tokens), pre-training on an emergent language corpus with just 2 million tokens reduces model perplexity by $24.6\%$ on average across ten natural languages. We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images. We find that our translation-based metric highly correlates with the downstream performance on modeling natural languages (for instance $ρ=0.83$ on Hebrew), while topographic similarity, a popular metric in previous work, shows surprisingly low correlation ($ρ=0.003$), hinting that simple properties like attribute disentanglement from synthetic domains might not capture the full complexities of natural language. Our findings also indicate potential benefits of moving language emergence forward with natural language resources and models.

cs.CL↗

arXiv2022

Multi-Stage Episodic Control for Strategic Exploration in Text Games

Text adventure games present unique challenges to reinforcement learning methods due to their combinatorially large action spaces and sparse rewards. The interplay of these two factors is particularly demanding because large action spaces require extensive exploration, while sparse rewards provide limited feedback. This work proposes to tackle the explore-vs-exploit dilemma using a multi-stage approach that explicitly disentangles these two strategies within each episode. Our algorithm, called eXploit-Then-eXplore (XTX), begins each episode using an exploitation policy that imitates a set of promising trajectories from the past, and then switches over to an exploration policy aimed at discovering novel actions that lead to unseen state spaces. This policy decomposition allows us to combine global decisions about which parts of the game space to return to with curiosity-based local exploration in that space, motivated by how a human may approach these games. Our method significantly outperforms prior approaches by 27% and 11% average normalized score over 12 games from the Jericho benchmark (Hausknecht et al., 2020) in both deterministic and stochastic settings, respectively. On the game of Zork1, in particular, XTX obtains a score of 103, more than a 2x improvement over prior methods, and pushes past several known bottlenecks in the game that have plagued previous state-of-the-art methods.

cs.CL↗

arXiv2022

Subleading Weingartens

Haar integrals over the unitary group contain subleading terms that are needed for unitarity. We study analogous effects in the time evolution operators of JT gravity and Brownian SYK. In JT gravity with bulk matter we find an explanation for the first subleading terms, and in Brownian SYK we find configurations that can explain the full series. An important role is played by slightly off-shell modes that are exponentially amplified by chaos.

hep-th↗

arXiv2022

DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering

While recent advances in deep neural networks have made it possible to render high-quality images, generating photo-realistic and personalized talking head remains challenging. With given audio, the key to tackling this task is synchronizing lip movement and simultaneously generating personalized attributes like head movement and eye blink. In this work, we observe that the input audio is highly correlated to lip motion while less correlated to other personalized attributes (e.g., head movements). Inspired by this, we propose a novel framework based on neural radiance field to pursue high-fidelity and personalized talking head generation. Specifically, neural radiance field takes lip movements features and personalized attributes as two disentangled conditions, where lip movements are directly predicted from the audio inputs to achieve lip-synchronized generation. In the meanwhile, personalized attributes are sampled from a probabilistic model, where we design a Transformer-based variational autoencoder sampled from Gaussian Process to learn plausible and natural-looking head pose and eye blink. Experiments on several benchmarks demonstrate that our method achieves significantly better results than state-of-the-art methods.

cs.CV↗

arXiv2021

Comments on wormholes and factorization

In AdS/CFT partition functions of decoupled copies of the CFT factorize. In bulk computations of such quantities contributions from spacetime wormholes which link separate asymptotic boundaries threaten to spoil this property, leading to a "factorization puzzle." Certain simple models like JT gravity have wormholes, but bulk computations in them correspond to averages over an ensemble of boundary systems. These averages need not factorize. We can formulate a toy version of the factorization puzzle in such models by focusing on a specific member of the ensemble where partition functions will again factorize. As Coleman and Giddings-Strominger pointed out in the 1980s, fixed members of ensembles are described in the bulk by "$α$-states" in a many-universe Hilbert space. In this paper we analyze in detail the bulk mechanism for factorization in such $α$-states in the topological model introduced by Marolf and Maxfield (the "MM model") and in JT gravity. In these models geometric calculations in $α$ states are poorly controlled. We circumvent this complication by working in $\textit{approximate}$ $α$ states where bulk calculations just involve the simplest topologies: disks and cylinders. One of our main results is an effective description of the factorization mechanism. In this effective description the many-universe contributions from the full $α$ state are replaced by a small number of effective boundaries. Our motivation in constructing this effective description, and more generally in studying these simple ensemble models, is that the lessons learned might have wider applicability. In fact the effective description lines up with a recent discussion of the SYK model with fixed couplings arXiv:2103.16754. We conclude with some discussion about the possible applicability of this effective model in more general contexts.

hep-th↗

arXiv2021

Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents

Text-based games simulate worlds and interact with players using natural language. Recent work has used them as a testbed for autonomous language-understanding agents, with the motivation being that understanding the meanings of words or semantics is a key component of how humans understand, reason, and act in these worlds. However, it remains unclear to what extent artificial agents utilize semantic understanding of the text. To this end, we perform experiments to systematically reduce the amount of semantic information available to a learning agent. Surprisingly, we find that an agent is capable of achieving high scores even in the complete absence of language semantics, indicating that the currently popular experimental setup and models may be poorly designed to understand and leverage game texts. To remedy this deficiency, we propose an inverse dynamics decoder to regularize the representation space and encourage exploration, which shows improved performance on several games including Zork I. We discuss the implications of our findings for designing future agents with stronger semantic understanding.

cs.CL↗

arXiv2021

Wormholes without averaging

After averaging over fermion couplings, SYK has a collective field description that sometimes has "wormhole" solutions. We study the fate of these wormholes when the couplings are fixed. Working mainly in a simple model, we find that the wormhole saddles persist, but that new saddles also appear elsewhere in the integration space -- "half-wormholes." The wormhole contributions depend only weakly on the specific choice of couplings, while the half-wormhole contributions are strongly sensitive. The half-wormholes are crucial for factorization of decoupled systems with fixed couplings, but they vanish after averaging, leaving the non-factorizing wormhole behind.

hep-th↗

arXiv2020

Keep CALM and Explore: Language Models for Action Generation in Text-based Games

Text-based games present a unique challenge for autonomous agents to operate in natural language and handle enormous action spaces. In this paper, we propose the Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. Our key insight is to train language models on human gameplay, where people demonstrate linguistic priors and a general game sense for promising actions conditioned on game history. We combine CALM with a reinforcement learning agent which re-ranks the generated action candidates to maximize in-game rewards. We evaluate our approach using the Jericho benchmark, on games unseen by CALM during training. Our method obtains a 69% relative improvement in average game score over the previous state-of-the-art model. Surprisingly, on half of these games, CALM is competitive with or better than other models that have access to ground truth admissible actions. Code and data are available at https://github.com/princeton-nlp/calm-textgame.

cs.CL↗