arXiv subjects

David Wu

Publications and source records attributed to David Wu.

32 records · Page 2Linked to original sources

arXiv2022

Improving Chess Commentaries by Combining Language Models with Symbolic Reasoning Engines

Despite many recent advancements in language modeling, state-of-the-art language models lack grounding in the real world and struggle with tasks involving complex reasoning. Meanwhile, advances in the symbolic reasoning capabilities of AI have led to systems that outperform humans in games like chess and Go (Silver et al., 2018). Chess commentary provides an interesting domain for bridging these two fields of research, as it requires reasoning over a complex board state and providing analyses in natural language. In this work we demonstrate how to combine symbolic reasoning engines with controllable language models to generate chess commentaries. We conduct experiments to demonstrate that our approach generates commentaries that are preferred by human judges over previous baselines.

cs.CL↗

arXiv2022

Self-Explaining Deviations for Coordination

Fully cooperative, partially observable multi-agent problems are ubiquitous in the real world. In this paper, we focus on a specific subclass of coordination problems in which humans are able to discover self-explaining deviations (SEDs). SEDs are actions that deviate from the common understanding of what reasonable behavior would be in normal circumstances. They are taken with the intention of causing another agent or other agents to realize, using theory of mind, that the circumstance must be abnormal. We first motivate SED with a real world example and formalize its definition. Next, we introduce a novel algorithm, improvement maximizing self-explaining deviations (IMPROVISED), to perform SEDs. Lastly, we evaluate IMPROVISED both in an illustrative toy setting and the popular benchmark setting Hanabi, where it is the first method to produce so called finesse plays, which are regarded as one of the more iconic examples of human theory of mind.

cs.AI↗

arXiv2022

Likelihood-based estimation and prediction for a measles outbreak in Samoa

Prediction of the progression of an infectious disease outbreak is important for planning and coordinating a response. Differential equations are often used to model an epidemic outbreak's behaviour but are challenging to parameterise. Furthermore, these models can suffer from misspecification, which biases predictions and parameter estimates. Stochastic models can help with misspecification but are even more expensive to simulate and perform inference with. Here, we develop an explicitly likelihood-based variation of the generalised profiling method as a tool for prediction and inference under model misspecification. Our approach allows us to carry out identifiability analysis and uncertainty quantification using profile likelihood-based methods without the need for marginalisation. We provide justification for this approach by introducing a new interpretation of the model approximation component as a stochastic constraint. This preserves the rationale for using profiling rather than integration to remove nuisance parameters while also providing a link back to stochastic models. We applied an initial version of this method during an outbreak of measles in Samoa in 2019-2020 and found that it achieved relatively fast, accurate predictions. Here we present the most recent version of our method and its application to this measles outbreak, along with additional validation.

q-bio.PE↗

arXiv2021

Maximum a Posteriori Inference of Random Dot Product Graphs via Conic Programming

We present a convex cone program to infer the latent probability matrix of a random dot product graph (RDPG). The optimization problem maximizes the Bernoulli maximum likelihood function with an added nuclear norm regularization term. The dual problem has a particularly nice form, related to the well-known semidefinite program relaxation of the MaxCut problem. Using the primal-dual optimality conditions, we bound the entries and rank of the primal and dual solutions. Furthermore, we bound the optimal objective value and prove asymptotic consistency of the probability estimates of a slightly modified model under mild technical assumptions. Our experiments on synthetic RDPGs not only recover natural clusters, but also reveal the underlying low-dimensional geometry of the original data. We also demonstrate that the method recovers latent structure in the Karate Club Graph and synthetic U.S. Senate vote graphs and is scalable to graphs with up to a few hundred nodes.

cs.LG↗

arXiv2021

$AIR^2$ for Interaction Prediction

The 2021 Waymo Interaction Prediction Challenge introduced a problem of predicting the future trajectories and confidences of two interacting agents jointly. We developed a solution that takes an anchored marginal motion prediction model with rasterization and augments it to model agent interaction. We do this by predicting the joint confidences using a rasterized image that highlights the ego agent and the interacting agent. Our solution operates on the cartesian product space of the anchors; hence the $"^2"$ in $AIR^2$. Our model achieved the highest mAP (the primary metric) on the leaderboard.

cs.CV↗

arXiv2021

QK Iteration: A Self-Supervised Representation Learning Algorithm for Image Similarity

Self-supervised representation learning is a fundamental problem in computer vision with many useful applications (e.g., image search, instance level recognition, copy detection). In this paper we present a new contrastive self-supervised representation learning algorithm in the context of Copy Detection in the 2021 Image Similarity Challenge hosted by Facebook AI Research. Previous work in contrastive self-supervised learning has identified the importance of being able to optimize representations while ``pushing'' against a large number of negative examples. Representative previous solutions either use large batches enabled by modern distributed training systems or maintain queues or memory banks holding recently evaluated representations while relaxing some consistency properties. We approach this problem from a new angle: We directly learn a query model and a key model jointly and push representations against a very large number (e.g., 1 million) of negative representations in each SGD step. We achieve this by freezing the backbone on one side and by alternating between a Q-optimization step and a K-optimization step. During the competition timeframe, our algorithms achieved a micro-AP score of 0.3401 on the Phase 1 leaderboard, significantly improving over the baseline $μ$AP of 0.1556. On the final Phase 2 leaderboard, our model scored 0.1919, while the baseline scored 0.0526. Continued training yielded further improvement. We conducted an empirical study to compare the proposed approach with a SimCLR style strategy where the negative examples are taken from the batch only. We found that our method ($μ$AP of 0.3403) significantly outperforms this SimCLR-style baseline ($μ$AP of 0.2001).

cs.CV↗

arXiv2021

No-Press Diplomacy from Scratch

Prior AI successes in complex games have largely focused on settings with at most hundreds of actions at each decision point. In contrast, Diplomacy is a game with more than 10^20 possible actions per turn. Previous attempts to address games with large branching factors, such as Diplomacy, StarCraft, and Dota, used human data to bootstrap the policy or used handcrafted reward shaping. In this paper, we describe an algorithm for action exploration and equilibrium approximation in games with combinatorial action spaces. This algorithm simultaneously performs value iteration while learning a policy proposal network. A double oracle step is used to explore additional actions to add to the policy proposals. At each state, the target state value and policy for the model training are computed via an equilibrium search procedure. Using this algorithm, we train an agent, DORA, completely from scratch for a popular two-player variant of Diplomacy and show that it achieves superhuman performance. Additionally, we extend our methods to full-scale no-press Diplomacy and for the first time train an agent from scratch with no human data. We present evidence that this agent plays a strategy that is incompatible with human-data bootstrapped agents. This presents the first strong evidence of multiple equilibria in Diplomacy and suggests that self play alone may be insufficient for achieving superhuman performance in Diplomacy.

cs.LG↗

arXiv2021

Off-Belief Learning

The standard problem setting in Dec-POMDPs is self-play, where the goal is to find a set of policies that play optimally together. Policies learned through self-play may adopt arbitrary conventions and implicitly rely on multi-step reasoning based on fragile assumptions about other agents' actions and thus fail when paired with humans or independently trained agents at test time. To address this, we present off-belief learning (OBL). At each timestep OBL agents follow a policy $π_1$ that is optimized assuming past actions were taken by a given, fixed policy ($π_0$), but assuming that future actions will be taken by $π_1$. When $π_0$ is uniform random, OBL converges to an optimal policy that does not rely on inferences based on other agents' behavior (an optimal grounded policy). OBL can be iterated in a hierarchy, where the optimal policy from one level becomes the input to the next, thereby introducing multi-level cognitive reasoning in a controlled manner. Unlike existing approaches, which may converge to any equilibrium policy, OBL converges to a unique policy, making it suitable for zero-shot coordination (ZSC). OBL can be scaled to high-dimensional settings with a fictitious transition mechanism and shows strong performance in both a toy-setting and the benchmark human-AI & ZSC problem Hanabi.

cs.AI↗

arXiv2019

Nonuniform Distributions of Residues of Prime Sequences in Prime Moduli

For positive integers $q$, Dirichlet's theorem states that there are infinitely many primes in each reduced residue class modulo $q$. A stronger form of the theorem states that the primes are equidistributed among the $φ(q)$ reduced residue classes modulo $q$. This paper considers patterns of sequences of consecutive primes $(p_n, p_{n+1}, \ldots, p_{n+k})$ modulo $q$. Numerical evidence suggests a preference for certain prime patterns. For example, computed frequencies of the pattern $(a,a)$ modulo $q$ up to $x$ are much less than the expected frequency $π(x)/φ(q)^2$. We begin to rigorously connect the Hardy-Littlewood prime $k$-tuple conjecture to a conjectured asymptotic formula for the frequencies of prime patterns modulo $q$.

math.NT↗

arXiv2019

Distribution of Nonzero Digits in a Greedy Sequence of Powers of Two

Understanding the distribution of digits in the expansions of perfect powers in different bases is difficult. Rather than consider the asymptotic digit distributions, we consider the base-10 digits of a restricted sequence of powers of two. We apply elementary methods to show that this sequence of powers of two can be constructed to preserve trailing digits while locally maximizing the number of zeroes between nonzero digits. We also provide a heuristic description of the trailing digits of these powers of two.

math.NT↗

arXiv2017

Exact non-linear equations for cosmological perturbations

We present a complete set of exact and fully non-linear equations describing all three types of cosmological perturbations -- scalar, vector and tensor perturbations. We derive the equations in a thoroughly gauge-ready manner, so that any spatial and temporal gauge conditions can be employed. The equations are completely general without any physical restriction except that we assume a flat homogeneous and isotropic universe as a background. We also comment briefly on the application of our formulation to the non-expanding Minkowski background.

gr-qc↗

arXiv2012

A Single-Molecule Hershey-Chase Experiment

Ever since Hershey and Chase used phages to establish DNA as the carrier of genetic information in 1952, the precise mechanisms of phage DNA translocation have been a mystery. While bulk measurements have set a time scale for in vivo DNA translocation during bacteriophage infection, measurements of DNA ejection by single bacteriophages have only been made in vitro. Here, we present direct visualization of single bacteriophages infecting individual Escherichia coli cells. For bacteriophage lambda, we establish a mean ejection time of roughly 5 minutes with significant cell-to-cell variability, including pausing events. In contrast, corresponding in vitro single-molecule ejections take only 10 seconds to reach completion and do not exhibit significant variability. Our data reveal that the velocity of ejection for two different genome lengths collapses onto a single curve. This suggests that in vivo ejections are controlled by the amount of DNA ejected, in contrast with in vitro DNA ejections, which are governed by the amount of DNA left inside the capsid. This analysis provides evidence against a purely intrastrand repulsion based mechanism, and suggests that cell-internal processes dominate. This provides a picture of the early stages of phage infection and sheds light on the problem of polymer translocation.

physics.bio-ph↗

arXiv2010

Ion-dependent dynamics of DNA ejections for bacteriophage lambda

We study the control parameters that govern the dynamics of in vitro DNA ejection in bacteriophage lambda. Past work has demonstrated that bacteriophage DNA is highly pressurized; this pressure has been hypothesized to help drive DNA ejection. Ions influence this process by screening charges on DNA; however, a systematic variation of salt concentrations to explore these effects has not been undertaken. To study the nature of the forces driving DNA ejection, we performed in vitro measurements of DNA ejection in bulk and at the single-phage level. We present measurements on the dynamics of ejection and on the self-repulsion force driving ejection. We examine the role of ion concentration and identity in both measurements, and show that the charge of counter-ions is an important control parameter. These measurements show that the frictional force acting on the ejecting DNA is subtly dependent on ionic concentrations for a given amount of DNA in the capsid. We also present evidence that phage DNA forms loops during ejection; we confirm that this effect occurs using optical tweezers. We speculate this facilitates circularization of the genome in the cytoplasm.

q-bio.BM↗

arXiv2008

A trajectory approach to two-state kinetics of single particles on sculpted energy landscapes

We study the trajectories of a single colloidal particle as it hops between two energy wells A and B, which are sculpted using adjacent optical traps by controlling their respective power levels and separation. Whereas the dynamical behaviors of such systems are often treated by master-equation methods that focus on particles as actors, we analyze them here instead using a trajectory-based variational method called Maximum Caliber, which utilizes a dynamical partition function. We show that the Caliber strategy accurately predicts the full dynamics that we observe in the experiments: from the observed averages, it predicts second and third moments and covariances, with no free parameters. The covariances are the dynamical equivalents of Maxwell-like equilibrium reciprocal relations and Onsager-like dynamical relations. In short, this work describes an experimental model system for exploring full trajectory distributions in one-particle two-state systems, and it validates the Caliber approach as a useful way to understand trajectory-based dynamical distribution functions in this system.

physics.data-an↗