arXiv subjects

David Saxton

Publications and source records attributed to David Saxton.

17 recordsLinked to original sources

arXiv2024

De novo design of high-affinity protein binders with AlphaProteo

Computational design of protein-binding proteins is a fundamental capability with broad utility in biomedical research and biotechnology. Recent methods have made strides against some target proteins, but on-demand creation of high-affinity binders without multiple rounds of experimental testing remains an unsolved challenge. This technical report introduces AlphaProteo, a family of machine learning models for protein design, and details its performance on the de novo binder design problem. With AlphaProteo, we achieve 3- to 300-fold better binding affinities and higher experimental success rates than the best existing methods on seven target proteins. Our results suggest that AlphaProteo can generate binders "ready-to-use" for many research applications using only one round of medium-throughput screening and no further optimization.

q-bio.BM↗

arXiv2019

Analysing Mathematical Reasoning Abilities of Neural Models

Mathematical reasoning---a core ability within human intelligence---presents some unique challenges as a domain: we do not come to understand and solve mathematical problems primarily on the back of experience and evidence, but on the basis of inferring, learning, and exploiting laws, axioms, and symbol manipulation rules. In this paper, we present a new challenge for the evaluation (and eventually the design) of neural architectures and similar system, developing a task suite of mathematics problems involving sequential questions and answers in a free-form textual input/output format. The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes. Having described the data generation process and its potential future expansions, we conduct a comprehensive analysis of models from two broad classes of the most powerful sequence-to-sequence architectures and find notable differences in their ability to resolve mathematical problems and generalize their knowledge.

cs.LG↗

arXiv2019

Meta-Learning surrogate models for sequential decision making

We introduce a unified probabilistic framework for solving sequential decision making problems ranging from Bayesian optimisation to contextual bandits and reinforcement learning. This is accomplished by a probabilistic model-based approach that explains observed data while capturing predictive uncertainty during the decision making process. Crucially, this probabilistic model is chosen to be a Meta-Learning system that allows learning from a distribution of related problems, allowing data efficient adaptation to a target task. As a suitable instantiation of this framework, we explore the use of Neural processes due to statistical and computational desiderata. We apply our framework to a broad range of problem domains, such as control problems, recommender systems and adversarial attacks on RL agents, demonstrating an efficient and general black-box learning approach.

stat.ML↗

arXiv2018

Conditional Neural Processes

Deep neural networks excel at function approximation, yet they are typically trained from scratch for each new function. On the other hand, Bayesian methods, such as Gaussian Processes (GPs), exploit prior knowledge to quickly infer the shape of a new function at test time. Yet GPs are computationally expensive, and it can be hard to design appropriate priors. In this paper we propose a family of neural models, Conditional Neural Processes (CNPs), that combine the benefits of both. CNPs are inspired by the flexibility of stochastic processes such as GPs, but are structured as neural networks and trained via gradient descent. CNPs make accurate predictions after observing only a handful of training data points, yet scale to complex functions and large datasets. We demonstrate the performance and versatility of the approach on a range of canonical machine learning tasks, including regression, classification and image completion.

cs.LG↗

arXiv2018

An asymmetric container lemma and the structure of graphs with no induced $4$-cycle

The method of hypergraph containers, introduced recently by Balogh, Morris, and Samotij, and independently by Saxton and Thomason, has proved to be an extremely useful tool in the study of various monotone graph properties. In particular, a fairly straightforward application of this technique allows one to locate, for each non-bipartite graph $H$, the threshold at which the distribution of edges in a typical $H$-free graph with a given number of edges undergoes a transition from 'random-like' to 'structured'. On the other hand, for non-monotone hereditary graph properties the standard version of this method does not allow one to establish even the existence of such a threshold. In this paper we introduce a refinement of the container method that takes into account the asymmetry between edges and non-edges in a sparse member of a hereditary graph property. As an application, we determine the approximate structure of a typical graph with $n$ vertices, $m$ edges, and no induced copy of the $4$-cycle, for each function $m = m(n)$ satisfying $n^{4/3} (\log n)^4 \leqslant m \ll n^2$. We show that almost all such graphs $G$ have the following property: the vertex set of $G$ can be partitioned into an 'almost-independent' set (a set with $o(m)$ edges) and an 'almost-clique' (a set inducing a subgraph with density $1-o(1)$). The lower bound on $m$ is optimal up to a polylogarithmic factor, as standard arguments show that if $n \ll m \ll n^{4/3}$, then almost all such graphs are 'random-like'. As a further consequence, we deduce that the random graph $G(n,p)$ conditioned to contain no induced $4$-cycles undergoes phase transitions at $p = n^{-2/3 + o(1)}$ and $p = n^{-1/3 + o(1)}$.

math.CO↗

arXiv2018

Unsupervised Predictive Memory in a Goal-Directed Agent

Animals execute goal-directed behaviours despite the limited range and scope of their sensors. To cope, they explore environments and store memories maintaining estimates of important information that is not presently available. Recently, progress has been made with artificial intelligence (AI) agents that learn to perform tasks from sensory input, even at a human level, by merging reinforcement learning (RL) algorithms with deep neural networks, and the excitement surrounding these results has led to the pursuit of related ideas as explanations of non-human animal learning. However, we demonstrate that contemporary RL algorithms struggle to solve simple tasks when enough information is concealed from the sensors of the agent, a property called "partial observability". An obvious requirement for handling partially observed tasks is access to extensive memory, but we show memory is not enough; it is critical that the right information be stored in the right format. We develop a model, the Memory, RL, and Inference Network (MERLIN), in which memory formation is guided by a process of predictive modeling. MERLIN facilitates the solution of tasks in 3D virtual reality environments for which partial observability is severe and memories must be maintained over long durations. Our model demonstrates a single learning agent architecture that can solve canonical behavioural tasks in psychology and neurobiology without strong simplifying assumptions about the dimensionality of sensory input or the duration of experiences.

cs.LG↗

arXiv2018

Can Neural Networks Understand Logical Entailment?

We introduce a new dataset of logical entailments for the purpose of measuring models' ability to capture and exploit the structure of logical expressions against an entailment prediction task. We use this task to compare a series of architectures which are ubiquitous in the sequence-processing literature, in addition to a new model class---PossibleWorldNets---which computes entailment as a "convolution over possible worlds". Results show that convolutional networks present the wrong inductive bias for this class of problems relative to LSTM RNNs, tree-structured neural networks outperform LSTM RNNs due to their enhanced ability to exploit the syntax of logic, and PossibleWorldNets outperform all benchmarks.

cs.NE↗

arXiv2017

Programmable Agents

We build deep RL agents that execute declarative programs expressed in formal language. The agents learn to ground the terms in this language in their environment, and can generalize their behavior at test time to execute new programs that refer to objects that were not referenced during training. The agents develop disentangled interpretable representations that allow them to generalize to a wide variety of zero-shot semantic tasks.

cs.AI↗

arXiv2016

Online containers for hypergraphs, with applications to linear equations

A set of containers for a hypergraph G is a collection of vertex subsets, such that for every independent (or, indeed, merely sparse) set in G there is some subset in the collection which contains it. No set in the collection should be large and the collection itself should be relatively small. Containers with useful properties have been exhibited by Balogh, Morris and Samotij and by the authors, along with several applications. Our purpose here is to give a simpler algorithm than the one we used previously, which nevertheless yields containers with all the properties needed for our previous theorem. Moreover this algorithm produces containers having the so-called online property, allowing previous colouring applications to be extended to all, not just simple, hypergraphs. For illustrative purposes, we include a complete proof of a slightly weaker but simpler version of the theorem, which for many (perhaps most) applications is plenty. We also present applications to the number of solution-free sets of linear equations, including the number of Sidon sets, announced previously but not proven.

math.CO↗

arXiv2016

Unifying Count-Based Exploration and Intrinsic Motivation

We consider an agent's uncertainty about its environment and the problem of generalizing this uncertainty across observations. Specifically, we focus on the problem of exploration in non-tabular reinforcement learning. Drawing inspiration from the intrinsic motivation literature, we use density models to measure uncertainty, and propose a novel algorithm for deriving a pseudo-count from an arbitrary density model. This technique enables us to generalize count-based exploration algorithms to the non-tabular case. We apply our ideas to Atari 2600 games, providing sensible pseudo-counts from raw pixels. We transform these pseudo-counts into intrinsic rewards and obtain significantly improved exploration in a number of hard games, including the infamously difficult Montezuma's Revenge.

cs.AI↗

arXiv2014

Simple containers for simple hypergraphs

We give an easy method for constructing containers for simple hypergraphs. Some applications are given; in particular, a very transparent calculation is offered for the number of H-free hypergraphs, where H is some fixed uniform hypergraph.

math.CO↗

arXiv2013

The number of $C_{2l}$-free graphs

One of the most basic questions one can ask about a graph $H$ is: how many $H$-free graphs on $n$ vertices are there? For non-bipartite $H$, the answer to this question has been well-understood since 1986, when Erd\H{o}s, Frankl and R\"odl proved that there are $2^{(1 + o(1)) ex(n,H)}$ such graphs. For bipartite graphs, however, much less is known: even the weaker bound $2^{O(ex(n,H))}$ has been proven in only a few special cases: for cycles of length four and six, and for some complete bipartite graphs. For even cycles, Bondy and Simonovits proved in the 1970s that ex$(n,C_{2l}) = O( n^{1 + 1/l} )$, and this bound is conjectured to be sharp up to the implicit constant. In this paper we prove that the number of $C_{2l}$-free graphs on $n$ vertices is at most $2^{O(n^{1 + 1/l})}$, confirming a conjecture of Erd\H{o}s. Our proof uses the hypergraph container method, which was developed recently (and independently) by Balogh, Morris and Samotij, and by Saxton and Thomason, together with a new 'balanced supersaturation theorem' for even cycles. We moreover show that there are at least $2^{(1 + c)ex(n,C_6)}$ $C_6$-free graphs on $n$ vertices for some $c > 0$ and infinitely many values of $n$, disproving a well-known and natural conjecture. As a further application of our method, we essentially resolve the so-called Tur\'an problem on the Erd\H{o}s-R\'enyi random graph $G(n,p)$ for both even cycles and complete bipartite graphs.

math.CO↗

arXiv2013

The Ramsey number of the clique and the hypercube

The Ramsey number r(K_s,Q_n) is the smallest positive integer N such that every red-blue colouring of the edges of the complete graph K_N on N vertices contains either a red n-dimensional hypercube, or a blue clique on s vertices. Answering a question of Burr and Erd\H{o}s from 1983, and improving on recent results of Conlon, Fox, Lee and Sudakov, and of the current authors, we show that r(K_s,Q_n) = (s-1) (2^n - 1) + 1 for every s \in \N and every sufficiently large n \in \N.

math.CO↗

arXiv2013

On the Ramsey number of the triangle and the cube

The Ramsey number r(K_3,Q_n) is the smallest integer N such that every red-blue colouring of the edges of the complete graph K_N contains either a red n-dimensional hypercube, or a blue triangle. Almost thirty years ago, Burr and Erdős conjectured that r(K_3,Q_n) = 2^{n+1} - 1 for every n \in \N, but the first non-trivial upper bound was obtained only recently, by Conlon, Fox, Lee and Sudakov, who proved that r(K_3,Q_n) \le 7000 \cdot 2^n. Here we show that r(K_3,Q_n) = (1 + o(1)) 2^{n+1} as n \to \infty.

math.CO↗

arXiv2012

Hypergraph containers

We develop a notion of containment for independent sets in hypergraphs. For every $r$-uniform hypergraph $G$, we find a relatively small collection $C$ of vertex subsets, such that every independent set of $G$ is contained within a member of $C$, and no member of $C$ is large; the collection, which is in various respects optimal, reveals an underlying structure to the independent sets. The containers offer a straightforward and unified approach to many combinatorial questions concerned (usually implicitly) with independence. With regard to colouring, it follows that simple $r$-uniform hypergraphs of average degree $d$ have list chromatic number at least $(1/(r-1)^2 + o(1)) \log_r d$. For $r = 2$ this improves a bound due to Alon and is tight. For $r \ge 3$, previous bounds were weak but the present inequality is close to optimal. In the context of extremal graph theory, it follows that, for each $\ell$-uniform hypergraph $H$ of order $k$, there is a collection $C$ of $\ell$-uniform hypergraphs of order $n$ each with $o(n^k)$ copies of $H$, such that every $H$-free $\ell$-uniform hypergraph of order $n$ is a subgraph of a hypergraph in $C$, and $\log |C| \le c n^{\ell-1/m(H)} \log n$ where $m(H)$ is a standard parameter (there is a similar statement for induced subgraphs). This yields simple proofs, for example, for the number of $H$-free hypergraphs, and for the sparsity theorems of Conlon-Gowers and Schacht. A slight variant yields a counting version of the K{\L}R conjecture. Likewise, for systems of linear equations the containers supply, for example, bounds on the number of solution-free sets, and the existence of solutions in sparse random subsets. Balogh, Morris and Samotij have independently obtained related results.

math.CO↗

arXiv2011

Supersaturation for hereditary properties

Let $\mathcal{F}$ be a collection of $r$-uniform hypergraphs, and let $0 < p < 1$. It is known that there exists $c = c(p,\mathcal{F})$ such that the probability of a random $r$-graph in $G(n,p)$ not containing an induced subgraph from $\mathcal{F}$ is $2^{(-c+o(1)){n \choose r}}$. Let each graph in $\mathcal{F}$ have at least $t$ vertices. We show that in fact for every $ε> 0$, there exists $δ= δ(ε, p,\mathcal{F}) > 0$ such that the probability of a random $r$-graph in $G(n,p)$ containing less than $δn^t$ induced subgraphs each lying in $\mathcal{F}$ is at most $2^{(-c+ε){n \choose r}}$. This statement is an analogue for hereditary properties of the supersaturation theorem of Erdős and Simonovits. In our applications we answer a question of Bollobás and Nikiforov.

math.CO↗

arXiv2010

Strictly monotonic multidimensional sequences and stable sets in pillage games

Let $S \subset \mathbb{R}^n$ have size $|S| > \ell^{2^n-1}$. We show that there are distinct points $\{x^1,..., x^{\ell+1}\} \subset S$ such that for each $i \in [n]$, the coordinate sequence $(x^j_i)_{j=1}^{\ell+1}$ is strictly increasing, strictly decreasing, or constant, and that this bound on $|S|$ is best possible. This is analogous to the \erdos-Szekeres theorem on monotonic sequences in $\real$. We apply these results to bound the size of a stable set in a pillage game. We also prove a theorem of independent combinatorial interest. Suppose $\{a^1,b^1,...,a^t,b^t\}$ is a set of $2t$ points in $\real^n$ such that the set of pairs of points not sharing a coordinate is precisely $\{\{a^1,b^1\},...,\{a^t,b^t\}\}$. We show that $t \leq 2^{n-1}$, and that this bound is best possible.

math.CO↗