arXiv subjects

Andrew Lewis-Pye

Publications and source records attributed to Andrew Lewis-Pye.

35 records · Page 2Linked to original sources

arXiv2023

Byzantine Generals in the Permissionless Setting

Consensus protocols have traditionally been studied in a setting where all participants are known to each other from the start of the protocol execution. In the parlance of the 'blockchain' literature, this is referred to as the permissioned setting. What differentiates Bitcoin from these previously studied protocols is that it operates in a permissionless setting, i.e. it is a protocol for establishing consensus over an unknown network of participants that anybody can join, with as many identities as they like in any role. The arrival of this new form of protocol brings with it many questions. Beyond Bitcoin, what can we prove about permissionless protocols in a general sense? How does recent work on permissionless protocols in the blockchain literature relate to the well-developed history of research on permissioned protocols in distributed computing? To answer these questions, we describe a formal framework for the analysis of both permissioned and permissionless systems. Our framework allows for "apples-to-apples" comparisons between different categories of protocols and, in turn, the development of theory to formally discuss their relative merits. A major benefit of the framework is that it facilitates the application of a rich history of proofs and techniques in distributed computing to problems in blockchain and the study of permissionless systems. Within our framework, we then address the questions above. We consider the Byzantine Generals Problem as a formalisation of the problem of reaching consensus, and address a programme of research that asks, "Under what adversarial conditions, and for what types of permissionless protocol, is consensus possible?" We prove a number of results for this programme, our main result being that deterministic consensus is not possible for decentralised permissionless protocols. To close, we give a list of eight open questions.

cs.DC↗

arXiv2022

Quadratic worst-case message complexity for State Machine Replication in the partial synchrony model

We consider the message complexity of State Machine Replication protocols dealing with Byzantine failures in the partial synchrony model. A result of Dolev and Reischuk gives a quadratic lower bound for the message complexity, but it was unknown whether this lower bound is tight, with the most efficient known protocols giving worst-case message complexity $O(n^3)$. We describe a protocol which meets Dolev and Reischuk's quadratic lower bound, while also satisfying other desirable properties. To specify these properties, suppose that we have $n$ replicas, $f$ of which display Byzantine faults (with $n\geq 3f+1$). Suppose that $Δ$ is an upper bound on message delay, i.e. if a message is sent at time $t$, then it is received by time $ \text{max} \{ t, GST \} +Δ$. We describe a deterministic protocol that simultaneously achieves $O(n^2)$ worst-case message complexity, optimistic responsiveness, $O(fΔ)$ time to first confirmation after $GST$ and $O(n)$ mean message complexity.

cs.DC↗

arXiv2021

How Does Blockchain Security Dictate Blockchain Implementation?

Blockchain protocols come with a variety of security guarantees. For example, BFT-inspired protocols such as Algorand tend to be secure in the partially synchronous setting, while longest chain protocols like Bitcoin will normally require stronger synchronicity to be secure. Another fundamental distinction, directly relevant to scalability solutions such as sharding, is whether or not a single untrusted user is able to point to *certificates*, which provide incontrovertible proof of block confirmation. Algorand produces such certificates, while Bitcoin does not. Are these properties accidental? Or are they inherent consequences of the paradigm of protocol design? Our aim in this paper is to understand what, fundamentally, governs the nature of security for permissionless blockchain protocols. Using the framework developed in (Lewis-Pye and Roughgarden, 2021), we prove general results showing that these questions relate directly to properties of the user selection process, i.e., the method (such as proof-of-work or proof-of-stake) which is used to select users with the task of updating state. Our results suffice to establish, for example, that the production of certificates is impossible for proof-of-work protocols, but is automatic for standard forms of proof-of-stake protocols. As a byproduct of our work, we also define a number of security notions and identify the equivalences and inequivalences among them.

cs.CR↗

arXiv2021

A General Framework for the Security Analysis of Blockchain Protocols

Blockchain protocols differ in fundamental ways, including the mechanics of selecting users to produce blocks (e.g., proof-of-work vs. proof-of-stake) and the method to establish consensus (e.g., longest chain rules vs. Byzantine fault-tolerant (BFT) inspired protocols). These fundamental differences have hindered "apples-to-apples" comparisons between different categories of blockchain protocols and, in turn, the development of theory to formally discuss their relative merits. This paper presents a parsimonious abstraction sufficient for capturing and comparing properties of many well-known permissionless blockchain protocols, simultaneously capturing essential properties of both proof-of-work (PoW) and proof-of-stake (PoS) protocols, and of both longest-chain-type and BFT-type protocols. Our framework blackboxes the precise mechanics of the user selection process, allowing us to isolate the properties of the selection process that are significant for protocol design. We demonstrate the utility of our general framework with several concrete results: 1. We prove a CAP-type impossibility theorem asserting that liveness with an unknown level of participation rules out security in a partially synchronous setting. 2. Delving deeper into the partially synchronous setting, we prove that a necessary and sufficient condition for security is the production of "certificates," meaning stand-alone proofs of block confirmation. 3. Restricting to synchronous settings, we prove that typical protocols with a known level of participation (including longest chain-type PoS protocols) can be adapted to provide certificates, but those with an unknown level of participation cannot. 4. Finally, we use our framework to articulate a modular two-step approach to blockchain security analysis that effectively reduces the permissionless case to the permissioned case.

cs.DC↗

arXiv2020

Resource Pools and the CAP Theorem

Blockchain protocols differ in fundamental ways, including the mechanics of selecting users to produce blocks (e.g., proof-of-work vs. proof-of-stake) and the method to establish consensus (e.g., longest chain rules vs. BFT-inspired protocols). These fundamental differences have hindered "apples-to-apples" comparisons between different categories of blockchain protocols and, in turn, the development of theory to formally discuss their relative merits. This paper presents a parsimonious abstraction sufficient for capturing and comparing properties of many well-known permissionless blockchain protocols, simultaneously capturing essential properties of both proof-of-work and proof-of-stake protocols, and of both longest-chain-type and BFT-type protocols. Our framework blackboxes the precise mechanics of the user selection process, allowing us to isolate the properties of the selection process which are significant for protocol design. We illustrate our framework's utility with two results. First, we prove an analog of the CAP theorem from distributed computing for our framework in a partially synchronous setting. This theorem shows that a fundamental dichotomy holds between protocols (such as Bitcoin) that are adaptive, in the sense that they can function given unpredictable levels of participation, and protocols (such as Algorand) that have certain finality properties. Second, we formalize the idea that proof-of-work (PoW) protocols and non-PoW protocols can be distinguished by the forms of permission that users are given to carry out updates to the state.

cs.DC↗

arXiv2019

Monotonous betting strategies in warped casinos

Suppose that the outcomes of a roulette table are not entirely random, in the sense that there exists a successful betting strategy. Is there a successful `separable' strategy, in the sense that it does not use the winnings from betting on red in order to bet on black, and vice-versa? We study this question from an algorithmic point of view and observe that every strategy $M$ can be replaced by a separable strategy which is computable from $M$ and successful on any outcome-sequence where $M$ is successful. We then consider the case of mixtures and show: (a) there exists an effective mixture of separable strategies which succeeds on every casino sequence with effective Hausdorff dimension less than 1/2; (b) there exists a casino sequence of effective Hausdorff dimension 1/2 on which no effective mixture of separable strategies succeeds. Finally we extend (b) to a more general class of strategies.

math.LO↗

arXiv2019

Compression of data streams down to their information content

According to Kolmogorov complexity, every finite binary string is compressible to a shortest code -- its information content -- from which it is effectively recoverable. We investigate the extent to which this holds for infinite binary sequences (streams). We devise a new coding method which uniformly codes every stream $X$ into an algorithmically random stream $Y$, in such a way that the first $n$ bits of $X$ are recoverable from the first $I(X\upharpoonright_n)$ bits of $Y$, where $I$ is any partial computable information content measure which is defined on all prefixes of $X$, and where $X\upharpoonright_n$ is the initial segment of $X$ of length $n$. As a consequence, if $g$ is any computable upper bound on the initial segment prefix-free complexity of $X$, then $X$ is computable from an algorithmically random $Y$ with oracle-use at most $g$. Alternatively (making no use of such a computable bound $g$) one can achieve an oracle-use bounded above by $K(X\upharpoonright_n)+\log n$. This provides a strong analogue of Shannon's source coding theorem for algorithmic information theory.

cs.IT↗

arXiv2019

The idemetric property: when most distances are (almost) the same

We introduce the \emph{idemetric} property, which formalises the idea that most nodes in a graph have similar distances between them, and which turns out to be quite standard amongst small-world network models. Modulo reasonable sparsity assumptions, we are then able to show that a strong form of idemetricity is actually equivalent to a very weak expander condition (PUMP). This provides a direct way of providing short proofs that small-world network models such as the Watts-Strogatz model are strongly idemetric (for a wide range of parameters), and also provides further evidence that being idemetric is a common property. We then consider how satisfaction of the idemetric property is relevant to algorithm design. For idemetric graphs we observe, for example, that a single breadth-first search provides a solution to the all-pairs shortest paths problem, so long as one is prepared to accept paths which are of stretch close to 2 with high probability. Since we are able to show that Kleinberg's model is idemetric, these results contrast nicely with the well known negative results of Kleinberg concerning efficient decentralised algorithms for finding short paths: for precisely the same model as Kleinberg's negative results hold, we are able to show that very efficient (and decentralised) algorithms exist if one allows for reasonable preprocessing. For deterministic distributed routing algorithms we are also able to obtain results proving that less routing information is required for idemetric graphs than the worst case in order to achieve stretch less than 3 with high probability: while $Ω(n^2)$ routing information is required in the worst case for stretch strictly less than 3 on almost all pairs, for idemetric graphs the total routing information required is $O(nlog(n))$.

cs.SI↗

arXiv2017

A note on the differences of computably enumerable reals

We show that given any non-computable left-c.e. real $α$ there exists a left-c.e. real $β$ such that $α\neq β+γ$ for all left-c.e. reals and all right-c.e. reals $γ$. The proof is non-uniform, the dichotomy being whether the given real $α$ is Martin-Loef random or not. It follows that given any universal machine $U$, there is another universal machine $V$ such that the halting probability of U is not a translation of the halting probability of V by a left-c.e. real. We do not know if there is a uniform proof of this fact.

math.LO↗

arXiv2017

Lower bounds on the redundancy in computations from random oracles via betting strategies with restricted wagers

The Kučera-Gács theorem is a landmark result in algorithmic randomness asserting that every real is computable from a Martin-Löf random real. If the computation of the first $n$ bits of a sequence requires $n+h(n)$ bits of the random oracle, then $h$ is the redundancy of the computation. Kučera implicitly achieved redundancy $n\log n$ while Gács used a more elaborate coding procedure which achieves redundancy $\sqrt{n}\log n$. A similar upper bound is implicit in the later proof by Merkle and Mihailović. In this paper we obtain strict optimal lower bounds on the redundancy in computations from Martin-Löf random oracles. We show that any nondecreasing computable function $g$ such that $\sum_n 2^{-g(n)}=\infty$ is not a general upper bound on the redundancy in computations from Martin-Löf random oracles. In fact, there exists a real $X$ such that the redundancy $g$ of any computation of $X$ from a Martin-Löf random oracle satisfies $\sum_n 2^{-g(n)}<\infty$. Moreover, the class of such reals is comeager and includes a $Δ^0_2$ real as well as all weakly 2-generic reals. This excludes many slow growing functions such as $\log n$ from bounding the redundancy in computations from random oracles for a large class of reals. On the other hand it was recently shown that if $\sum_n 2^{-g(n)}<\infty$ then $g$ is a general upper bound for the redundancy in computations of any real from some Martin-Löf random oracle. Our results are obtained as an application of a theory of effective betting strategies with restricted wagers which we develop.

cs.CC↗

arXiv2017

Optimal redundancy in computations from random oracles

A classic result in algorithmic information theory is that every infinite binary sequence is computable from a Martin-Loef random infinite binary sequence. Proved independently by Kucera and Gacs, this result answered a question by Charles Bennett and has seen numerous applications in the last 30 years. The optimal redundancy in such a coding process has, however, remained unknown. If the computation of the first n bits of a sequence requires n + g(n) bits of the random oracle, then g is the redundancy of the computation. Kucera implicitly achieved redundancy n log n while Gacs used a more elaborate block-coding procedure which achieved redundancy sqrt(n) log n. Different approaches to coding such as the one by Merkle and Mihailovic have not improved this redundancy bound. In this paper we devise a new coding method that achieves optimal logarithmic redundancy. Our redundancy bound is exponentially smaller than the previously best known bound and is known to be the best possible. It follows that redundancy r log n in computation from a random oracle is possible for every stream, if and only if r > 1.

cs.CC↗

arXiv2017

Differences of halting probabilities

The halting probabilities of universal prefix-free machines are universal for the class of reals with computably enumerable left cut (also known as left-c.e. reals), and coincide with the Martin-Loef random elements of this class. We study the differences of Martin-Loef random left-c.e. reals and show that for each pair of such reals a, b there exists a unique number r > 0 such that qa - b is a 1-random left-c.e. real for each positive rational q > r and a 1-random right-c.e. real for each positive rational q < r. Based on this result we develop a theory of differences of halting probabilities, which answers a number of questions about Martin-Loef random left-c.e. reals, including one of the few remaining open problems from the list of open questions in algorithmic randomness by Miller and Nies in 2006. The halting probability of a prefix-free machine M restricted to a set X is the probability that the machine halts and outputs an element of X. These numbers Omega_M(X) were studied by a number of authors in the last decade as a way to obtain concrete highly random numbers. When X is the complement of a computably enumerable set, the number Omega_M(X) is the difference of two halting probabilities. Becher, Figueira, Grigorieff, and Miller asked whether Omega_U(X) is Martin-Loef random when U is universal and X is the complement of a computably enumerable set. This problem has resisted numerous attempts in the last decade. We apply our theory of differences of halting probabilities to give a positive answer, and show that Omega_U(X) is a Martin-Loef random left-c.e. real whenever X is nonempty and the complement of a computably enumerable set.

cs.CC↗

arXiv2017

Limits of the Kucera-Gacs coding method

Every real is computable from a Martin-Loef random real. This well known result in algorithmic randomness was proved by Kucera and Gacs. In this survey article we discuss various approaches to the problem of coding an arbitrary real into a Martin-Loef random real,and also describe new results concerning optimal methods of coding. We start with a simple presentation of the original methods of Kucera and Gacs and then rigorously demonstrate their limitations in terms of the size of the redundancy in the codes that they produce. Armed with a deeper understanding of these methods, we then proceed to motivate and illustrate aspects of the new coding method that was recently introduced by Barmpalias and Lewis-Pye and which achieves optimal logarithmic redundancy, an exponential improvement over the original redundancy bounds.

math.LO↗

arXiv2017

Pointed computations and Martin-Löf randomness

Schnorr showed that a real is Martin-Loef random if and only if all of its initial segments are incompressible with respect to prefix-free complexity. Fortnow and independently Nies, Stephan and Terwijn noticed that this statement remains true if we can merely require that the initial segments of the real corresponding to a computable increasing sequence of lengths are incompressible. The purpose of this note is to establish the following generalization of this fact. We show that a real is X Martin-Loef random if and only if its initial segments corresponding to a pointedly X-computable sequence (r_n) (where r_n is computable from X in a self-delimiting way, so that at most the first r_n bits of X are queried in the computation) of lengths are incompressible. On the other hand we also show that there are reals which are very far from being Martin-Loef random, yet they compute an increasing sequence of lengths at which their initial segments are incompressible.

cs.CC↗

arXiv2016

Computing halting probabilities from other halting probabilities

The halting probability of a Turing machine is the probability that the machine will halt if it starts with a random stream written on its one-way input tape. When the machine is universal, this probability is referred to as Chaitin's omega number, and is the most well known example of a real which is random in the sense of Martin-Löf. Although omega numbers depend on the underlying universal Turing machine, they are robust in the sense that they all have the same Turing degree, namely the degree of the halting problem. In this paper we give precise bounds on the redundancy growth rate that is generally required for the computation of an omega number from another omega number. We show that for each $ε>1$, any pair of omega numbers compute each other with redundancy $ε\log n$. On the other hand, this is not true for $ε=1$. In fact, we show that for each omega number there exists another omega number which is not computable from the first one with redundancy $\log n$. This latter result improves an older result of Frank Stephan.

cs.CC↗

arXiv2016

Optimal asymptotic bounds on the oracle use in computations from Chaitin's Omega

Chaitin's number Omega is the halting probability of a universal prefix-free machine, and although it depends on the underlying enumeration of prefix-free machines, it is always Turing-complete. It can be observed, in fact, that for every computably enumerable (c.e.) real, there exists a Turing functional via which Omega computes it, and such that the number of bits of omega that are needed for the computation of the first n bits of the given number (i.e. the use on argument n) is bounded above by a computable function h(n) = n+o(n). We characterise the asymptotic upper bounds on the use of Chaitin's omega in oracle computations of halting probabilities (i.e. c.e. reals). We show that the following two conditions are equivalent for any computable function h such that h(n)-n is non-decreasing: (1) h(n)-n is an information content measure, (2) for every c.e. real there exists a Turing functional via which omega computes the real with use bounded by h. We also give a similar characterisation with respect to computations of c.e. sets from Omega, by showing that the following are equivalent for any computable non-decreasing function g: (1) g is an information-content measure, (2) for every c.e. set A, Omega computes A with use bounded by g. Further results and some connections with Solovay functions are given.

math.LO↗

arXiv2015

A mathematical analysis of the evolutionary benefits of sexual reproduction

The question as to why most higher organisms reproduce sexually has remained open despite extensive research, and has been called "the queen of problems in evolutionary biology". Theories dating back to Weismann have suggested that the key must lie in the creation of increased variability in offspring, causing enhanced response to selection. Rigorously quantifying the effects of assorted mechanisms which might lead to such increased variability, and establishing that these beneficial effects outweigh the immediate costs of sexual reproduction has, however, proved problematic. Here we introduce an approach which does not focus on particular mechanisms influencing factors such as the fixation of beneficial mutants or the ability of populations to deal with deleterious mutations, but rather tracks the entire distribution of a population of genotypes as it moves across vast fitness landscapes. In this setting simulations now show sex robustly outperforming asex across a broad spectrum of finite or infinite population models. Concentrating on the additive infinite populations model, we are able to give a rigorous mathematical proof establishing that sexual reproduction acts as a more efficient optimiser of mean fitness, thereby solving the problem for this model. Some of the key features of this analysis carry through to the finite populations case.

q-bio.PE↗