arXiv subjects

Jelle Hellings

Publications and source records attributed to Jelle Hellings.

12 recordsLinked to original sources

arXiv2023

SpotLess: Concurrent Rotational Consensus Made Practical through Rapid View Synchronization

The emergence of blockchain technology has renewed the interest in consensus-based data management systems that are resilient to failures. To maximize the throughput of these systems, we have recently seen several prototype consensus solutions that optimize for throughput at the expense of overall implementation complexity, high costs, and reliability. Due to this, it remains unclear how these prototypes will perform in real-world environments. In this paper, we present SPOTLESS, a novel concurrent rotational consensus protocol made practical. Central to SPOTLESS is the combination of (1) a chained rotational consensus design for replicating requests with a reduced message cost and low-cost failure recovery that eliminates the traditional complex, error-prone view-change protocol; (2) the novel Rapid View Synchronization protocol that enables SPOTLESS to work in more general network assumptions, without a need for a Global Synchronization Time to synchronize view, and recover valid earlier views with the aid of non-faulty replicas without the need to rely on the primary; (3) a high-performance concurrent consensus architecture in which independent instances of the chained consensus operate concurrently to process requests with high throughput, thereby avoiding the bottlenecks seen in other rotational protocols. Due to the concurrent consensus architecture, SPOTLESS greatly outperforms traditional primary-backup consensus protocols such as PBFT (by up to 430%), NARWHAL-HS (by up to 137%), and HOTSTUFF (by up to 3803%). Due to its reduced message cost, SPOTLESS is even able to outperform RCC, a state-of-the-art high-throughput concurrent consensus protocol, by up to 23%. Furthermore, SPOTLESS is able to maintain a stable and low latency and consistently high throughput even during failures.

cs.DB↗

arXiv2022

On the Correctness of Speculative Consensus

The introduction of Bitcoin fueled the development of blockchain-based resilient data management systems that are resilient against failures, enable federated data management, and can support data provenance. The key factor determining the performance of such resilient data management systems is the consensus protocol used by the system to replicate client transactions among all participants. Unfortunately, existing high-throughput consensus protocols are costly and impose significant latencies on transaction processing, which rules out their usage in responsive high-performance data management systems. In this work, we improve on this situation by introducing the Proof-of-Execution consensus protocol (PoE), a consensus protocol designed for high-performance low-latency resilient data management. PoE introduces speculative execution, which minimizes latencies by starting execution before consensus is reached, and PoE introduces proof-of-executions to guarantee successful execution to clients. Furthermore, PoE introduces a single-round check-commit protocol to reduce the overall communication costs of consensus. Hence, we believe that PoE is a promising step towards flexible general-purpose low-latency resilient data management systems.

cs.DB↗

arXiv2021

Byzantine Cluster-Sending in Expected Constant Communication

Traditional resilient systems operate on fully-replicated fault-tolerant clusters, which limits their scalability and performance. One way to make the step towards resilient high-performance systems that can deal with huge workloads, is by enabling independent fault-tolerant clusters to efficiently communicate and cooperate with each other, as this also enables the usage of high-performance techniques such as sharding and parallel processing. Recently, such inter-cluster communication was formalized as the Byzantine cluster-sending problem, and worst-case optimal protocols have been proposed that solve this problem. Unfortunately, these protocols have an all-case linear complexity in the size of the clusters involved. In this paper, we propose probabilistic cluster-sending techniques that can reliably send messages from one Byzantine fault-tolerant cluster to another with only an expected constant message complexity, this independent of the size of the clusters involved. Depending on the robustness of the clusters involved, our techniques require only two-to-four message round-trips. Furthermore, our protocols can support worst-case linear communication between clusters, which is optimal, and deal with asynchronous and unreliable communication. As such, our work provides a strong foundation for the further development of resilient high-performance systems.

cs.DB↗

arXiv2021

Proof-of-Execution: Reaching Consensus through Fault-Tolerant Speculation

Multi-party data management and blockchain systems require data sharing among participants. To provide resilient and consistent data sharing, transactions engines rely on Byzantine FaultTolerant consensus (BFT), which enables operations during failures and malicious behavior. Unfortunately, existing BFT protocols are unsuitable for high-throughput applications due to their high computational costs, high communication costs, high client latencies, and/or reliance on twin-paths and non-faulty clients. In this paper, we present the Proof-of-Execution consensus protocol (PoE) that alleviates these challenges. At the core of PoE are out-of-order processing and speculative execution, which allow PoE to execute transactions before consensus is reached among the replicas. With these techniques, PoE manages to reduce the costs of BFT in normal cases, while guaranteeing reliable consensus for clients in all cases. We envision the use of PoE in high-throughput multi-party data-management and blockchain systems. To validate this vision, we implement PoE in our efficient ResilientDB fabric and extensively evaluate PoE against several state-of-the-art BFT protocols. Our evaluation showcases that PoE achieves up-to-80% higher throughputs than existing BFT protocols in the presence of failures.

cs.DB↗

arXiv2020

RCC: Resilient Concurrent Consensus for High-Throughput Secure Transaction Processing

Recently, we saw the emergence of consensus-based database systems that promise resilience against failures, strong data provenance, and federated data management. Typically, these fully-replicated systems are operated on top of a primary-backup consensus protocol, which limits the throughput of these systems to the capabilities of a single replica (the primary). To push throughput beyond this single-replica limit, we propose concurrent consensus. In concurrent consensus, replicas independently propose transactions, thereby reducing the influence of any single replica on performance. To put this idea in practice, we propose our RCC paradigm that can turn any primary-backup consensus protocol into a concurrent consensus protocol by running many consensus instances concurrently. RCC is designed with performance in mind and requires minimal coordination between instances. Furthermore, RCC also promises increased resilience against failures. We put the design of RCC to the test by implementing it in ResilientDB, our high-performance resilient blockchain fabric, and comparing it with state-of-the-art primary-backup consensus protocols. Our experiments show that RCC achieves up to 2.75x higher throughput than other consensus protocols and can be scaled to 91 replicas.

cs.DB↗

arXiv2020

Cerberus: Minimalistic Multi-shard Byzantine-resilient Transaction Processing

To enable high-performance and scalable blockchains, we need to step away from traditional consensus-based fully-replicated designs. One direction is to explore the usage of sharding in which we partition the managed dataset over many shards that independently operate as blockchains. Sharding requires an efficient fault-tolerant primitive for the ordering and execution of multi-shard transactions, however. In this work, we seek to design such a primitive suitable for distributed ledger networks with high transaction throughput. To do so, we propose Cerberus, a set of minimalistic primitives for processing single-shard and multi-shard UTXO-like transactions. Cerberus aims at maximizing parallel processing at shards while minimizing coordination within and between shards. First, we propose Core-Cerberus, that uses strict environmental requirements to enable simple yet powerful multi-shard transaction processing. In our intended UTXO-environment, Core-Cerberus will operate perfectly with respect to all transactions proposed and approved by well-behaved clients, but does not provide any guarantees for other transactions. To also support more general-purpose environments, we propose two generalizations of Core-Cerberus: we propose Optimistic-Cerberus, a protocol that does not require any additional coordination phases in the well-behaved optimistic case, while requiring intricate coordination when recovering from attacks; and we propose Pessimistic-Cerberus, a protocol that adds sufficient coordination to the well-behaved case of Core-Cerberus, allowing it to operate in a general-purpose fault-tolerant environments without significant costs to recover from attacks. Finally, we compare the three protocols, showing their potential scalability and high transaction throughput in practical environments.

cs.DC↗

arXiv2020

ResilientDB: Global Scale Resilient Blockchain Fabric

Recent developments in blockchain technology have inspired innovative new designs in resilient distributed and database systems. At their core, these blockchain applications typically use Byzantine fault-tolerant consensus protocols to maintain a common state across all replicas, even if some replicas are faulty or malicious. Unfortunately, existing consensus protocols are not designed to deal with geo-scale deployments in which many replicas spread across a geographically large area participate in consensus. To address this, we present the Geo-Scale Byzantine FaultTolerant consensus protocol (GeoBFT). GeoBFT is designed for excellent scalability by using a topological-aware grouping of replicas in local clusters, by introducing parallelization of consensus at the local level, and by minimizing communication between clusters. To validate our vision of high-performance geo-scale resilient distributed systems, we implement GeoBFT in our efficient ResilientDB permissioned blockchain fabric. We show that GeoBFT is not only sound and provides great scalability, but also outperforms state-of-the-art consensus protocols by a factor of six in geo-scale deployments.

cs.DB↗

arXiv2019

Revisiting consensus protocols through wait-free parallelization

The recent surge of blockchain systems has renewed the interest in traditional Byzantine fault-tolerant consensus protocols. Many such consensus protocols have a primary-backup design in which an assigned replica, the primary, is responsible for coordinating the consensus protocol. Although the primary-backup design leads to relatively simple and high performance consensus protocols, it places an unreasonable burden on a good primary and allows malicious primaries to substantially affect the system performance. In this paper, we propose a protocol-agnostic approach to improve the design of primary backup consensus protocols. At the core of our approach is a novel wait-free approach of running several instances of the underlying consensus protocol in parallel. To yield a high performance parallelized design, we present coordination-free techniques to order operations across parallel instances, deal with instance failures, and assign clients to specific instances. Consequently, the design we present is able to reduce the load on individual instances and primaries, while also reducing the adverse effects of any malicious replicas.

cs.DC↗

arXiv2019

The fault-tolerant cluster-sending problem

The development of fault-tolerant distributed systems that can tolerate Byzantine behavior has traditionally been focused on consensus protocols, which support fully-replicated designs. For the development of more sophisticated high-performance Byzantine distributed systems, more specialized fault-tolerant communication primitives are necessary, however. In this paper, we identify an essential communication primitive and study it in depth. In specifics, we formalize the cluster-sending problem, the problem of sending a message from one Byzantine cluster to another Byzantine cluster in a reliable manner. We not only formalize this fundamental problem, but also establish lower bounds on the complexity of this problem under crash failures and Byzantine failures. Furthermore, we develop practical cluster-sending protocols that meet these lower bounds and, hence, have optimal complexity. As such, our work provides a strong foundation for the further exploration of novel designs that address challenges encountered in fault-tolerant distributed systems.

cs.DC↗

arXiv2018

Comparing Downward Fragments of the Relational Calculus with Transitive Closure on Trees

Motivated by the continuing interest in the tree data model, we study the expressive power of downward navigational query languages on trees and chains. Basic navigational queries are built from the identity relation and edge relations using composition and union. We study the effects on relative expressiveness when we add transitive closure, projections, coprojections, intersection, and difference; this for boolean queries and path queries on labeled and unlabeled structures. In all cases, we present the complete Hasse diagram. In particular, we establish, for each query language fragment that we study on trees, whether it is closed under difference and intersection.

cs.DB↗

arXiv2016

Querying for Paths in Graphs using Context-Free Path Queries

Navigational queries for graph-structured data, such as the regular path queries and the context-free path queries, are usually evaluated to a relation of node-pairs $(m, n)$ such that there is a path from $m$ to $n$ satisfying the conditions of the query. Although this relational query semantics has practical value, we believe that the relational query semantics can only provide limited insight in the structure of the graph data. To address the limits of the relational query semantics, we introduce the all-path query semantics and the single-path query semantics. Under these path-based query semantics, a query is evaluated to all paths satisfying the conditions of the query, or, respectively, to a single such path. While focusing on context-free path queries, we provide a formal framework for evaluating queries on graphs using both path-based query semantics. For the all-path query semantics, we show that the result of a query can be represented by a finite context-free grammar annotated with node-information relevant for deriving each path in the query result. For the single-path query semantics, we propose to search for a path of minimum length. We reduce the problem of finding such a path of minimum length to finding a string of minimum length in a context-free language, and for deriving such a string we propose a novel algorithm. Our initial results show that the path-based query semantics have added practical value and that query evaluation for both path-based query semantics is feasible, even when query results grow very large. For the single-path query semantics, determining strict worst-case upper bounds on the size of the query result remains the focus of future work.

cs.DB↗

arXiv2011

I/O efficient bisimulation partitioning on very large directed acyclic graphs

In this paper we introduce the first efficient external-memory algorithm to compute the bisimilarity equivalence classes of a directed acyclic graph (DAG). DAGs are commonly used to model data in a wide variety of practical applications, ranging from XML documents and data provenance models, to web taxonomies and scientific workflows. In the study of efficient reasoning over massive graphs, the notion of node bisimilarity plays a central role. For example, grouping together bisimilar nodes in an XML data set is the first step in many sophisticated approaches to building indexing data structures for efficient XPath query evaluation. To date, however, only internal-memory bisimulation algorithms have been investigated. As the size of real-world DAG data sets often exceeds available main memory, storage in external memory becomes necessary. Hence, there is a practical need for an efficient approach to computing bisimulation in external memory. Our general algorithm has a worst-case IO-complexity of O(Sort(|N| + |E|)), where |N| and |E| are the numbers of nodes and edges, resp., in the data graph and Sort(n) is the number of accesses to external memory needed to sort an input of size n. We also study specializations of this algorithm to common variations of bisimulation for tree-structured XML data sets. We empirically verify efficient performance of the algorithms on graphs and XML documents having billions of nodes and edges, and find that the algorithms can process such graphs efficiently even when very limited internal memory is available. The proposed algorithms are simple enough for practical implementation and use, and open the door for further study of external-memory bisimulation algorithms. To this end, the full open-source C++ implementation has been made freely available.

cs.DS↗