arXiv subjects

Christopher S. Meiklejohn

Publications and source records attributed to Christopher S. Meiklejohn.

7 recordsLinked to original sources

arXiv2022

Distributed Execution Indexing

This work-in-progress report presents both the design and partial evaluation of distributed execution indexing, a technique for microservice applications that precisely identifies dynamic instances of inter-service remote procedure calls (RPCs). Such an indexing scheme is critical for request-level fault injection techniques, which aim to automatically find failure-handling bugs in microservice applications.Distributed execution indexes enable granular specification of request-level faults, while also establishing a correspondence between inter-service RPCs across multiple executions, as is required to perform a systematic search of the fault space.In this paper, we formally define the general concept of a distributed execution index, which can be parameterized on different ways of identifying an RPC in a single service. We identify an instantiation that maintains precision in the presence of a variety of program structure complexities such as loops, function indirection, and concurrency with scheduling nondeterminism. We demonstrate that this particular instantiation addresses gaps in the state-of-the-art in request-level fault injection and show that they are all special cases of distributed execution indexing. We discuss the implementation challenges and provide an implementation of distributed execution indexing as an extension of \Filibuster{}, a resilience testing tool for microservice applications for the Java programming language, which supports fault injection for gRPC and HTTP.

cs.DC↗

arXiv2021

Serverless Workflows with Durable Functions and Netherite

Serverless is an increasingly popular choice for service architects because it can provide elasticity and load-based billing with minimal developer effort. A common and important use case is to compose serverless functions and cloud storage into reliable workflows. However, existing solutions for authoring workflows provide a rudimentary experience compared to writing standard code in a modern programming language. Furthermore, executing workflows reliably in an elastic serverless environment poses significant performance challenges. To address these, we propose Durable Functions, a programming model for serverless workflows, and Netherite, a distributed execution engine to execute them efficiently. Workflows in Durable Functions are expressed as task-parallel code in a host language of choice. Internally, the workflows are translated to fine-grained stateful communicating processes, which are load-balanced over an elastic cluster. The main challenge is to minimize the cost of reliably persisting progress to storage while supporting elastic scale. Netherite solves this by introducing partitioning, recovery logs, asynchronous snapshots, and speculative communication. Our results show that Durable Functions simplifies the expression of complex workflows, and that Netherite achieves lower latency and higher throughput than the prevailing approaches for serverless workflows in Azure and AWS, by orders of magnitude in some cases.

cs.DC↗

arXiv2018

Towards A Systems Approach To Distributed Programming

It is undeniable that most developers today are building distributed applications. However, most of these applications are developed by composing existing systems together through unspecified APIs exposed to the application developer. Systems are not going away: they solve a particular problem and most applications today need to rely on several of these systems working in concert. Given this, we propose a research direction where higher-level languages with well defined semantics target underlying systems infrastructure as a middle-ground.

cs.DC↗

arXiv2017

Practical Evaluation of the Lasp Programming Model at Large Scale - An Experience Report

Programming models for building large-scale distributed applications assist the developer in reasoning about consistency and distribution. However, many of the programming models for weak consistency, which promise the largest scalability gains, have little in the way of evaluation to demonstrate the promised scalability. We present an experience report on the implementation and large-scale evaluation of one of these models, Lasp, originally presented at PPDP `15, which provides a declarative, functional programming style for distributed applications. We demonstrate the scalability of Lasp's prototype runtime implementation up to 1024 nodes in the Amazon cloud computing environment. It achieves high scalability by uniquely combining hybrid gossip with a programming model based on convergent computation. We report on the engineering challenges of this implementation and its evaluation, specifically related to operating research prototypes in a production cloud environment.

cs.DC↗

arXiv2017

A Certain Tendency Of The Database Community

We posit that striving for distributed systems that provide "single system image" semantics is fundamentally flawed and at odds with how systems operate in the physical world. We realize the database as an optimization of this system: a required, essential optimization in practice that facilitates central data placement and ease of access to participants in a system. We motivate a new model of computation that is designed to address the problems of computation over "eventually consistent" information in a large-scale distributed system.

cs.DC↗

arXiv2017

On the Design of Distributed Programming Models

Programming large-scale distributed applications requires new abstractions and models to be done well. We demonstrate that these models are possible. Following from both the FLP result and CAP theorem, we show that concurrent programming models are necessary, but not sufficient, in the construction of large-scale distributed systems because of the problem of failure and network partitions: languages need to be able to capture and encode the tradeoffs between consistency and availability. We present two programming models, Lasp and Austere, each of which makes a strong tradeoff with respects to the CAP theorem. These two models outline the bounds of distributed model design: strictly AP or strictly CP. We argue that all possible distributed programming models must come from this design space, and present one practical design that allows declarative specification of consistency tradeoffs, called Spry.

cs.DC↗

arXiv2016

Dynamic Path Contraction for Distributed, Dynamic Dataflow Languages

We present a work in progress report on applying deforestation to distributed, dynamic dataflow programming models. We propose a novel algorithm, dynamic path contraction, that applies and reverses optimizations to a distributed dataflow application as the program executes. With this algorithm, data and control flow is tracked by the runtime system used to identify potential optimizations as the system is running. We demonstrate and present preliminary results regarding this technique on an actor-based distributed programming model, Lasp, implemented on the Erlang virtual machine.

cs.DC↗