FaaSched: A Jitter-Aware Serverless Scheduler

Abstract

Serverless computing systems are becoming very popular. Large corporations such as Netflix, Airbnb, and Coca-Cola use such systems for running their websites and IT systems. The advantages of such systems include superior support for auto-scaling, load balancing, and fast distributed processing. These are multi-QoS systems where different classes of applications have different latency and jitter (variation in the latency) requirements: we consider a mix of latency-sensitive (LS) and latency-desirable (LD) applications. Ensuring proper schedulability and QoS enforcement of LS applications is non-trivial. We need to minimize the jitter without increasing the response latency of LS applications, and we also need to keep the degradation of the response latency of LD applications in check. This is the first paper in this domain that achieves a trade-off between the jitter suffered by LS applications and the response latency of LD applications. We minimize the former with a bound on the latter using a reinforcement learning (RL) based scheme. To design such an RL scheme, we performed detailed characterization studies to find the input variables of interest, defined novel state representations, and proposed a bespoke reward function that allows us to achieve this trade-off. For an aggressive use case comprising five popular LS and LD applications each, we show a reduction in response time variance and mean latency of 50.31% and 27.4%, respectively, for LS applications. The mean degradation in the execution latency of LD applications was limited to 19.88%.