Fundamentals of Caching Layered Data objects

Abstract

The effective management of large amounts of data processed or required by today's cloud or edge computing systems remains a fundamental challenge. This paper focuses on cache management for applications where data objects can be stored in layered representations. In such representations, each additional data layer enhances the "quality" of the object's version but comes with an incremental cost of memory space. This layered approach proves beneficial in various scenarios, including the delivery of zoomable maps, video coding, future Virtual Reality gaming, and layered neural network models where additional data layers improve inference accuracy. In systems where users or devices demand different versions of a data object, layered representations offer flexibility for caching policies to achieve improved hit rates. In this paper, we explore the performance of various traditionally studied caching policies, such as Belady, LRU, and LFU, both with and without layering. To this end, we develop an asymptotically accurate analytical model for Layered LRU (LLRU). We study how the performance of LLRU is impacted by factors such as the number of layers, the popularity of different objects and layers, and overheads associated with storing layered representations. For instance, we show that, for LLRU, more layers are not always beneficial and indeed performance depends in subtle ways on the popularity and size profiles of layers.