Agora: Bridging the GPU Cloud Resource-Price Disconnect

Abstract

The historic trend of Moore's Law, which predicted exponential growth in computational performance per dollar, has diverged for modern Graphics Processing Units (GPUs). While Floating Point Operations per Second (FLOPs) capabilities have continued to scale economically, memory bandwidth has not, creating a significant price-performance disconnect. This paper argues that the prevailing time-based pricing models for cloud GPUs are economically inefficient for bandwidth-bound workloads. These models fail to account for the rising marginal cost of memory bandwidth, leading to market distortions and suboptimal hardware allocation. To address this, we propose a novel feature-based pricing framework that directly links cost to resource consumption, including but not limited to memory bandwidth. We provide a robust economic and algorithmic definition of this framework and introduce Agora, a practical and secure system architecture for its implementation. Our implementation of Agora shows that a 50us sampling provides nearly perfect pricing as what ideal sampling would provide - losing only 5\% of revenue. 10us sampling is even better result in 2.4\% loss. Modern telemetry systems can already provide this rate of measurement, and our prototype implementation shows the system design for feature-based pricing is buildable. Our evaluation across diverse GPU applications and hardware generations empirically validates the effectiveness of our approach in creating a more transparent and efficient market for cloud GPU resources.