Searcharxiv⌕ Search

arXiv subjects

Indrani Paul

Publications and source records attributed to Indrani Paul.

2 recordsLinked to original sources

PACER: GPU Excursion Management for ML Workloads

Variability is an inherent characteristic of modern GPU systems, arising from a combination of process variation, dynamic environmental conditions, and power management decisions. Prior work has characterized inter-GPU variability and found consistent performance differences across GPUs. However, we observe that modern workloads, particularly large language models (LLMs), exhibit significant intra-GPU variability. Their short-running kernels interact poorly with the coarse granularity of dynamic power management mechanisms, leading to unpredictable performance. A key source of this intra-GPU transient variability is power excursions, also referred to as electrical design power peaks (EDP$_{p}$), where power temporarily exceeds the GPU's rated power limits. While power management mechanisms are designed to ensure reliable operation despite the presence of EDP$_{p}$ sustained power peaks cause throttling that detrimentally impacts overall application performance. To the best of our knowledge, our work is the first to characterize the magnitude and frequency of EDP$_{p}$, showing that frequent, significant power overshoots are especially common in ML workloads, which combine high power draw, short kernels, and rapidly changing compute demands. To address this, we design and leverage hardware and firmware mechanisms to provide consistent power management behavior that reduces intra-GPU variability due to EDP$_{p}$ and improves overall performance. We evaluate our techniques across a range of popular workloads. Our approach improves performance by 3-12% over state-of-the-art GPU power management and comes within 3% of an oracular solution.

cs.DC↗

Implications of Integrated CPU-GPU Processors on Thermal and Power Management Techniques

Heterogeneous processors with architecturally different cores (CPU and GPU) integrated on the same die lead to new challenges and opportunities for thermal and power management techniques because of shared thermal/power budgets between these cores. In this paper, we show that new parallel programming paradigms (e.g., OpenCL) for CPU-GPU processors create a tighter coupling between the workload, the thermal/power management unit and the operating system. Using detailed thermal and power maps of the die from infrared imaging, we demonstrate that in contrast to traditional multi-core CPUs, heterogeneous processors exhibit higher coupled behavior for dynamic voltage and frequency scaling and workload scheduling, in terms of their effect on performance, power, and temperature. Further, we show that by taking the differences in core architectures and relative proximity of different computing cores on the die into consideration, better scheduling schemes could be implemented to reduce both the power density and peak temperature of the die. The findings presented in the paper can be used to improve thermal and power efficiency of heterogeneous CPU-GPU processors.

cs.AR↗