arXiv subjects

Z. Jonny Kong

Publications and source records attributed to Z. Jonny Kong.

4 recordsLinked to original sources

arXiv2026

DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS

Prefill/decode disaggregation is increasingly adopted in LLM serving to improve the latency-throughput tradeoff and meet strict TTFT and TPOT SLOs. However, LLM inference remains energy-hungry: autoscaling alone is too coarse-grained to track fast workload fluctuations, and applying fine-grained DVFS under disaggregation is complicated by phase-asymmetric dynamics and coupling between provisioning and frequency control. We present DualScale, a two-tier energy optimization framework for disaggregated LLM serving. DualScale jointly optimizes placement and DVFS across prefill and decode using predictive latency and power models. At coarse timescales, DualScale computes phase-aware placement and baseline frequencies that minimize energy while satisfying SLO constraints. At fine timescales, DualScale dynamically adapts GPU frequency per iteration using stage-specific control: model predictive control (MPC) for prefill to account for queue evolution and future TTFT impact, and lightweight slack-aware adaptation for decode to exploit its smoother, memory-bound dynamics. This hierarchical design enables coordinated control across timescales while preserving strict serving SLOs. Evaluation on a 16x H100 cluster serving Llama 3.3 70B with production-style traces shows that DualScale meets TTFT/TPOT SLOs while reducing energy by up to 39% in prefill and 48% in decode relative to DistServe.

cs.DC↗

arXiv2025

Handover Configurations in Operational 5G Networks: Diversity, Evolution, and Impact on Performance

Mobility management in cellular networks, especially the handover (HO) process, plays a key role in providing seamless and ubiquitous Internet access. The wide-scale deployment of 5G and the resulting co-existence of 4G/5G in the past six years have significantly changed the landscape of all mobile network operators and made the HO process much more complex than before. While several recent works have studied the impact of HOs on user experience, why and how HOs occur and how HO configurations affect performance in 5G operational networks remains largely unknown. Through four cross-country driving trips across the US spread out over a 27-month period, we conduct an in-depth measurement study of HO configurations across all three major US operators. Our study reveals (a) new types of HOs and new HO events used by operators to handle these new types of HOs, (b) overly aggressive HO configurations that result in unnecessarily high signaling overhead, (c) large diversity in HO configuration parameter values, which also differ across operators, but significantly lower diversity in 5G compared to LTE, and (d) sub-optimal HO configurations/decisions leading to poor pre- or post-HO performance. Our findings have many implications for mobile operators, as they keep fine-tuning their 5G HO configurations.

cs.NI↗

arXiv2025

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism

With the rapid innovation of GPUs, heterogeneous GPU clusters in both public clouds and on-premise data centers have become increasingly commonplace. In this paper, we demonstrate how pipeline parallelism, a technique wellstudied for throughput-oriented deep learning model training, can be used effectively for serving latency-bound model inference, e.g., in video analytics systems, on heterogeneous GPU clusters. Our work exploits the synergy between diversity in model layers and diversity in GPU architectures, which results in comparable inference latency for many layers when running on low-class and high-class GPUs. We explore how such overlooked capability of low-class GPUs can be exploited using pipeline parallelism and present a novel inference serving system, PPipe, that employs pool-based pipeline parallelism via an MILP-based control plane and a data plane that performs resource reservation-based adaptive batching. Evaluation results on diverse workloads (18 CNN models) show that PPipe achieves 41.1% - 65.5% higher utilization of low-class GPUs while maintaining high utilization of high-class GPUs, leading to 32.2% - 75.1% higher serving throughput compared to various baselines.

cs.DC↗

arXiv2024

High-Fidelity Cellular Network Control-Plane Traffic Generation without Domain Knowledge

With rapid evolution of mobile core network (MCN) architectures, large-scale control-plane traffic (CPT) traces are critical to studying MCN design and performance optimization by the R&D community. The prior-art control-plane traffic generator SMM heavily relies on domain knowledge which requires re-design as the domain evolves. In this work, we study the feasibility of developing a high-fidelity MCN control plane traffic generator by leveraging generative ML models. We identify key challenges in synthesizing high-fidelity CPT including generic (to data-plane) requirements such as multimodality feature relationships and unique requirements such as stateful semantics and long-term (time-of-day) data variations. We show state-of-the-art, generative adversarial network (GAN)-based approaches shown to work well for data-plane traffic cannot meet these fidelity requirements of CPT, and develop a transformer-based model, CPT-GPT, that accurately captures complex dependencies among the samples in each traffic stream (control events by the same UE) without the need for GAN. Our evaluation of CPT-GPT on a large-scale control-plane traffic trace shows that (1) it does not rely on domain knowledge yet synthesizes control-plane traffic with comparable fidelity as SMM; (2) compared to the prior-art GAN-based approach, it reduces the fraction of streams that violate stateful semantics by two orders of magnitude, the max y-distance of sojourn time distributions of streams by 16.0%, and the transfer learning time in deriving new hourly models by 3.36x.

cs.NI↗