arXiv subjects

Kaustabha Ray

Publications and source records attributed to Kaustabha Ray.

4 recordsLinked to original sources

arXiv2025

Unsupervised Cycle Detection in Agentic Applications

Agentic applications powered by Large Language Models exhibit non-deterministic behaviors that can form hidden execution cycles, silently consuming resources without triggering explicit errors. Traditional observability platforms fail to detect these costly inefficiencies. We present an unsupervised cycle detection framework that combines structural and semantic analysis. Our approach first applies computationally efficient temporal call stack analysis to identify explicit loops and then leverages semantic similarity analysis to uncover subtle cycles characterized by redundant content generation. Evaluated on 1575 trajectories from a LangGraph-based stock market application, our hybrid approach achieves an F1 score of 0.72 (precision: 0.62, recall: 0.86), significantly outperforming individual structural (F1: 0.08) and semantic methods (F1: 0.28). While these results are encouraging, there remains substantial scope for improvement, and future work is needed to refine the approach and address its current limitations.

cs.CL↗

arXiv2025

Statistical Modeling and Uncertainty Estimation of LLM Inference Systems

Large Language Model (LLM) inference systems present significant challenges in statistical performance characterization due to dynamic workload variations, diverse hardware architectures, and complex interactions between model size, batch processing, and throughput requirements. Accurate statistical characterization enables better workload scheduling, adaptive resource provisioning, and cost-aware inference optimization, making it crucial for improving efficiency in large-scale AI deployments. Traditional analytical models provide explainability but cannot cover the vast diversity of real-world workloads, making it impossible to benchmark every scenario in advance. Machine learning (ML) approaches effectively predict performance for non-benchmarked cases but struggle when extrapolating beyond their observed training space. To address these limitations for LLM inference systems, we propose an Analytical with Learning Augmentation (ALA) framework that bridges analytical modeling with \ml for robust statistical prediction and uncertainty estimation in LLM inference workloads. Our method employs an analytical throughput model with parameters estimated for benchmarked workloads, then extends to unobserved configurations using \ml predictions. We enhance this with simulated annealing to exploit subsets of the workload data point combinations and develop an error predictor. Finally, we quantify uncertainty based on vector space similarity between new and observed workloads to ensure robust generalization. Through extensive experimentation on diverse LLM inference workloads, we demonstrate that our framework achieves low median errors while maintaining adaptability to new inference scenarios.

cs.PF↗

arXiv2025

On the Partitioning of GPU Power among Multi-Instances

Efficient power management in cloud data centers is essential for reducing costs, enhancing performance, and minimizing environmental impact. GPUs, critical for tasks like machine learning (ML) and GenAI, are major contributors to power consumption. NVIDIA's Multi-Instance GPU (MIG) technology improves GPU utilization by enabling isolated partitions with per-partition resource tracking, facilitating GPU sharing by multiple tenants. However, accurately apportioning GPU power consumption among MIG instances remains challenging due to a lack of hardware support. This paper addresses this challenge by developing software methods to estimate power usage per MIG partition. We analyze NVIDIA GPU utilization metrics and find that light-weight methods with good accuracy can be difficult to construct. We hence explore the use of ML-based power models to enable accurate, partition-level power estimation. Our findings reveal that a single generic offline power model or modeling method is not applicable across diverse workloads, especially with concurrent MIG usage, and that online models constructed using partition-level utilization metrics of workloads under execution can significantly improve accuracy. Using NVIDIA A100 GPUs, we demonstrate this approach for accurate partition-level power estimation for workloads including matrix multiplication and Large Language Model inference, contributing to transparent and fair carbon reporting.

cs.DC↗

arXiv2025

Metric Criticality Identification for Cloud Microservices

Modern cloud-native applications built on microservice architectures present unprecedented challenges for system monitoring and alerting. Site Reliability Engineers (SREs) face the daunting challenge of defining effective monitoring strategies across multitude of metrics to ensure system reliability, a task that traditionally requires extensive manual expertise. The distributed nature of microservices, characterized by stochastic execution patterns and intricate inter-service dependencies, renders the traditional manual approach of navigating the vast metrics landscape computationally and operationally prohibitive. To address this critical challenge, we propose KIMetrix, a data-driven system that automatically identifies minimal yet comprehensive metric subsets to aid SREs in monitoring microservice applications. KIMetrix leverages information-theoretic measures, specifically entropy and mutual information, to quantify metric criticality while considering the stochastic execution patterns inherent in microservice topologies. Our approach operates solely on lightweight metrics and traces, eliminating the need for expensive processing of unstructured logs, and requires no expert-defined training data. Experimental evaluation on state-of-the-art real-world microservice benchmark datasets demonstrates KIMetrix's effectiveness in identifying critical metric subsets that provide comprehensive system coverage while significantly reducing the burden on SREs. By automating the identification of essential metrics for alerting, KIMetrix enables more reliable system monitoring without overwhelming operators with false positives or missing critical system events.

cs.DC↗