arXiv subjects

Ayush Choure

Publications and source records attributed to Ayush Choure.

9 recordsLinked to original sources

arXiv2026

A Resource-centric Analysis and Optimization of NoSQL Workloads using Distressed Resource Volume Metric

Large-scale managed cloud databases leverage sophisticated load Packing and Migration (PAM) algorithms, which provide the efficiencies necessary for running these services at scale on cloud resources. Research into optimizing the resources and reliability of cloud databases at massive scales is limited by a lack of public NoSQL workloads. We address this in the context of Cosmos DB, Microsoft's flagship cloud-hosted NoSQL database. We first propose open-source NoSQL workloads from real Cosmos DB clusters, and analyze these traces to derive a novel reliability metric, Distressed Resource Volume (DRV), which captures the quality of service experienced by the end user. We then develop an open-source policy simulation framework, LoadStar, powered by a non-parametric statistical model of estimating the QoS of real traffic patterns. These form a reusable benchmark pipeline for validating policies for resource-centric NoSQL workloads. We then define a resource optimization problem for placing Cosmos DB replicas onto VM nodes, develop the Luna model for forecasting future load distributions, and the Orbit PAM algorithm that uses these forecasts to trigger and rebalance stressed replicas, to reduce tail-errors. Our experiments, validated using LoadStar for these workloads, demonstrate Orbit's benefits over the existing Cosmos DB policy and a worst-fit optimized baseline, with higher load delivered at lower error rates and up to $35\%$ reduction in resources. These have been deployed in production, with potential savings of $\$100M$s/yr while improving service reliability for millions of customers.

cs.DC↗

arXiv2026

Attention Enhanced Entity Recommendation for Intelligent Monitoring in Cloud Systems

In this paper, we present DiRecGNN, an attention-enhanced entity recommendation framework for monitoring cloud services at Microsoft. We provide insights on the usefulness of this feature as perceived by the cloud service owners and lessons learned from deployment. Specifically, we introduce the problem of recommending the optimal subset of attributes (dimensions) that should be tracked by an automated watchdog (monitor) for cloud services. To begin, we construct the monitor heterogeneous graph at production-scale. The interaction dynamics of these entities are often characterized by limited structural and engagement information, resulting in inferior performance of state-of-the-art approaches. Moreover, traditional methods fail to capture the dependencies between entities spanning a long range due to their homophilic nature. Therefore, we propose an attention-enhanced entity ranking model inspired by transformer architectures. Our model utilizes a multi-head attention mechanism to focus on heterogeneous neighbors and their attributes, and further attends to paths sampled using random walks to capture long-range dependencies. We also employ multi-faceted loss functions to optimize for relevant recommendations while respecting the inherent sparsity of the data. Empirical evaluations demonstrate significant improvements over existing methods, with our model achieving a 43.1% increase in MRR. Furthermore, product teams who consumed these features perceive the feature as useful and rated it 4.5 out of 5.

cs.LG↗

arXiv2026

A Holistic Framework for Automated Configuration Recommendation for Cloud Service Monitoring

Reliability of large-scale cloud services is critical for user satisfaction and business continuity. Despite significant investments in reliability engineering, production incidents remain inevitable, often leading to customer impact and operational overhead. In large cloud companies, multiple services are deployed across regions necessitating robust health monitoring systems. However, the current monitor configuration process is manual, largely reactive and ad hoc, resulting in gaps in coverage and redundant alerts. In this paper, we present a comprehensive study of monitor creation in Microsoft, identifying key components in the existing process. We further design a modular recommendation framework that processes the graph structured service entities to suggest optimal monitor configurations. Through extensive experimentation on historical data and user study of recommendations for production services at Microsoft, we demonstrate the efficacy of our approach in providing relevant recommendations for monitor configurations.

cs.DC↗

arXiv2024

Intelligent Monitoring Framework for Cloud Services: A Data-Driven Approach

Cloud service owners need to continuously monitor their services to ensure high availability and reliability. Gaps in monitoring can lead to delay in incident detection and significant negative customer impact. Current process of monitor creation is ad-hoc and reactive in nature. Developers create monitors using their tribal knowledge and, primarily, a trial and error based process. As a result, monitors often have incomplete coverage which leads to production issues, or, redundancy which results in noise and wasted effort. In this work, we address this issue by proposing an intelligent monitoring framework that recommends monitors for cloud services based on their service properties. We start by mining the attributes of 30,000+ monitors from 791 production services at Microsoft and derive a structured ontology for monitors. We focus on two crucial dimensions: what to monitor (resources) and which metrics to monitor. We conduct an extensive empirical study and derive key insights on the major classes of monitors employed by cloud services at Microsoft, their associated dimensions, and the interrelationship between service properties and this ontology. Using these insights, we propose a deep learning based framework that recommends monitors based on the service properties. Finally, we conduct a user study with engineers from Microsoft which demonstrates the usefulness of the proposed framework. The proposed framework along with the ontology driven projections, succeeded in creating production quality recommendations for majority of resource classes. This was also validated by the users from the study who rated the framework's usefulness as 4.27 out of 5.

cs.NI↗

arXiv2020

Rich-Item Recommendations for Rich-Users: Exploiting Dynamic and Static Side Information

In this paper, we study the problem of recommendation system where the users and items to be recommended are rich data structures with multiple entity types and with multiple sources of side-information in the form of graphs. We provide a general formulation for the problem that captures the complexities of modern real-world recommendations and generalizes many existing formulations. In our formulation, each user/document that requires a recommendation and each item or tag that is to be recommended, both are modeled by a set of static entities and a dynamic component. The relationships between entities are captured by several weighted bipartite graphs. To effectively exploit these complex interactions and learn the recommendation model, we propose MEDRES- a multiple graph-CNN based novel deep-learning architecture. MEDRES uses AL-GCN, a novel graph convolution network block, that harnesses strong representative features from the underlying graphs. Moreover, in order to capture highly heterogeneous engagement of different users with the system and constraints on the number of items to be recommended, we propose a novel ranking metric pAp@k along with a method to optimize the metric directly. We demonstrate effectiveness of our method on two benchmarks: a) citation data, b) Flickr data. In addition, we present two real-world case studies of our formulation and the MEDRES architecture. We show how our technique can be used to naturally model the message recommendation problem and the teams recommendation problem in the Microsoft Teams (MSTeams) product and demonstrate that it is 5-6% points more accurate than the production-grade models.

cs.LG↗

arXiv2013

Improved bounds on the sandpile diffusions on Grid graphs

The Abelian Sandpile Model is a discrete diffusion process defined on graphs (Dhar [10], Dhar et al. [11]) which serves as the standard model of self-organized criticality. The transience class of a sandpile is defined as the maximum number of particles that can be added without making the system recurrent ([3]). Using elementary combinatorial arguments and symmetry properties, Babai and Gorodezky (SODA 2007,[2]) demonstrated a bound of O(n^30) on the transience class of an nxn grid. This was later improved by Choure and Vishwanathan (SODA 2012,[7]) to O(n^7) using techniques based on harmonic functions on graphs. We improve this bound to O(n^7 log n). We also demonstrate tight bounds on certain resistance ratios over grid networks. The tools used for deriving these bounds may be of independent interest.

math-ph↗

arXiv2012

On graph parameters guaranteeing fast Sandpile diffusion

The Abelian Sandpile Model is a discrete diffusion process defined on graphs (Dhar \cite{DD90}, Dhar et al. \cite{DD95}) which serves as the standard model of self-organized criticality. The transience class of a sandpile is defined as the maximum number of particles that can be added without making the system recurrent (\cite{BT05}). We demonstrate a class of sandpile which have polynomially bound transience classes by identifying key graph properties that play a role in the rapid diffusion process. These are the volume growth parameters, boundary regularity type properties and non-empty interior type constraints. This generalizes a previous result by Babai and Gorodezky (SODA 2007,\cite{LB07}), in which they establish polynomial bounds on $n \times n$ grid. Indeed the properties we show are based on ideas extracted from their proof as well as the continuous analogs in complex analysis. We conclude with a discussion on the notion of degeneracy and dimensions in graphs.

cs.DM↗

arXiv2012

Random Walks, Electric Networks and The Transience Class problem of Sandpiles

The Abelian Sandpile Model is a discrete diffusion process defined on graphs (Dhar \cite{DD90}, Dhar et al. \cite{DD95}) which serves as the standard model of \textit{self-organized criticality}. The transience class of a sandpile is defined as the maximum number of particles that can be added without making the system recurrent (\cite{BT05}). We develop the theory of discrete diffusions in contrast to continuous harmonic functions on graphs and establish deep connections between standard results in the study of random walks on graphs and sandpiles on graphs. Using this connection and building other necessary machinery we improve the main result of Babai and Gorodezky (SODA 2007,\cite{LB07}) of the bound on the transience class of an $n \times n$ grid, from $O(n^{30})$ to $O(n^{7})$. Proving that the transience class is small validates the general notion that for most natural phenomenon, the time during which the system is transient is small. In addition, we use the machinery developed to prove a number of auxiliary results. We exhibit an equivalence between two other tessellations of plane, the honeycomb and triangular lattices. We give general upper bounds on the transience class as a function of the number of edges to the sink. Further, for planar sandpiles we derive an explicit algebraic expression which provably approximates the transience class of $G$ to within $O(|E(G)|)$. This expression is based on the spectrum of the Laplacian of the dual of the graph $G$. We also show a lower bound of $Ω(n^{3})$ on the transience class on the grid improving the obvious bound of $Ω(n^{2})$.

cs.DM↗

arXiv2012

Structural properties of Stochastic Abelian Sandpile

We present some combinatorial results on the stochastic abelian sandpile model. These models are characterized by nondeterministic toppling rules. The recurrence checking for the deterministic case can be performed using the well known burning test which detects presence of forbidden sub-configurations (FSC) in strongly polynomial time. In the stochastic case, however, even for Manna's model, which is perhaps the simplest non-trivial example, no such procedure is known. In this paper, we address the decision problem of the existence of any FSC in a general stochastic sandpile. We demonstrate a polynomial time algorithm which, given the sandpile graph and toppling rules, decides if there exists an FSC. In the event of a positive answer, it generates at least one FSC for the given sandpile. Repeated application of the algorithm can be used to find many distinct FSCs. We also demonstrate a procedure for creating larger FSCs from smaller ones and use this to create FSCs for the Manna's model. We hope that the structural analysis of stochastic sandpile we perform in this paper, will prove useful in the eventual formulation of a deterministic procedure to decide recurrence.

math.CO↗