arXiv subjects

Haiyu Huang

Publications and source records attributed to Haiyu Huang.

9 recordsLinked to original sources

arXiv2026

From General Agents to RCA Experts: A Self-Evolving Harness for Root Cause Analysis

Automated root cause analysis (RCA) with large language models (LLMs) has drawn growing attention. Today, SREs typically automate RCA with LLMs in one of two ways: directly using a general-purpose agent (e.g., Codex or Claude Code) for diagnosis, or building a specialized RCA agent from scratch. As mainstream general agents grow more capable and iterate quickly, our quantitative study finds that the former now often surpasses the latter. Its accuracy, however, still falls short of production needs, and this gap stems mainly from the external adaptation layer outside the agent's general capabilities, namely the harness. We therefore argue that LLM-based RCA should focus on this external harness, reusing the strong general capabilities of a modern agent rather than rebuilding an agent from scratch. A key capability of such a harness is to self-evolve, accumulating system-specific experience from past diagnoses so that it gets better the more it is used. We introduce OpsHarness, a self-evolving RCA harness that turns diagnosis experience into reusable expertise. Its data plane combines layered operational knowledge with an idea-card tool library, while its control plane coordinates setup, diagnosis, evolution, and verification. During evolution, OpsHarness contrasts successful and failed trajectories, converts their evidence into atomic proposals, and admits updates only through a dual-gate verification process designed to prevent overfitting and regression. Across two public benchmarks and an industrial deployment, OpsHarness achieves 59.0\% top-1 accuracy, improving over a bare general agent by 63.4\% and over baseline RCA agents by 4.02$\times$.

cs.SE↗

arXiv2026

Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systems

LLM agents are increasingly explored for automating root cause analysis (RCA) in cloud-native systems, creating a need to evaluate both diagnostic correctness and the quality of the supporting investigation. Existing static benchmarks offer repeatable inputs but limited system-facing interaction, while live testbeds expose realistic tools but hinder controlled comparison because incident evidence varies across runs; both paradigms focus primarily on final answers. To address these limitations, we present Cloud-OpsBench, an evaluation infrastructure for interactive and evidence-grounded cloud RCA. It comprises 754 runtime-verified cases across 57 fault types on two microservice workloads spanning application services and Kubernetes platform layers. Each fault is captured as a state snapshot and replayed through standard diagnostic interfaces, with outcome labels and diagnostic evidence graphs that enable matched comparisons and process-level analysis. Across ten LLM agents, the strongest Joint RCA Accuracy (JRA) reaches 0.76 on OnlineBoutique and 0.68 on TrainTicket, while the corresponding Evidence Closure Rates (ECR) are only 0.38 and 0.15. This outcome--process gap shows that final-answer correctness alone substantially overestimates agents' ability to perform evidence-grounded diagnosis.

cs.SE↗

arXiv2025

LogPilot: Intent-aware and Scalable Alert Diagnosis for Large-scale Online Service Systems

Effective alert diagnosis is essential for ensuring the reliability of large-scale online service systems. However, on-call engineers are often burdened with manually inspecting massive volumes of logs to identify root causes. While various automated tools have been proposed, they struggle in practice due to alert-agnostic log scoping and the inability to organize complex data effectively for reasoning. To overcome these limitations, we introduce LogPilot, an intent-aware and scalable framework powered by Large Language Models (LLMs) for automated log-based alert diagnosis. LogPilot introduces an intent-aware approach, interpreting the logic in alert definitions (e.g., PromQL) to precisely identify causally related logs and requests. To achieve scalability, it reconstructs each request's execution into a spatiotemporal log chain, clusters similar chains to identify recurring execution patterns, and provides representative samples to the LLMs for diagnosis. This clustering-based approach ensures the input is both rich in diagnostic detail and compact enough to fit within the LLM's context window. Evaluated on real-world alerts from Volcano Engine Cloud, LogPilot improves the usefulness of root cause summarization by 50.34% and exact localization accuracy by 54.79% over state-of-the-art methods. With a diagnosis time under one minute and a cost of only $0.074 per alert, LogPilot has been successfully deployed in production, offering an automated and practical solution for service alert diagnosis.

cs.SE↗

arXiv2025

InferLog: Accelerating LLM Inference for Online Log Parsing via ICL-oriented Prefix Caching

Modern software systems generate massive volumes of runtime logs, necessitating efficient and accurate log parsing to enable critical downstream tasks such as anomaly detection and root cause analysis. Recently, large language models (LLMs) have achieved advanced accuracy on log parsing, but their deployment in production environments faces two major limitations: (1) the privacy risks associated with commercial LLMs, driving the adoption of local deployment, and (2) the stringent latency and throughput requirements imposed by high-volume log streams, which existing LLM-based parsers fail to meet. Although recent efforts have reduced the number of LLM queries, they overlook the high latency of the LLM invocations, where concurrent log parsing requests can cause serve performance degradation of LLM inference system. In this study, we present InferLog, the first LLM inference optimization method for online log parsing. Our key insight is that the inference efficiency emerges as the vital bottleneck in LLM-based online log parsing, rather than parsing accuracy. InferLog accelerates inference by designing (1) A Prefix-aware ICL Refinement policy to refine the examples and permutation of in-context learning to improve the prefix caching efficiency. (2) A rapid and task-specific configuration tuning pipeline based on meta-learning to find the optimal LLM scheduling-related configuration for dynamic log parsing workloads. The experimental results based on Loghub dataset and vLLM demonstrate that InferLog significantly outperforms existing inference optimization methods and markedly accelerates the state-of-the-art LLM-based log parser without compromising parsing accuracy.

cs.SE↗

arXiv2025

Phase transition and critical behavior in hierarchical integer-valued Gaussian and Coulomb gas models

Given a square box $Λ_n\subseteq\mathbb Z^2$ of side length $L^n$ with $L,n>1$, we study hierarchical random fields $\{ϕ_x\colon x\inΛ_n\}$ with law proportional to ${\rm e}^{\frac12β(ϕ,Δ_nϕ)}\prod_{x\inΛ_n}ν({\rm d}ϕ_x)$, where $β>0$ is the inverse temperature, $Δ_n$ is a hierarchical Laplacian on $Λ_n$, and $ν$ is a non-degenerate $1$-periodic measure on $\mathbb R$. Our setting includes the integer-valued Gaussian field (a.k.a. DG model or Villain Coulomb gas) and the sine-Gordon model. Relying on renormalization group analysis we derive sharp asymptotic formulas, in the limit as $n\to\infty$, for the covariance $\langleϕ_xϕ_y\rangle$ and the fractional charge $\langle {\rm e}^{2π{\rm i}α(ϕ_x-ϕ_y)}\rangle$ in the subcritical $β<β_{\rm c}:=π^2/\log L$, critical $β=β_{\rm c}$ and slightly supercritical $β>β_{\rm c}$ regimes. The field exhibits logarithmic correlations throughout albeit with a distinct $β$-dependence of both the covariance scale and the fractional-charge exponents in the sub/supercritical regimes. Explicit logarithmic corrections appear at the critical point.

math.PR↗

arXiv2025

Orthogonal Calibration for Asynchronous Federated Learning

Asynchronous federated learning mitigates the inefficiency of conventional synchronous aggregation by integrating updates as they arrive and adjusting their influence based on staleness. Due to asynchrony and data heterogeneity, learning objectives at the global and local levels are inherently inconsistent -- global optimization trajectories may conflict with ongoing local updates. Existing asynchronous methods simply distribute the latest global weights to clients, which can overwrite local progress and cause model drift. In this paper, we propose OrthoFL, an orthogonal calibration framework that decouples global and local learning progress and adjusts global shifts to minimize interference before merging them into local models. In OrthoFL, clients and the server maintain separate model weights. Upon receiving an update, the server aggregates it into the global weights via a moving average. For client weights, the server computes the global weight shift accumulated during the client's delay and removes the components aligned with the direction of the received update. The resulting parameters lie in a subspace orthogonal to the client update and preserve the maximal information from the global progress. The calibrated global shift is then merged into the client weights for further training. Extensive experiments show that OrthoFL improves accuracy by 9.6% and achieves a 12$\times$ speedup compared to synchronous methods. Moreover, it consistently outperforms state-of-the-art asynchronous baselines under various delay patterns and heterogeneity scenarios.

cs.LG↗

arXiv2024

FaaSRCA: Full Lifecycle Root Cause Analysis for Serverless Applications

Serverless becomes popular as a novel computing paradigms for cloud native services. However, the complexity and dynamic nature of serverless applications present significant challenges to ensure system availability and performance. There are many root cause analysis (RCA) methods for microservice systems, but they are not suitable for precise modeling serverless applications. This is because: (1) Compared to microservice, serverless applications exhibit a highly dynamic nature. They have short lifecycle and only generate instantaneous pulse-like data, lacking long-term continuous information. (2) Existing methods solely focus on analyzing the running stage and overlook other stages, failing to encompass the entire lifecycle of serverless applications. To address these limitations, we propose FaaSRCA, a full lifecycle root cause analysis method for serverless applications. It integrates multi-modal observability data generated from platform and application side by using Global Call Graph. We train a Graph Attention Network (GAT) based graph auto-encoder to compute reconstruction scores for the nodes in global call graph. Based on the scores, we determine the root cause at the granularity of the lifecycle stage of serverless functions. We conduct experimental evaluations on two serverless benchmarks, the results show that FaaSRCA outperforms other baseline methods with a top-k precision improvement ranging from 21.25% to 81.63%.

cs.SE↗

arXiv2024

Mint: Cost-Efficient Tracing with All Requests Collection via Commonality and Variability Analysis

Distributed traces contain valuable information but are often massive in volume, posing a core challenge in tracing framework design: balancing the tradeoff between preserving essential trace information and reducing trace volume. To address this tradeoff, previous approaches typically used a '1 or 0' sampling strategy: retaining sampled traces while completely discarding unsampled ones. However, based on an empirical study on real-world production traces, we discover that the '1 or 0' strategy actually fails to effectively balance this tradeoff. To achieve a more balanced outcome, we shift the strategy from the '1 or 0' paradigm to the 'commonality + variability' paradigm. The core of 'commonality + variability' paradigm is to first parse traces into common patterns and variable parameters, then aggregate the patterns and filter the parameters. We propose a cost-efficient tracing framework, Mint, which implements the 'commonality + variability' paradigm on the agent side to enable all requests capturing. Our experiments show that Mint can capture all traces and retain more trace information while optimizing trace storage (reduced to an average of 2.7%) and network overhead (reduced to an average of 4.2%). Moreover, experiments also demonstrate that Mint is lightweight enough for production use.

cs.SE↗

arXiv2023

A limit law for the maximum of subcritical DG-model on a hierarchical lattice

We study the extremal properties of the "integer-valued Gaussian" a.k.a.\ DG-model on the hierarchical lattice $Λ_n:=\{1,\dots,b\}^n$ (with $b\ge2$) of depth $n$. This is a random field $φ\in\mathbb Z^{Λ_n}$ with law proportional to $e^{\frac12β(φ,Δ_nφ)}\prod_{x\inΛ_n}\#(dφ_x)$, where $Δ_n$ is the hierarchical Laplacian, $β$ is the inverse temperature and $\#$ is the counting measure on $\mathbb Z$. Denoting $β_c:=2π^2/\log b$ and $m_n:=β^{-1/2}[(2\log b)^{1/2}n-\frac32(2\log b)^{-1/2}\log n]$, for $0<β<β_c$ we prove that, along increasing sequences of $n$ such that the fractional part of $m_{n}$ converges to an $s\in[0,1)$, the centered maximum $\max_{x\inΛ_n}φ_x-\lfloor m_n\rfloor$ tends (as $n\to\infty$) in law to a discrete variant of a randomly shifted Gumbel law with the shift depending non-trivially on $s$. The convergence extends to the extremal process whose law tends to a decorated Poisson point process with a random intensity measure. The proofs rely on renormalization-group analysis which enables a tight coupling of the DG-model to a Gaussian Free Field. The interval $(0,β_c]$ marks the full range of values of $β$ for which the renormalization-group iterations tend to a "trivial" fixed point.

math.PR↗