arXiv subjects

Zihao Rao

Publications and source records attributed to Zihao Rao.

4 recordsLinked to original sources

arXiv2026

SkillAudit: From Fixed-Suite Benchmarking to Skill-Centered Assessment

Agent skills have become a practical way to extend large language model agents, but the growing skill ecosystem still lacks a reliable way to judge whether a skill is worth deploying. Existing evaluation methods remain largely anchored to fixed task suites, assessing skills through performance on predefined tasks and environments. As skill marketplaces expand, this paradigm becomes inadequate: fixed suites can conflate a skill's marginal contribution with backbone strength and miss its value when tasks fall outside the skill's intended scope. We introduce SkillAudit, an end-to-end framework for skill-centered assessment that takes an arbitrary agent skill as input and automatically generates a comprehensive, multi-dimensional evaluation report spanning utility, efficiency/cost, and safety. SkillAudit focuses on the skill artifact itself and constructs capability-aligned evaluation tasks directly from the skill package. The generated tasks are conducted in isolated sandbox environments to collect execution evidence, followed by automated checks with LLM-based judging to produce auditable results. To dissect the agent skills, we propose the baseline comparison principle to measure utility and efficiency/cost, and introduce a two-stage detection paradigm combining static semantic analysis with dynamic runtime verification to assess safety risks. After scanning top-ranked real-world skill packages spanning 23 occupational categories, we found that over 7% of skills are at risky status.

cs.AI↗

arXiv2025

Annotating and Auditing the Safety Properties of Unsafe Rust

In Rust, unsafe code is the sole source of potential undefined behaviors. To avoid misuse, Rust developers should clarify the safety properties for each unsafe API. However, the community currently lacks a key standard for safety documentation: existing safety comments in the source code and safety documentation can be ad hoc and incomplete. This paper presents a tag-centric methodology for auditing the consistency and completeness of safety documentation. We first derive a taxonomy of Safety Tags to formalize natural-language requirements. Second, because API soundness frequently relies on struct invariants, we propose a set of empirical rules to systematically audit the structural consistency of safety documentation. We implemented this methodology in safety-tool, a static linter that automatically enforces structural consistency between local safety annotations and callee requirements. Our approach was applied to the Rust standard library, fixing documentation issues on 27 APIs with 61 safety tags and identifying safety tags that are applicable to 96.1% of the public unsafe APIs in libstd. Furthermore, we have formalized the tagging idea through a Rust RFC to the wider community. We believe that the approach establishes a standardized practice of safety documentation and helps significantly reduce safety perils.

cs.PL↗

arXiv2024

Characterizing Unsafe Code Encapsulation In Real-world Rust Systems

Interior unsafe is an essential design paradigm advocated by the Rust community in system software development. However, there is little official guidance or few best practices regarding how to encapsulate unsafe code and achieve interior unsafe. The problem is critical because the Rust compiler is incapable of verifying the soundness of a safe function containing unsafe code. Falsely declaring an interior unsafe function as safe may undermine the fundamental memory-safety guarantee of Rust. To address this issue, this paper studies how interior unsafe is achieved in practice, aiming to identify best practices to guide Rust code design concerning unsafe code encapsulation. Specifically, we propose a novel unsafety isolation graph to model the essential usage and encapsulation of unsafe code. Based on the graph, we further propose four major isolation types and nine structural patterns to split a graph into several small self-contained subgraphs. These subgraphs can serve as useful audit units for examining the soundness of unsafe code encapsulation. We applied our approach to four real-world Rust projects. The experimental results demonstrate that our method is effective in characterizing their encapsulation code. Additionally, we identified two common issues in these projects that could complicate soundness verification or incur unsoundness issues.

cs.SE↗

arXiv2023

Revealing Performance Issues in Server-side WebAssembly Runtimes via Differential Testing

WebAssembly (Wasm) is a bytecode format originally serving as a compilation target for Web applications. It has recently been used increasingly on the server side, e.g., providing a safer, faster, and more portable alternative to Linux containers. With the popularity of server-side Wasm applications, it is essential to study performance issues (i.e., abnormal latency) in Wasm runtimes, as they may cause a significant impact on server-side applications. However, there is still a lack of attention to performance issues in server-side Wasm runtimes. In this paper, we design a novel differential testing approach WarpDiff to identify performance issues in server-side Wasm runtimes. The key insight is that in normal cases, the execution time of the same test case on different Wasm runtimes should follow an oracle ratio. We identify abnormal cases where the execution time ratio significantly deviates from the oracle ratio and subsequently locate the Wasm runtimes that cause the performance issues. We apply WarpDiff to test five popular server-side Wasm runtimes using 123 test cases from the LLVM test suite and demonstrate the top 10 abnormal cases we identified. We further conduct an in-depth analysis of these abnormal cases and summarize seven performance issues, all of which have been confirmed by the developers. We hope our work can inspire future investigation on improving Wasm runtime implementation and thus promoting the development of server-side Wasm applications.

cs.SE↗