Xinlei Ying | Searcharxiv

arXiv2026

Is My RPC Response Reliable? Detecting RPC Bugs in Ethereum Blockchain Client under Context

Blockchain clients are fundamental software for running blockchain nodes. They provide users with various RPC (Remote Procedure Call) interfaces to interact with the blockchain. These RPC methods are expected to follow the same specification across different blockchain nodes, providing users with seamless interaction. However, there have been continuous reports on various RPC bugs that can cause unexpected responses or even Denial of Service weakness. Existing studies on blockchain RPC bug detection mainly focus on generating the RPC method calls for testing blockchain clients. However, a wide range of the reported RPC bugs are triggered in various blockchain contexts. To the best of our knowledge, little attention is paid to generating proper contexts that can trigger these context-dependent RPC bugs. In this work, we propose EthCRAFT, a Context-aware RPC Analysis and Fuzzing Tool for client RPC bug detection. EthCRAFT first proposes to explore the state transition program space of blockchain clients and generate various transactions to construct the context. EthCRAFT then designs a context-aware RPC method call generation method to send RPC calls to the blockchain clients. The responses of 5 different client implementations are used as cross-referring oracles to detect the RPC bugs. We evaluate EthCRAFT on real-world RPC bugs collected from the GitHub issues of Ethereum client implementations. Experiment results show that EthCRAFT outperforms existing client RPC detectors by detecting more RPC bugs. Moreover, EthCRAFT has found six new bugs in major Ethereum clients and reported them to the developers. One of the bug fixes has been written into breaking changes in the client's updates. Three of our bug reports have been offered a vulnerability bounty by the Ethereum Foundation.

cs.SE↗

arXiv2025

ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal

Large Language Models (LLMs) increasingly exhibit over-refusal - erroneously rejecting benign queries due to overly conservative safety measures - a critical functional flaw that undermines their reliability and usability. Current methods for testing this behavior are demonstrably inadequate, suffering from flawed benchmarks and limited test generation capabilities, as highlighted by our empirical user study. To the best of our knowledge, this paper introduces the first evolutionary testing framework, ORFuzz, for the systematic detection and analysis of LLM over-refusals. ORFuzz uniquely integrates three core components: (1) safety category-aware seed selection for comprehensive test coverage, (2) adaptive mutator optimization using reasoning LLMs to generate effective test cases, and (3) OR-Judge, a human-aligned judge model validated to accurately reflect user perception of toxicity and refusal. Our extensive evaluations demonstrate that ORFuzz generates diverse, validated over-refusal instances at a rate (6.98% average) more than double that of leading baselines, effectively uncovering vulnerabilities. Furthermore, ORFuzz's outputs form the basis of ORFuzzSet, a new benchmark of 1,855 highly transferable test cases that achieves a superior 63.56% average over-refusal rate across 10 diverse LLMs, significantly outperforming existing datasets. ORFuzz and ORFuzzSet provide a robust automated testing framework and a valuable community resource, paving the way for developing more reliable and trustworthy LLM-based software systems.

cs.SE↗