arXiv subjects

Jiangrui Zheng

Publications and source records attributed to Jiangrui Zheng.

4 recordsLinked to original sources

arXiv2026

SPFinder: Improving the Context Length and Scalability for Tracing Known Vulnerability Patches

An upstream task for vulnerability management is the accurate localization of the patch that fixes a vulnerability. Existing works have proposed several approaches to trace or retrieve the patching commit for fixing a CVE. However, they suffer from two major challenges: (1) they cannot effectively handle the long diff code in patch commits, which is common when commit messages are non-informative; and (2) they do not scale to the full repository with satisfactory accuracy in realistic settings. We propose SPFinder, a scalable and effective retrieval framework for tracing known vulnerability patches. To address the long-context challenge, SPFinder introduces a hierarchical embedding technique that efficiently extends context coverage while mitigating long-context degradation, enabling effective modeling of all files in the commit. To address the scalability challenge, SPFinder adopts a three-phase retrieval framework that balances effectiveness and efficiency, achieving high recall at the full-repository level. Our evaluation on two datasets shows that SPFinder outperforms state-of-the-art patch tracing methods, including PatchFinder, PatchScout, and VFCFinder, by a large margin, and surpasses VoyageAI, a leading commercial code embedding model, on MRR and Recall@10 by 18% and 28%, respectively. Using SPFinder, we successfully traced and merged patch links for 35 CVEs in the GitHub Advisory Database, demonstrating its practical applicability. An ablation study further confirms that hierarchical embedding is a practically effective solution for handling long context in patch retrieval. Our artifacts and online demo are publicly available at https://github.com/AnonySE26/SPFinder and http://spfinder.org/.

cs.CR↗

arXiv2025

From Reviewers' Lens: Understanding Bug Bounty Report Invalid Reasons with LLMs

Bug bounty platforms (e.g., HackerOne, BugCrowd) leverage crowd-sourced vulnerability discovery to improve continuous coverage, reduce the cost of discovery, and serve as an integral complement to internal red teams. With the rise of AI-generated bug reports, little work exists to help bug hunters understand why these reports are labeled as invalid. To improve report quality and reduce reviewers' burden, it is critical to predict invalid reports and interpret invalid reasons. In this work, we conduct an empirical study with the purpose of helping bug hunters understand the validity of reports. We collect a dataset of 9,942 disclosed bug bounty reports, including 1,400 invalid reports, and evaluate whether state-of-the-art large language models can identify invalid reports. While models such as GPT-5, DeepSeek, and a fine-tuned RoBERTa achieve strong overall accuracy, they consistently struggle to detect invalid cases, showing a tendency to over-accept reports. To improve invalidity detection, we build a taxonomy of rejection reasons for Information Disclosure vulnerabilities and incorporate it into a retrieval-augmented generation (RAG) framework. This approach substantially improves classification consistency and reduces bias. We also examine whether reviewer decisions may be influenced by factors beyond the content of the report. Our analysis shows that reporters with higher reputations tend to receive more favorable outcomes in borderline cases, suggesting that perceived expertise can influence review judgments. Overall, our findings highlight the challenges of invalid report identification and show that combining LLMs with structured reviewer knowledge can support more transparent and consistent vulnerability report review.

cs.SE↗

arXiv2024

Can Highlighting Help GitHub Maintainers Track Security Fixes?

In recent years, the rapid growth of security vulnerabilities poses great challenges to tracing and managing them. For example, it was reported that the NVD database experienced significant delays due to the shortage of maintainers. Such delay creates challenges for third-party security personnel (e.g., administrators) to trace the information related to the CVE. To help security personnel trace a vulnerability patch, we build a retrieval system that automatically retrieves the patch in the repository. Inspired by existing work on explainable machine learning, we ask the following research question: can explanations help security maintainers make decisions in patch tracing? First, we investigate using LIME (a widely used explainable machine learning method) to highlight the rationale tokens in the commit message and code. In addition, we propose an explanation method called TfIdf-Highlight, which leverages the Tf-Idf statistics to select the most informative words in the repository and the dataset. We evaluate the effectiveness of highlighting using two experiments. First, we compare LIME and TfIdf-Highlight using a faithfulness score (i.e., sufficiency and comprehensiveness) defined for ranking. We find that TfIdf-Highlight significantly outperforms LIME's sufficiency scores by 15\% and slightly outperforms the comprehensiveness scores. Second, we conduct a blind human labeling experiment by asking the annotators to guess the patch under 3 settings (TfIdf-Highlight, LIME, and no highlight). We find that the helpfulness score for TfIdf-Highlight is higher than LIME while the labeling accuracies of LIME and TfIdf-Highlight are similar. Nevertheless, highlighting does not improve the accuracy over non-highlighting.

cs.CR↗

arXiv2024

HateModerate: Testing Hate Speech Detectors against Content Moderation Policies

To protect users from massive hateful content, existing works studied automated hate speech detection. Despite the existing efforts, one question remains: do automated hate speech detectors conform to social media content policies? A platform's content policies are a checklist of content moderated by the social media platform. Because content moderation rules are often uniquely defined, existing hate speech datasets cannot directly answer this question. This work seeks to answer this question by creating HateModerate, a dataset for testing the behaviors of automated content moderators against content policies. First, we engage 28 annotators and GPT in a six-step annotation process, resulting in a list of hateful and non-hateful test suites matching each of Facebook's 41 hate speech policies. Second, we test the performance of state-of-the-art hate speech detectors against HateModerate, revealing substantial failures these models have in their conformity to the policies. Third, using HateModerate, we augment the training data of a top-downloaded hate detector on HuggingFace. We observe significant improvement in the models' conformity to content policies while having comparable scores on the original test data. Our dataset and code can be found in the attachment.

cs.SE↗