arXiv subjects

Rui Shu

Publications and source records attributed to Rui Shu.

At least 19 recordsLinked to original sources

arXiv2026

A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents

Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs. Yet existing repository-level benchmarks typically evaluate only whether the final patch passes tests. Satisfying a user request requires a long chain of interdependent reasoning and decisions: an agent must recover explicit and implicit requirements, formulate a repository-grounded implementation plan, and translate it into correct code. A pass/fail outcome cannot characterize how an unsuccessful trajectory diverges from the requirements and implementation process needed for a correct patch. To address this gap, we introduce SWE-RPG, a repository-level benchmark that combines executable patch evaluation with validated ground-truth references (GTs) for (1) Requirement Clarification and (2) Implementation Planning. These intermediate GTs support retrospective, GT-aligned diagnosis of complete coding-agent trajectories across clarification, planning, code generation, and artifact submission. SWE-RPG comprises 163 tasks from 31 Python and Java repositories, including 113 bug fixes and 50 feature additions. We evaluate 3 coding agents, including Claude Code, Codex, and OpenCode, with 6 large language model backends, including Claude-Sonnet-5 and GPT-5.6-Terra. Results show that the evaluated popular coding agents still struggle to implement user requests in existing repositories, achieving an average resolved rate of only 31.5% on SWE-RPG. Intermediate-GT diagnosis further identifies implicit requirement recovery as the main bottleneck, accounting for 24.5%--46.0% of agent runs. This result suggests implicit-requirement recovery as a key candidate direction for improving coding agents. The benchmark data and evaluation code are available at https://github.com/Xin-Zhou-smu/SWE-RPG-Bench.

cs.SE↗

arXiv2026

What Resolve Rate Hides: Trajectory Structure Diagnostics for Coding Agents

Coding agents are ranked almost entirely by resolve rate: whether their final patch passes the target tests. Yet two agents can reach the same outcome through very different processes, and a single pass/fail label says nothing about why a run failed or why an accepted run spent extra steps, time, or tokens. This process evidence lives in the trajectory, which records a run's searches, reads, edits, tool calls, validation, and reversions. However, raw traces are heterogeneous and hard to compare across runs. We present TraceProbe, a trajectory-diagnostic framework that recovers what resolve rate hides. TraceProbe normalizes each raw run into a canonical nine-type action taxonomy with deterministic effect labels, then applies two rule-based modules: Insight names single-trajectory anti-patterns adapted from established debugging practice (e.g., search loops, verification skips), while Converge aligns pairs of runs and classifies where their behavior diverges under controlled references. Applying TraceProbe to 2,500 trajectories from five production settings on SWE-Bench Verified, we find that (i) file choice is too coarse to separate success from failure, whereas function selection and completion behavior localize it; (ii) Insight anti-patterns act mainly as corpus-level difficulty clues, with search loops the most stable; and (iii) even resolved runs differ in how quickly they reach relevant code and how much failed work they incur. Trajectory structure thus adds auditable diagnostic context to outcomes by localizing inspection targets, suggesting failure hypotheses, and prioritizing runs for review.

cs.SE↗

arXiv2026

SemiScope: Disentangling Classifier Tuning and Joint Optimization in Semi-Supervised Security Classification

Background. Labeled data for security classification is scarce. Semi-supervised learning (SSL) propagates labels from a small labeled pool to larger unlabeled pools. Yet security applications often use SSL as a black box: default parameters, a fixed classifier, and no handling of pseudo-label-induced class imbalance. Aims. Recent work reports sizeable gains from optimizing SSL pipelines via joint search, AutoML, or per-component tuning. These gains are hard to attribute: they may reflect useful SSL-classifier interactions, or mostly from simply tuning the downstream classifier. We disentangle these effects for binary tabular security data with classical SSL and tree-based classifiers. Method. We build SemiScope as an analysis instrument, not a deployment recommendation. It uses Bayesian Optimization to jointly tune SSL settings, confidence filtering, oversampling, and the classifier. The key control, Tuned-Clf, fixes SSL to defaults but gets the same 100-trial classifier budget and validation-set threshold tuning as SemiScope. At 10% labels, we compare them with paired TOST using a +/-1.0 g-measure smallest effect of interest. Results. SemiScope beats every default SSL baseline on all five datasets, improving over the strongest by 0.7-12.7 points. Under the equal-budget control, Tuned-Clf is statistically equivalent to the full pipeline on 4 of 5 datasets; Phishing is inconclusive. Classifier HPO alone recovers a median 86% of SemiScope's gain over Default Self-Training (ST) + Random Forest (RF). Conclusions. The reusable contribution is the decomposition protocol. A simpler recipe suffices: use Self-Training, tune the classifier with Bayesian Optimization, and tune the decision threshold on validation data. It reaches within 1 g-measure of Supervised RF at 20-30% labels on four datasets and 40% on Drebin, at the same or lower label rate than Default ST + RF on every dataset.

cs.LG↗

arXiv2025

Competitive Programming with Large Reasoning Models

We show that reinforcement learning applied to large language models (LLMs) significantly boosts performance on complex coding and reasoning tasks. Additionally, we compare two general-purpose reasoning models - OpenAI o1 and an early checkpoint of o3 - with a domain-specific system, o1-ioi, which uses hand-engineered inference strategies designed for competing in the 2024 International Olympiad in Informatics (IOI). We competed live at IOI 2024 with o1-ioi and, using hand-crafted test-time strategies, placed in the 49th percentile. Under relaxed competition constraints, o1-ioi achieved a gold medal. However, when evaluating later models such as o3, we find that o3 achieves gold without hand-crafted domain-specific strategies or relaxed constraints. Our findings show that although specialized pipelines such as o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model surpasses those results without relying on hand-crafted inference heuristics. Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces rating on par with elite human competitors. Overall, these results indicate that scaling general-purpose reinforcement learning, rather than relying on domain-specific techniques, offers a robust path toward state-of-the-art AI in reasoning domains, such as competitive programming.

cs.LG↗

arXiv2023

Thermoelectric properties and electronic structure of Cr(Mo,V)Nx thin films studied by synchrotron and lab-based X-ray spectroscopy

Chromium-based nitrides are used in hard, resilient coatings, and show promise for thermoelectric applications due to their combination of structural, thermal, and electronic properties. Here, we investigated the electronic structures and chemical bonding correlated to the thermoelectric properties of epitaxially grown chromium-based multicomponent nitride Cr(Mo,V)Nx thin films. Due to minuscule N vacancies, finite population of Cr 3d and N 2p states appear at the Fermi level and diminishes the band opening for Cr0.51N0.49. Incorporating holes by alloying V in N deficient CrN matrix results in enhanced thermoelectric power factor with marginal change in the charge transfer of Cr to N compared to Cr0.51N0.49. Further alloying Mo isoelectronic to Cr increases the density of states across the Fermi level due to hybridization of the (Cr, V) 3d and Mo 4d-N 2p states in Cr(Mo,V)Nx. The hybridization effect with reduced N 2p states off from stoichiometry drives the system towards metal like electrical resistivity and reduction in Seebeck coefficient compensating the overall power factor still comparable to Cr0.51N0.49. The N deficiency also depicts a critical role in reduction of the charge transfer from metal to N site. The present work envisages ways for enhancing thermoelectric properties through electronic band engineering by alloying and competing effects of N vacancies.

cond-mat.mtrl-sci↗

arXiv2022

Phase formation in CrFeCoNi nitride thin films

As a single-phase alloy, CrFeCoNi is a face centered cubic (fcc) material related to the archetypical high-entropy Cantor alloy CrFeCoNiMn. For thin films, CrFeCoNi of approximately equimolar composition tends to assume an fcc structure when grown at room temperature by magnetron sputtering. However, the single-phase solid solution state is typically not achieved for thin films grown at higher temperatures. The same holds true for Cantor alloy-based ceramics (nitrides and oxides), where phase formation is extremely sensitive to process parameters such as the amount of reactive gas. This study combines theoretical and experimental methods to understand the phase formation in nitrogen-containing CrFeCoNi thin films. Density functional theory calculations considering three competing phases (CrN, Fe-Ni and Co) show that the free energy of mixing, delta G of (CrFeCoNi)1-xNx solid solutions has a maximum at x = 0.20-0.25, and delta G becomes lower when x less than 0.20, greater than 0.25. Thin films of (CrFeCoNi)1-xNx (x = 0.14-0.41) grown by magnetron sputtering show stabilization of the metallic fcc when x lesser than or equal to 0.22 and the stabilization of the NaCl B1 structure when x is greater than 0.33, consistent with the theoretical prediction. In contrast, films with intermediate amounts of nitrogen (x = 0.22) grown at higher temperatures show segregation into multiple phases of CrN, Fe-Ni-rich and Co. These results offer an explanation for the requirement of kinetically limited growth conditions at low temperature for obtaining single-phase CrFeCoNi Cantor-like nitrogen-containing thin films and are of importance for understanding the phase-formation mechanisms in multicomponent ceramics.

cond-mat.mtrl-sci↗

arXiv2022

Single photon detection performance of highly disordered NbTiN thin films

We experimentally investigated the detection performance of highly disordered NbxTi1-xN based superconducting nanowire single photon detectors (SNSPDs). The dependence on the composition of the transition temperature Tc for NbxTi1-xN films show a dome-like behavior on the Nb content, with a maximal Tc at xNb~0.65 , and the Nb0.65Ti0.35N films also combine relatively large sheet resistance and intermediate residual resistivity ratio. Moreover, 60-nm-wide and 7-nm-thick Nb0.65Ti0.35N nanowires show a switching current as high as 14.5 uA, and saturated intrinsic detection efficiency with a plateau of more than 2 uA at 2.4 K. Finally, the corresponding SNSPDs on an alternative SiO2/Ta2O5 dielectric mirror showed a system detection efficiency of approximately 92% for 1550 nm photons, and the timing jitter is around 26 ps. Our results demonstrate that the highly disordered NbxTi1-xN films are promising for fabricating SNSPDs for near- and middle-infrared single photons with high detection efficiency and low timing jitter.

cond-mat.supr-con↗

arXiv2022

Do I really need all this work to find vulnerabilities? An empirical case study comparing vulnerability detection techniques on a Java application

CONTEXT: Applying vulnerability detection techniques is one of many tasks using the limited resources of a software project. OBJECTIVE: The goal of this research is to assist managers and other decision-makers in making informed choices about the use of software vulnerability detection techniques through an empirical study of the efficiency and effectiveness of four techniques on a Java-based web application. METHOD: We apply four different categories of vulnerability detection techniques \textendash~ systematic manual penetration testing (SMPT), exploratory manual penetration testing (EMPT), dynamic application security testing (DAST), and static application security testing (SAST) \textendash\ to an open-source medical records system. RESULTS: We found the most vulnerabilities using SAST. However, EMPT found more severe vulnerabilities. With each technique, we found unique vulnerabilities not found using the other techniques. The efficiency of manual techniques (EMPT, SMPT) was comparable to or better than the efficiency of automated techniques (DAST, SAST) in terms of Vulnerabilities per Hour (VpH). CONCLUSIONS: The vulnerability detection technique practitioners should select may vary based on the goals and available resources of the project. If the goal of an organization is to find "all" vulnerabilities in a project, they need to use as many techniques as their resources allow.

cs.SE↗

arXiv2022

Reducing the Cost of Training Security Classifier (via Optimized Semi-Supervised Learning)

Background: Most of the existing machine learning models for security tasks, such as spam detection, malware detection, or network intrusion detection, are built on supervised machine learning algorithms. In such a paradigm, models need a large amount of labeled data to learn the useful relationships between selected features and the target class. However, such labeled data can be scarce and expensive to acquire. Goal: To help security practitioners train useful security classification models when few labeled training data and many unlabeled training data are available. Method: We propose an adaptive framework called Dapper, which optimizes 1) semi-supervised learning algorithms to assign pseudo-labels to unlabeled data in a propagation paradigm and 2) the machine learning classifier (i.e., random forest). When the dataset class is highly imbalanced, Dapper then adaptively integrates and optimizes a data oversampling method called SMOTE. We use the novel Bayesian Optimization to search a large hyperparameter space of these tuning targets. Result: We evaluate Dapper with three security datasets, i.e., the Twitter spam dataset, the malware URLs dataset, and the CIC-IDS-2017 dataset. Experimental results indicate that we can use as low as 10% of original labeled data but achieve close or even better classification performance than using 100% labeled data in a supervised way. Conclusion: Based on those results, we would recommend using hyperparameter optimization with semi-supervised learning when dealing with shortages of labeled security data.

cs.CR↗

arXiv2022

Dazzle: Using Optimized Generative Adversarial Networks to Address Security Data Class Imbalance Issue

Background: Machine learning techniques have been widely used and demonstrate promising performance in many software security tasks such as software vulnerability prediction. However, the class ratio within software vulnerability datasets is often highly imbalanced (since the percentage of observed vulnerability is usually very low). Goal: To help security practitioners address software security data class imbalanced issues and further help build better prediction models with resampled datasets. Method: We introduce an approach called Dazzle which is an optimized version of conditional Wasserstein Generative Adversarial Networks with gradient penalty (cWGAN-GP). Dazzle explores the architecture hyperparameters of cWGAN-GP with a novel optimizer called Bayesian Optimization. We use Dazzle to generate minority class samples to resample the original imbalanced training dataset. Results: We evaluate Dazzle with three software security datasets, i.e., Moodle vulnerable files, Ambari bug reports, and JavaScript function code. We show that Dazzle is practical to use and demonstrates promising improvement over existing state-of-the-art oversampling techniques such as SMOTE (e.g., with an average of about 60% improvement rate over SMOTE in recall among all datasets). Conclusion: Based on this study, we would suggest the use of optimized GANs as an alternative method for security vulnerability data class imbalanced issues.

cs.CR↗

arXiv2021

Solid-state Janus nanoprecipitation enables amorphous-like heat conduction in crystalline Mg3Sb2-based thermoelectric materials

Solid-state precipitation can be used to tailor materials properties, ranging from ferromagnets and catalysts to mechanical strengthening and energy storage. Thermoelectric properties can be modified by precipitation to enhance phonon scattering while retaining charge-carrier transmission. Here, we uncover unconventional dual Janus-type nanoprecipitates in Mg3Sb1.5Bi0.5 formed by side-by-side Bi- and Ge-rich appendages, in contrast to separate nanoprecipitate formation. These Janus nanoprecipitates result from local co-melting of Bi and Ge during sintering, enabling an amorphous-like lattice thermal conductivity. A precipitate size effect on phonon scattering is observed due to the balance between alloy-disorder and nanoprecipitate scattering. The thermoelectric figure-of-merit ZT reaches 0.6 near room temperature and 1.6 at 773 K. The Janus nanoprecipitation can be introduced into other materials and may act as a general property-tailoring mechanism.

cond-mat.mtrl-sci↗

arXiv2021

Temporal Predictive Coding For Model-Based Planning In Latent Space

High-dimensional observations are a major challenge in the application of model-based reinforcement learning (MBRL) to real-world environments. To handle high-dimensional sensory inputs, existing approaches use representation learning to map high-dimensional observations into a lower-dimensional latent space that is more amenable to dynamics estimation and planning. In this work, we present an information-theoretic approach that employs temporal predictive coding to encode elements in the environment that can be predicted across time. Since this approach focuses on encoding temporally-predictable information, we implicitly prioritize the encoding of task-relevant components over nuisance information within the environment that are provably task-irrelevant. By learning this representation in conjunction with a recurrent state space model, we can then perform planning in latent space. We evaluate our model on a challenging modification of standard DMControl tasks where the background is replaced with natural videos that contain complex but irrelevant information to the planning task. Our experiments show that our model is superior to existing methods in the challenging complex-background setting while remaining competitive with current state-of-the-art models in the standard setting.

cs.LG↗

arXiv2021

Structuring a Comprehensive Software Security Course Around the OWASP Application Security Verification Standard

Lack of security expertise among software practitioners is a problem with many implications. First, there is a deficit of security professionals to meet current needs. Additionally, even practitioners who do not plan to work in security may benefit from increased understanding of security. The goal of this paper is to aid software engineering educators in designing a comprehensive software security course by sharing an experience running a software security course for the eleventh time. Through all the eleven years of running the software security course, the course objectives have been comprehensive - ranging from security testing, to secure design and coding, to security requirements to security risk management. For the first time in this eleventh year, a theme of the course assignments was to map vulnerability discovery to the security controls of the Open Web Application Security Project (OWASP) Application Security Verification Standard (ASVS). Based upon student performance on a final exploratory penetration testing project, this mapping may have increased students' depth of understanding of a wider range of security topics. The students efficiently detected 191 unique and verified vulnerabilities of 28 different Common Weakness Enumeration (CWE) types during a three-hour period in the OpenMRS project, an electronic health record application in active use.

cs.SE↗

arXiv2021

Anytime Sampling for Autoregressive Models via Ordered Autoencoding

Autoregressive models are widely used for tasks such as image and audio generation. The sampling process of these models, however, does not allow interruptions and cannot adapt to real-time computational resources. This challenge impedes the deployment of powerful autoregressive models, which involve a slow sampling process that is sequential in nature and typically scales linearly with respect to the data dimension. To address this difficulty, we propose a new family of autoregressive models that enables anytime sampling. Inspired by Principal Component Analysis, we learn a structured representation space where dimensions are ordered based on their importance with respect to reconstruction. Using an autoregressive model in this latent space, we trade off sample quality for computational efficiency by truncating the generation process before decoding into the original data space. Experimentally, we demonstrate in several image and audio generation tasks that sample quality degrades gracefully as we reduce the computational budget for sampling. The approach suffers almost no loss in sample quality (measured by FID) using only 60\% to 80\% of all latent dimensions for image data. Code is available at https://github.com/Newbeeer/Anytime-Auto-Regressive-Model .

cs.LG↗

arXiv2020

Omni: Automated Ensemble with Unexpected Models against Adversarial Evasion Attack

Background: Machine learning-based security detection models have become prevalent in modern malware and intrusion detection systems. However, previous studies show that such models are susceptible to adversarial evasion attacks. In this type of attack, inputs (i.e., adversarial examples) are specially crafted by intelligent malicious adversaries, with the aim of being misclassified by existing state-of-the-art models (e.g., deep neural networks). Once the attackers can fool a classifier to think that a malicious input is actually benign, they can render a machine learning-based malware or intrusion detection system ineffective. Goal: To help security practitioners and researchers build a more robust model against non-adaptive, white-box, and non-targeted adversarial evasion attacks through the idea of an ensemble model. Method: We propose an approach called Omni, the main idea of which is to explore methods that create an ensemble of "unexpected models"; i.e., models whose control hyperparameters have a large distance to the hyperparameters of an adversary's target model, with which we then make an optimized weighted ensemble prediction. Result: In studies with five types of adversarial evasion attacks (FGSM, BIM, JSMA, DeepFooland Carlini-Wagner) on five security datasets (NSL-KDD, CIC-IDS-2017, CSE-CIC-IDS2018, CICAnd-Mal2017, and the Contagio PDF dataset), we show Omni is a promising approach as a defense strategy against adversarial attacks when compared with other baseline treatments. Conclusion: When employing ensemble defense against adversarial evasion attacks, we suggest creating an ensemble with unexpected models that are distant from the attacker's expected model (i.e., target model) through methods such as hyperparameter optimization.

cs.CR↗

arXiv2020

Predicting Health Indicators for Open Source Projects (using Hyperparameter Optimization)

Software developed on public platform is a source of data that can be used to make predictions about those projects. While the individual developing activity may be random and hard to predict, the developing behavior on project level can be predicted with good accuracy when large groups of developers work together on software projects. To demonstrate this, we use 64,181 months of data from 1,159 GitHub projects to make various predictions about the recent status of those projects (as of April 2020). We find that traditional estimation algorithms make many mistakes. Algorithms like $k$-nearest neighbors (KNN), support vector regression (SVR), random forest (RFT), linear regression (LNR), and regression trees (CART) have high error rates. But that error rate can be greatly reduced using hyperparameter optimization. To the best of our knowledge, this is the largest study yet conducted, using recent data for predicting multiple health indicators of open-source projects.

cs.SE↗

arXiv2020

Predictive Coding for Locally-Linear Control

High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks. The Learning Controllable Embedding (LCE) framework addresses these challenges by embedding the observations into a lower dimensional latent space, estimating the latent dynamics, and then performing control directly in the latent space. To ensure the learned latent dynamics are predictive of next-observations, all existing LCE approaches decode back into the observation space and explicitly perform next-observation prediction---a challenging high-dimensional task that furthermore introduces a large number of nuisance parameters (i.e., the decoder) which are discarded during control. In this paper, we propose a novel information-theoretic LCE approach and show theoretically that explicit next-observation prediction can be replaced with predictive coding. We then use predictive coding to develop a decoder-free LCE model whose latent dynamics are amenable to locally-linear control. Extensive experiments on benchmark tasks show that our model reliably learns a controllable latent space that leads to superior performance when compared with state-of-the-art LCE baselines.

cs.LG↗

arXiv2019

Sequential Model Optimization for Software Process Control

Many methods have been proposed to estimate how much effort is required to build and maintain software. Much of that research assumes a ``classic'' waterfall-based approach rather than contemporary projects (where the developing process may be more iterative than linear in nature). Also, much of that work tries to recommend a single method-- an approach that makes the dubious assumption that one method can handle the diversity of software project data. To address these drawbacks, we apply a configuration technique called ``ROME'' (Rapid Optimizing Methods for Estimation), which uses sequential model-based optimization (SMO) to find what combination of effort estimation techniques works best for a particular data set. We test this method using data from 1161 classic waterfall projects and 120 contemporary projects (from Github). In terms of magnitude of relative error and standardized accuracy, we find that ROME achieves better performance than existing state-of-the-art methods for both classic and contemporary problems. In addition, we conclude that we should not recommend one method for estimation. Rather, it is better to search through a wide range of different methods to find what works best for local data. To the best of our knowledge, this is the largest effort estimation experiment yet attempted and the only one to test its methods on classic and contemporary projects.

cs.SE↗