arXiv subjects

Tasfia Tasnim

Publications and source records attributed to Tasfia Tasnim.

2 recordsLinked to original sources

arXiv2026

PITMuS: A Tool for Automated Bug Dataset Generation via Source-Level Mutant Reconstruction

Mutation testing provides a scalable mechanism for generating controlled faults for software testing and empirical software engineering. In Java, PIT is a widely used mutation testing tool that creates large numbers of mutants for evaluating test suites. However, PIT creates mutants at the bytecode level and reports mutation metadata rather than the corresponding source edits. Consequently, these mutants are difficult to inspect, replay, and reuse as structured bug artifacts. We present PITMuS, a tool that reconstructs PIT mutants at the source level and automatically generates reusable mutation-based bug datasets. Given standard PIT reports and the original Java source, and using compiled bytecode when available, PITMuS localizes the affected source statement, applies the corresponding edit, and associates the result with its enclosing method, documentation, and mutation metadata. The tool also supports source-level mutant injection for inspection, replay, and downstream experimentation. We evaluate PITMuS on eight open-source Java systems. From 69,229 mutations reported by PIT, PITMuS produces 69,198 source-level original-mutant method pairs across 1,913 source files, yielding source-level original-mutant pairs for 99.96% of the reported mutation records. These results show that bytecode-level mutation reports can be converted at high coverage into inspectable, context-rich source artifacts for software testing, program repair, and learning-based software engineering.

cs.SE↗

arXiv2026

Documentation vs. Code Patterns: What Drives LLM-Based Exception Oracle Generation?

LLM-based test oracle generation (TOG) methods report high accuracy on exception oracle generation, but it remains unclear what evidence drives these predictions. In particular, do models use explicit exceptional-behavior documentation such as Javadoc @throws clauses, or do they rely on recurring patterns in tests, code, and documentation? We investigate this question through a large-scale intervention-based study of three TOG systems spanning classifier-based and generative architectures and model sizes from roughly 110M to 7B parameters, evaluated on three real-world benchmarks comprising two generated-test datasets and a new benchmark of developer-written tests. We first remove Javadoc @throws clauses and find that accuracy changes only marginally, with the largest drop below one percentage point. This indicates that structured exception documentation is not the primary driver of exception-oracle prediction. We then apply attribution-guided substitution ablations to identify the signals that predictions depend on. The results show that high accuracy can be driven by shortcut signals: some models are highly sensitive to a small number of structural tokens, while others distribute reliance across many lexical cues. These findings challenge the assumption that strong exception-oracle accuracy reflects robust use of exception semantics. Future TOG systems should therefore be evaluated not only by whether they predict the correct oracle type, but also by whether their predictions are grounded in meaningful exception-triggering evidence.

cs.SE↗