Searcharxiv⌕ Search

arXiv subjects

Lakmal Deshapriya

Publications and source records attributed to Lakmal Deshapriya.

3 recordsLinked to original sources

Using machine learning approaches to classify false-positives generated by static analysis tools - An empirical study using small datasets

Static Code Analysis (SCA) tools are vital for software development to reduce the cost and time required for manual code reviews. It was established in our previous study that the best tools reported in the software engineering community return high false-positive and false-negative rates when used for code analysis. Several approaches have been taken in existing studies to identify false-positive alarms generated by SCA tools, in helping to enhance the utility of these tools. However, existing studies reveal several limitations, chiefly among which is the tendency to focus on a limited number of alarm types or failing to evaluate a large breadth of Machine-Learning (ML) techniques comprehensively. This study has addressed this opportunity. We identified the best Code Representation Learning (CRL) techniques and ML algorithms for identifying false-positives, where results show that performance outcomes vary based on the characteristics of the training datasets and the goal of the model building. Given the evidence established in this study, we recommend further investigations targeting specific configurations of advanced CRLs and ML Algorithms, including simple ML models.

cs.SE↗

Beyond Benchmarks: Evaluating Large Language Models and Agentic Patterns for Static Code Analysis Warning Classification

Context: Static Code Analysis (SCA) tools support software verification and validation, reducing the time and cost of code reviews. However, the best SCA tools in the practitioner community generate a high volume of false positives, which developers tend to ignore, leading to genuine warnings being overlooked. Accordingly, studies increasingly adopt Large Language Models (LLMs) to identify and reduce false positives. Existing studies have produced promising results on public datasets, but underperformed on unseen datasets. Moreover, researchers do not leverage the full potential of Agentic patterns in LLMs or incorporate established code quality standards for warning classification. Objectives: This study utilises four manually annotated datasets to evaluate the performance of three contemporary LLMs for SCA warning classification in real-world applications, and incorporates Agentic patterns and established coding standards to enhance models' performance, addressing gaps identified in existing studies. Method: Three LLMs were selected using the LiveCodeBench benchmark, and the Structured Chain-of-Thought (SCoT) prompting technique was used to evaluate these LLMs. Furthermore, we incorporated Agentic patterns, such as Debating and Self-Consistency, to potentially improve performance in SCA warning classification. Results: Our results show that the performance of different LLMs varies with the characteristics of the dataset, ranging from 0.52 to 0.96 F1-Score for the best models. Additionally, the use of Agentic patterns does not improve LLMs' performance in most scenarios. Conclusions: SCA warning classification using LLMs works best only with certain warning types. We recommend evaluating LLMs in real-world applications rather than using publicly available datasets/benchmarks to understand their full utility.

cs.SE↗

Taking a Closer Look at Warnings Generated by PMD and SonarQube, their Rules and Compliance to Established Coding Standards

Context: Static code analysis (SCA) tools play a vital role in software development, reducing the cost and time required for code reviews. However, high false-positive and false-negative rates are reported for the best tools in the community. Accordingly, studies often aim to develop datasets for learning SCA warning patterns to reduce false results. These datasets are meant to possess high-quality and high-volume in covering the full range of faults/rules that typically result in false warnings and be compliant with established coding standards. However, existing studies have not utilised such datasets or identified the breadth of rules that are prone to false positives and their compliance to coding standards. Objectives: We analysed code from Stack Overflow and Apache Tomcat to capture variations in code length and style in detecting false-positive warnings from best-performing tools PMD and SonarQube, addressing this gap. Method: In deriving false-positive warnings, outcomes from the tools were labelled using established coding standards. Deeper analyses were then conducted to identify the rules that are prone to false-positives, reasons for these, and agreement/gaps between SCA rules and established standards. Results: Among our main outcomes, we observe that only a few SCA rules generate false positives, ranging from 4.64% to 18.45% across four datasets. Additionally, eliminating rules that contradict established standards significantly reduce the false-positive rate. Additionally, our findings reveal discrepancies between tools and established standards. Conclusion: Given the evidence established in this study, we recommend further investigations into gaps between tools and established standards, including the use of machine learning approaches to annotate larger datasets.

cs.SE↗