arXiv subjects

Sankardas Roy

Publications and source records attributed to Sankardas Roy.

2 recordsLinked to original sources

arXiv2025

Benchmarking Android Malware Detection: Traditional vs. Deep Learning Models

Android malware detection has been extensively studied using both traditional machine learning (ML) and deep learning (DL) approaches. While many state-of-the-art detection models, particularly those based on DL, claim superior performance, they often rely on limited comparisons, lacking comprehensive benchmarking against traditional ML models across diverse datasets. This raises concerns about the robustness of DL-based approaches' performance and the potential oversight of simpler, more efficient ML models. In this paper, we conduct a systematic evaluation of Android malware detection models across four datasets: three recently published, publicly available datasets and a large-scale dataset we systematically collected. We implement a range of traditional ML models, including Random Forests (RF) and CatBoost, alongside advanced DL models such as Capsule Graph Neural Networks (CapsGNN), BERT-based models, and ExcelFormer based models. Our results reveal that in many cases simpler and more computationally efficient ML models achieve comparable or even superior performance compared with DL models. These findings highlight the need for rigorous benchmarking in Android malware detection research. We encourage future studies to conduct more comprehensive benchmarking comparisons between traditional and advanced models to ensure a more accurate assessment of detection capabilities. To facilitate further research, we provide access to our dataset, including app IDs, hash values, and labels.

cs.CR↗

arXiv2024

The Impact of Train-Test Leakage on Machine Learning-based Android Malware Detection

When machine learning is used for Android malware detection, an app needs to be represented in a numerical format for training and testing. We identify a widespread occurrence of distinct Android apps that have identical or nearly identical app representations. In particular, among app samples in the testing dataset, there can be a significant percentage of apps that have an identical or nearly identical representation to an app in the training dataset. This will lead to a data leakage problem that inflates a machine learning model's performance as measured on the testing dataset. The data leakage not only could lead to overly optimistic perceptions on the machine learning models' ability to generalize beyond the data on which they are trained, in some cases it could also lead to qualitatively different conclusions being drawn from the research. We present two case studies to illustrate this impact. In the first case study, the data leakage inflated the performance results but did not impact the overall conclusions made by the researchers in a qualitative way. In the second case study, the data leakage problem would have led to qualitatively different conclusions being drawn from the research. We further examine the real-world impact of the data leakage by dissecting the capability of memorization and the capability of generalization of a machine learning model, and show that by removing leakage from testing data, the evaluation results better reflect the machine learning model's utility in real-world Android malware detection scenarios.

cs.CR↗