destroR: A Benchmark and Adversarial-Training Defense for Bangla Transfer Models under Meaning-Preserving Attacks

Abstract

Transformer-based transfer models now dominate Bangla sentiment classification, yet their adversarial robustness remains largely unexamined, and no prior study pairs a Bangla attack suite with a defense that measurably recovers robustness. We address this gap with destroR, a unified pipeline for evaluating and hardening Bangla text classifiers. First, we introduce three meaning-preserving Bangla attack recipes a paraphrase attack, a back-translation attack, and a one-hot word-swap attack that perturb inputs while regenerating fluent, semantically faithful sentences, inducing model prediction perplexity rather than input noise. Second, we construct a robustness benchmark that evaluates five transfer models (BanglaBERT, BanglishBERT, XLM-RoBERTa, MuRIL, and IndicBERTv2) across four datasets against five attacks, placing our recipes against two strong word-substitution baselines, TextFooler and BAE, under an identical protocol. Third, we harden every model through adversarial training and report a full robustness matrix. Our analysis yields three findings: word-substitution baselines are more potent than semantically constrained recipes (BAE reaches a 54.2% attack success rate); adversarial training on the union of all attack families lowers residual attack success for every attack; and, contrary to expectation, the Indic-multilingual MuRIL backbone is markedly more robust than the Bangla-dedicated models. All models, adversarial data, and code are released for full reproducibility.