arXiv subjects

Ivan Girardi

Publications and source records attributed to Ivan Girardi.

9 recordsLinked to original sources

arXiv2022

Estimating the Adversarial Robustness of Attributions in Text with Transformers

Explanations are crucial parts of deep neural network (DNN) classifiers. In high stakes applications, faithful and robust explanations are important to understand and gain trust in DNN classifiers. However, recent work has shown that state-of-the-art attribution methods in text classifiers are susceptible to imperceptible adversarial perturbations that alter explanations significantly while maintaining the correct prediction outcome. If undetected, this can critically mislead the users of DNNs. Thus, it is crucial to understand the influence of such adversarial perturbations on the networks' explanations and their perceptibility. In this work, we establish a novel definition of attribution robustness (AR) in text classification, based on Lipschitz continuity. Crucially, it reflects both attribution change induced by adversarial input alterations and perceptibility of such alterations. Moreover, we introduce a wide set of text similarity measures to effectively capture locality between two text samples and imperceptibility of adversarial perturbations in text. We then propose our novel TransformerExplanationAttack (TEA), a strong adversary that provides a tight estimation for attribution robustness in text classification. TEA uses state-of-the-art language models to extract word substitutions that result in fluent, contextual adversarial samples. Finally, with experiments on several text classification architectures, we show that TEA consistently outperforms current state-of-the-art AR estimators, yielding perturbations that alter explanations to a greater extent while being more fluent and less perceptible.

cs.LG↗

arXiv2022

Fooling Explanations in Text Classifiers

State-of-the-art text classification models are becoming increasingly reliant on deep neural networks (DNNs). Due to their black-box nature, faithful and robust explanation methods need to accompany classifiers for deployment in real-life scenarios. However, it has been shown in vision applications that explanation methods are susceptible to local, imperceptible perturbations that can significantly alter the explanations without changing the predicted classes. We show here that the existence of such perturbations extends to text classifiers as well. Specifically, we introduceTextExplanationFooler (TEF), a novel explanation attack algorithm that alters text input samples imperceptibly so that the outcome of widely-used explanation methods changes considerably while leaving classifier predictions unchanged. We evaluate the performance of the attribution robustness estimation performance in TEF on five sequence classification datasets, utilizing three DNN architectures and three transformer architectures for each dataset. TEF can significantly decrease the correlation between unchanged and perturbed input attributions, which shows that all models and explanation methods are susceptible to TEF perturbations. Moreover, we evaluate how the perturbations transfer to other model architectures and attribution methods, and show that TEF perturbations are also effective in scenarios where the target model and explanation method are unknown. Finally, we introduce a semi-universal attack that is able to compute fast, computationally light perturbations with no knowledge of the attacked classifier nor explanation method. Overall, our work shows that explanations in text classifiers are very fragile and users need to carefully address their robustness before relying on them in critical applications.

cs.LG↗

arXiv2022

FAR: A General Framework for Attributional Robustness

Attribution maps are popular tools for explaining neural networks predictions. By assigning an importance value to each input dimension that represents its impact towards the outcome, they give an intuitive explanation of the decision process. However, recent work has discovered vulnerability of these maps to imperceptible adversarial changes, which can prove critical in safety-relevant domains such as healthcare. Therefore, we define a novel generic framework for attributional robustness (FAR) as general problem formulation for training models with robust attributions. This framework consist of a generic regularization term and training objective that minimize the maximal dissimilarity of attribution maps in a local neighbourhood of the input. We show that FAR is a generalized, less constrained formulation of currently existing training methods. We then propose two new instantiations of this framework, AAT and AdvAAT, that directly optimize for both robust attributions and predictions. Experiments performed on widely used vision datasets show that our methods perform better or comparably to current ones in terms of attributional robustness while being more generally applicable. We finally show that our methods mitigate undesired dependencies between attributional robustness and some training and estimation parameters, which seem to critically affect other competitor methods.

cs.LG↗

arXiv2021

On the explainability of hospitalization prediction on a large COVID-19 patient dataset

We develop various AI models to predict hospitalization on a large (over 110$k$) cohort of COVID-19 positive-tested US patients, sourced from March 2020 to February 2021. Models range from Random Forest to Neural Network (NN) and Time Convolutional NN, where combination of the data modalities (tabular and time dependent) are performed at different stages (early vs. model fusion). Despite high data unbalance, the models reach average precision 0.96-0.98 (0.75-0.85), recall 0.96-0.98 (0.74-0.85), and $F_1$-score 0.97-0.98 (0.79-0.83) on the non-hospitalized (or hospitalized) class. Performances do not significantly drop even when selected lists of features are removed to study model adaptability to different scenarios. However, a systematic study of the SHAP feature importance values for the developed models in the different scenarios shows a large variability across models and use cases. This calls for even more complete studies on several explainability methods before their adoption in high-stakes scenarios.

cs.LG↗

arXiv2020

Artificial Intelligence Decision Support for Medical Triage

Applying state-of-the-art machine learning and natural language processing on approximately one million of teleconsultation records, we developed a triage system, now certified and in use at the largest European telemedicine provider. The system evaluates care alternatives through interactions with patients via a mobile application. Reasoning on an initial set of provided symptoms, the triage application generates AI-powered, personalized questions to better characterize the problem and recommends the most appropriate point of care and time frame for a consultation. The underlying technology was developed to meet the needs for performance, transparency, user acceptance and ease of use, central aspects to the adoption of AI-based decision support systems. Providing such remote guidance at the beginning of the chain of care has significant potential for improving cost efficiency, patient experience and outcomes. Being remote, always available and highly scalable, this service is fundamental in high demand situations, such as the current COVID-19 outbreak.

cs.AI↗

arXiv2018

Patient Risk Assessment and Warning Symptom Detection Using Deep Attention-Based Neural Networks

We present an operational component of a real-world patient triage system. Given a specific patient presentation, the system is able to assess the level of medical urgency and issue the most appropriate recommendation in terms of best point of care and time to treat. We use an attention-based convolutional neural network architecture trained on 600,000 doctor notes in German. We compare two approaches, one that uses the full text of the medical notes and one that uses only a selected list of medical entities extracted from the text. These approaches achieve 79% and 66% precision, respectively, but on a confidence threshold of 0.6, precision increases to 85% and 75%, respectively. In addition, a method to detect warning symptoms is implemented to render the classification task transparent from a medical perspective. The method is based on the learning of attention scores and a method of automatic validation using the same data.

cs.CL↗

arXiv2014

Constraining Sterile Neutrinos Using Reactor Neutrino Experiments

Models of neutrino mixing involving one or more sterile neutrinos have resurrected their importance in the light of recent cosmological data. In this case, reactor antineutrino experiments offer an ideal place to look for signatures of sterile neutrinos due to their impact on neutrino flavor transitions. In this work, we show that the high-precision data of the Daya Bay experi\-ment constrain the 3+1 neutrino scenario imposing upper bounds on the relevant active-sterile mixing angle $\sin^2 2 θ_{14} \lesssim 0.06$ at 3$σ$ confidence level for the mass-squared difference $Δm^2_{41}$ in the range $(10^{-3},10^{-1}) \, {\rm eV^2}$. The latter bound can be improved by six years of running of the JUNO experiment, $\sin^22θ_{14} \lesssim 0.016$, although in the smaller mass range $ Δm^2_{41} \in (10^{-4} ,10^{-3}) \, {\rm eV}^2$. We have also investigated the impact of sterile neutrinos on precision measurements of the standard neutrino oscillation parameters $θ_{13}$ and $Δm^2_{31}$ (at Daya Bay and JUNO), $θ_{12}$ and $Δm^2_{21}$ (at JUNO), and most importantly, the neutrino mass hierarchy (at JUNO). We find that, except for the obvious situation where $Δm^2_{41}\sim Δm^2_{31}$, sterile states do not affect these measurements substantially.

hep-ph↗

arXiv2014

Generalised Geometrical CP Violation in a $T^{\prime}$ Lepton Flavour Model

We analyse the interplay of generalised CP transformations and the non-Abelian discrete group $T^{\prime}$ and use the semi-direct product $G_f= T^{\prime}\rtimes H_{\text{CP}}$, as family symmetry acting in the lepton sector. The family symmetry is shown to be spontaneously broken in a geometrical manner. In the resulting flavour model, naturally small Majorana neutrino masses for the light active neutrinos are obtained through the type I see-saw mechanism. The known masses of the charged leptons, lepton mixing angles and the two neutrino mass squared differences are reproduced by the model with a good accuracy. The model allows for two neutrino mass spectra with normal ordering (NO) and one with inverted ordering (IO). For each of the three spectra the absolute scale of neutrino masses is predicted with relatively small uncertainty. The value of the Dirac CP violation (CPV) phase $δ$ in the lepton mixing matrix is predicted to be $δ\cong π/2~{\rm or}~ 3π/2$. Thus, the CP violating effects in neutrino oscillations are predicted to be maximal (given the values of the neutrino mixing angles) and experimentally observable. We present also predictions for the sum of the neutrino masses, for the Majorana CPV phases and for the effective Majorana mass in neutrinoless double beta decay. The predictions of the model can be tested in a variety of ongoing and future planned neutrino experiments.

hep-ph↗

arXiv2013

Neutrinoless Double Beta Decay in the Presence of Light Sterile Neutrinos

We investigate the predictions for neutrinoless double beta ($(ββ)_{0 ν}$-) decay effective Majorana mass $\left| \langle \, m \, \rangle \right|$ in the 3+1 and 3+2 schemes with one and two additional sterile neutrinos with masses at the eV scale. The two schemes are suggested by the neutrino oscillation interpretation of the reactor neutrino and Gallium "anomalies" and of the data of the LSND and MiniBooNE experiments. We analyse in detail the possibility of a complete or partial cancellation between the different terms in $\left| \langle \, m \, \rangle \right|$, leading to a strong suppression of $\left| \langle \, m \, \rangle \right|$. We determine the regions of the relevant parameter spaces where such a suppression can occure. This allows us to derive the conditions under which the effective Majorana mass satisfies $\left| \langle \, m \, \rangle \right| > 0.01$ eV, which is the range planned to be exploited by the next generation of $(ββ)_{0 ν}$-experiments.

hep-ph↗