arXiv subjects

Patrick McDaniel

Publications and source records attributed to Patrick McDaniel.

79 records · Page 5Linked to original sources

arXiv2016

Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples

Many machine learning models are vulnerable to adversarial examples: inputs that are specially crafted to cause a machine learning model to produce an incorrect output. Adversarial examples that affect one model often affect another model, even if the two models have different architectures or were trained on different training sets, so long as both models were trained to perform the same task. An attacker may therefore train their own substitute model, craft adversarial examples against the substitute, and transfer them to a victim model, with very little information about the victim. Recent work has further developed a technique that uses the victim model as an oracle to label a synthetic training set for the substitute, so the attacker need not even collect a training set to mount the attack. We extend these recent techniques using reservoir sampling to greatly enhance the efficiency of the training procedure for the substitute model. We introduce new transferability attacks between previously unexplored (substitute, victim) pairs of machine learning model classes, most notably SVMs and decision trees. We demonstrate our attacks on two commercial machine learning classification systems from Amazon (96.19% misclassification rate) and Google (88.94%) using only 800 queries of the victim model, thereby showing that existing machine learning approaches are in general vulnerable to systematic black-box attacks regardless of their structure.

cs.CR↗

arXiv2016

Crafting Adversarial Input Sequences for Recurrent Neural Networks

Machine learning models are frequently used to solve complex security problems, as well as to make decisions in sensitive situations like guiding autonomous vehicles or predicting financial market behaviors. Previous efforts have shown that numerous machine learning models were vulnerable to adversarial manipulations of their inputs taking the form of adversarial samples. Such inputs are crafted by adding carefully selected perturbations to legitimate inputs so as to force the machine learning model to misbehave, for instance by outputting a wrong class if the machine learning task of interest is classification. In fact, to the best of our knowledge, all previous work on adversarial samples crafting for neural network considered models used to solve classification tasks, most frequently in computer vision applications. In this paper, we contribute to the field of adversarial machine learning by investigating adversarial input sequences for recurrent neural networks processing sequential data. We show that the classes of algorithms introduced previously to craft adversarial samples misclassified by feed-forward neural networks can be adapted to recurrent neural networks. In a experiment, we show that adversaries can craft adversarial sequences misleading both categorical and sequential recurrent neural networks.

cs.CR↗

arXiv2016

Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks

Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 10^30. We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.

cs.CR↗

arXiv2016

Towards Least Privilege Containers with Cimplifier

Application containers, such as Docker containers, have recently gained popularity as a solution for agile and seamless deployment of applications. These light-weight virtualization environments run applications that are packed together with their resources and configuration information, and thus can be deployed across various software platforms. However, these software ecosystems are not conducive to the true and tried security principles of privilege separation (PS) and principle of least privilege (PLP). We propose algorithms and a tool Cimplifier, which address these concerns in the context of containers. Specifically, given a container our tool partitions them into simpler containers, which are only provided enough resources to perform their functionality. As part our solution, we develop techniques for analyzing resource usage, for performing partitioning, and gluing the containers together to preserve functionality. Our evaluation on real-world containers demonstrates that Cimplifier can preserve the original functionality, leads to reduction in image size of 58-95%, and processes even large containers in under thirty seconds.

cs.CR↗

arXiv2015

The Limitations of Deep Learning in Adversarial Settings

Deep learning takes advantage of large datasets and computationally efficient training algorithms to outperform other approaches at various machine learning tasks. However, imperfections in the training phase of deep neural networks make them vulnerable to adversarial samples: inputs crafted by adversaries with the intent of causing deep neural networks to misclassify. In this work, we formalize the space of adversaries against deep neural networks (DNNs) and introduce a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs. In an application to computer vision, we show that our algorithms can reliably produce samples correctly classified by human subjects but misclassified in specific targets by a DNN with a 97% adversarial success rate while only modifying on average 4.02% of the input features per sample. We then evaluate the vulnerability of different sample classes to adversarial perturbations by defining a hardness measure. Finally, we describe preliminary work outlining defenses against adversarial samples by defining a predictive measure of distance between a benign input and a target classification.

cs.CR↗

arXiv2015

Six Potential Game-Changers in Cyber Security: Towards Priorities in Cyber Science and Engineering

The fields of study encompassed by cyber science and engineering are broad and poorly defined at this time. As national governments and research communities increase their recognition of the importance, urgency and technical richness of these disciplines, a question of priorities arises: what specific sub-areas of research should be the foci of attention and funding? In this paper we point to an approach to answering this question. We explore results of a recent workshop that postulated possible game-changers or disruptive changes that might occur in cyber security within the next 15 years. We suggest that such game-changers may be useful in focusing attention of research communities on high-priority topics. Indeed, if a drastic, important change is likely to occur, should we not focus our research efforts on the nature and ramifications of the phenomena pertaining to that change? We illustrate each of the game-changers examples of related current research, and then offer recommendations for advancement of cyber science and engineering with respect to each of the six game-changers.

cs.CR↗

arXiv2014

I know what leaked in your pocket: uncovering privacy leaks on Android Apps with Static Taint Analysis

Android applications may leak privacy data carelessly or maliciously. In this work we perform inter-component data-flow analysis to detect privacy leaks between components of Android applications. Unlike all current approaches, our tool, called IccTA, propagates the context between the components, which improves the precision of the analysis. IccTA outperforms all other available tools by reaching a precision of 95.0% and a recall of 82.6% on DroidBench. Our approach detects 147 inter-component based privacy leaks in 14 applications in a set of 3000 real-world applications with a precision of 88.4%. With the help of ApkCombiner, our approach is able to detect inter-app based privacy leaks.

cs.SE↗