Searcharxiv⌕ Search

arXiv subjects

Ross Anderson

Publications and source records attributed to Ross Anderson.

49 records · Page 3Linked to original sources
arXiv2020

Towards Certifiable Adversarial Sample Detection

Convolutional Neural Networks (CNNs) are deployed in more and more classification systems, but adversarial samples can be maliciously crafted to trick them, and are becoming a real threat. There have been various proposals to improve CNNs' adversarial robustness but these all suffer performance penalties or other limitations. In this paper, we provide a new approach in the form of a certifiable adversarial detection scheme, the Certifiable Taboo Trap (CTT). The system can provide certifiable guarantees of detection of adversarial inputs for certain $l_{\infty}$ sizes on a reasonable assumption, namely that the training data have the same distribution as the test data. We develop and evaluate several versions of CTT with a range of defense capabilities, training overheads and certifiability on adversarial samples. Against adversaries with various $l_p$ norms, CTT outperforms existing defense methods that focus purely on improving network robustness. We show that CTT has small false positive rates on clean test data, minimal compute overheads when deployed, and can support complex security policies.

cs.LG↗
arXiv2020

Strong mixed-integer programming formulations for trained neural networks

We present strong mixed-integer programming (MIP) formulations for high-dimensional piecewise linear functions that correspond to trained neural networks. These formulations can be used for a number of important tasks, such as verifying that an image classification network is robust to adversarial inputs, or solving decision problems where the objective function is a machine learning model. We present a generic framework, which may be of independent interest, that provides a way to construct sharp or ideal formulations for the maximum of d affine functions over arbitrary polyhedral input domains. We apply this result to derive MIP formulations for a number of the most popular nonlinear operations (e.g. ReLU and max pooling) that are strictly stronger than other approaches from the literature. We corroborate this computationally, showing that our formulations are able to offer substantial improvements in solve time on verification tasks for image classification networks.

math.OC↗
arXiv2019

The Taboo Trap: Behavioural Detection of Adversarial Samples

Deep Neural Networks (DNNs) have become a powerful toolfor a wide range of problems. Yet recent work has found an increasing variety of adversarial samplesthat can fool them. Most existing detection mechanisms against adversarial attacksimpose significant costs, either by using additional classifiers to spot adversarial samples, or by requiring the DNN to be restructured. In this paper, we introduce a novel defence. We train our DNN so that, as long as it is workingas intended on the kind of inputs we expect, its behavior is constrained, in that some set of behaviors are taboo. If it is exposed to adversarial samples, they will often cause a taboo behavior, which we can detect. Taboos can be both subtle and diverse, so their choice can encode and hide information. It is a well-established design principle that the security of a system should not depend on the obscurity of its design, but on some variable (the key) which can differ between implementations and bechanged as necessary. We discuss how taboos can be used to equip a classifier with just such a key, and how to tune the keying mechanism to adversaries of various capabilities. We evaluate the performance of a prototype against a wide range of attacks and show how our simple defense can defend against cheap attacks at scale with zero run-time computation overhead, making it a suitable defense method for IoT devices.

cs.LG↗
arXiv2019

Sitatapatra: Blocking the Transfer of Adversarial Samples

Convolutional Neural Networks (CNNs) are widely used to solve classification tasks in computer vision. However, they can be tricked into misclassifying specially crafted `adversarial' samples -- and samples built to trick one model often work alarmingly well against other models trained on the same task. In this paper we introduce Sitatapatra, a system designed to block the transfer of adversarial samples. It diversifies neural networks using a key, as in cryptography, and provides a mechanism for detecting attacks. What's more, when adversarial samples are detected they can typically be traced back to the individual device that was used to develop them. The run-time overheads are minimal permitting the use of Sitatapatra on constrained systems.

cs.LG↗
arXiv2019

Blackbox Attacks on Reinforcement Learning Agents Using Approximated Temporal Information

Recent research on reinforcement learning (RL) has suggested that trained agents are vulnerable to maliciously crafted adversarial samples. In this work, we show how such samples can be generalised from White-box and Grey-box attacks to a strong Black-box case, where the attacker has no knowledge of the agents, their training parameters and their training methods. We use sequence-to-sequence models to predict a single action or a sequence of future actions that a trained agent will make. First, we show our approximation model, based on time-series information from the agent, consistently predicts RL agents' future actions with high accuracy in a Black-box setup on a wide range of games and RL algorithms. Second, we find that although adversarial samples are transferable from the target model to our RL agents, they often outperform random Gaussian noise only marginally. This highlights a serious methodological deficiency in previous work on such agents; random jamming should have been taken as the baseline for evaluation. Third, we propose a novel use for adversarial samplesin Black-box attacks of RL agents: they can be used to trigger a trained agent to misbehave after a specific time delay. This appears to be a genuinely new type of attack. It potentially enables an attacker to use devices controlled by RL agents as time bombs.

cs.LG↗
arXiv2019

The gift of the gab: Are rental scammers skilled at the art of persuasion?

Rental scams are a type of advance fee fraud, in which the scammer tries to get a victim to pay a deposit to rent an apartment of which the scammer pretends to be the landlord. We specifically focused on fraudulent long-term rentals advertised in the UK on Craigslist. After a victim responds to the scammer's advertisement, the scammer attempts to persuade them to transfer money without having seen the property. We were interested in which persuasion techniques scammers use, and in assessing their skill at the art of persuasion. During a period of three weeks, we scraped 2112 letting advertisements, identified the fraudulent advertisements and had 44 conversations of around 4 or 5 emails each with the scammers. Our analysis indicates that Cialdini`s marketing-based social persuasion strategies, such as liking, appeal to authority, and the need for commitment and consistency are extensively implemented by rental scammers. Of Stajano and Wilson's scam-based persuasion strategies, an appeal to sympathy (i.e., kindness) and need for greed were commonly used. We identified two further social persuasion strategies: establishing credibility and removing objections. At a superficial level, rental scammers seem skilled at their job, because they mimic genuine landlords and use a range of effective persuasion techniques. However, when examining their emails more closely, we see they often use pre-scripted emails, their mimicry is often incompetent, and they have a lack of language skills and cultural knowledge that may tip people off. They appear to be the criminal equivalent of a boilerhouse sales operation, a modus operandi that has not previously been studied by cybercrime researchers.

cs.CY↗
arXiv2019

Hearing your touch: A new acoustic side channel on smartphones

We present the first acoustic side-channel attack that recovers what users type on the virtual keyboard of their touch-screen smartphone or tablet. When a user taps the screen with a finger, the tap generates a sound wave that propagates on the screen surface and in the air. We found the device's microphone(s) can recover this wave and "hear" the finger's touch, and the wave's distortions are characteristic of the tap's location on the screen. Hence, by recording audio through the built-in microphone(s), a malicious app can infer text as the user enters it on their device. We evaluate the effectiveness of the attack with 45 participants in a real-world environment on an Android tablet and an Android smartphone. For the tablet, we recover 61% of 200 4-digit PIN-codes within 20 attempts, even if the model is not trained with the victim's data. For the smartphone, we recover 9 words of size 7--13 letters with 50 attempts in a common side-channel attack benchmark. Our results suggest that it not always sufficient to rely on isolation mechanisms such as TrustZone to protect user input. We propose and discuss hardware, operating-system and application-level mechanisms to block this attack more effectively. Mobile devices may need a richer capability model, a more user-friendly notification system for sensor usage and a more thorough evaluation of the information leaked by the underlying hardware.

cs.CR↗
arXiv2019

Strong mixed-integer programming formulations for trained neural networks

We present an ideal mixed-integer programming (MIP) formulation for a rectified linear unit (ReLU) appearing in a trained neural network. Our formulation requires a single binary variable and no additional continuous variables beyond the input and output variables of the ReLU. We contrast it with an ideal "extended" formulation with a linear number of additional continuous variables, derived through standard techniques. An apparent drawback of our formulation is that it requires an exponential number of inequality constraints, but we provide a routine to separate the inequalities in linear time. We also prove that these exponentially-many constraints are facet-defining under mild conditions. Finally, we study network verification problems and observe that dynamically separating from the exponential inequalities 1) is much more computationally efficient and scalable than the extended formulation, 2) decreases the solve time of a state-of-the-art MIP solver by a factor of 7 on smaller instances, and 3) nearly matches the dual bounds of a state-of-the-art MIP solver on harder instances, after just a few rounds of separation and in orders of magnitude less time.

math.OC↗
arXiv2015

Maxwell's fluid model of magnetism

In 1861, Maxwell derived two of his equations of electromagnetism by modelling a magnetic line of force as a `molecular vortex' in a fluid-like medium. Later, in 1980, Berry and colleagues conducted experiments on a `phase vortex', a wave geometry in a fluid which is analogous to a magnetic line of force and also exhibits behaviour corresponding to the quantisation of magnetic flux. Here we unify these approaches by writing down a solution to the equations of motion for a compressible fluid which behaves in the same way as a magnetic line of force. We then revisit Maxwell's historical inspiration, namely Faraday's 1846 model of light as disturbances in lines of force. Using our unified model, we show that such disturbances resemble photons: they are polarised, absorbed discretely, obey Maxwell's full equations of electromagnetism to first order, and quantitatively reproduce the correlation that is observed in the Bell tests.

quant-ph↗
arXiv2014

Why bouncing droplets are a pretty good model of quantum mechanics

In 2005, Couder, Protiere, Fort and Badouad showed that oil droplets bouncing on a vibrating tray of oil can display nonlocal interactions reminiscent of the particle-wave associations in quantum mechanics; in particular they can move, attract, repel and orbit each other. Subsequent experimental work by Couder, Fort, Protiere, Eddi, Sultan, Moukhtar, Rossi, Molacek, Bush and Sbitnev has established that bouncing drops exhibit single-slit and double-slit diffraction, tunnelling, quantised energy levels, Anderson localisation and the creation/annihilation of droplet/bubble pairs. In this paper we explain why. We show first that the surface waves guiding the droplets are Lorentz covariant with the characteristic speed c of the surface waves; second, that pairs of bouncing droplets experience an inverse-square force of attraction or repulsion according to their relative phase, and an analogue of the magnetic force; third, that bouncing droplets are governed by an analogue of Schroedinger's equation where Planck's constant is replaced by an appropriate constant of the motion; and fourth, that orbiting droplet pairs exhibit spin-half symmetry and align antisymmetrically as in the Pauli exclusion principle. Our analysis explains the similarities between bouncing-droplet experiments and the behaviour of quantum-mechanical particles. It also enables us to highlight some differences, and to predict some surprising phenomena that can be tested in feasible experiments.

quant-ph↗
arXiv2013

Why quantum computing is hard - and quantum cryptography is not provably secure

Despite high hopes for quantum computation in the 1990s, progress in the past decade has been slow; we still cannot perform computation with more than about three qubits and are no closer to solving problems of real interest than a decade ago. Separately, recent experiments in fluid mechanics have demonstrated the emergence of a full range of quantum phenomena from completely classical motion. We present two specific hypotheses. First, Kuramoto theory may give a basis for geometrical thinking about entanglement. Second, we consider a recent soliton model of the electron, in which the quantum-mechanical wave function is a phase modulation of a carrier wave. Both models are consistent with one another and with observation. Both models suggest how entanglement and decoherence may be related to device geometry. Both models predict that it will be difficult to maintain phase coherence of more than three qubits in the plane, or four qubits in a three-dimensional structure. The soliton model also shows that the experimental work which appeared to demonstrate a violation of Bell's inequalities might not actually do so; regardless of whether it is a correct description of the world, it exposes a flaw in the logic of the Bell tests. Thus the case for the security of EPR-based quantum cryptography has just not been made. We propose experiments in quantum computation to test this. Finally, we examine two possible interpretations of such soliton models: one is consistent with the transactional interpretation of quantum mechanics, while the other is an entirely classical model in which we do not have to abandon the idea of a single world where action is local and causal.

quant-ph↗
arXiv2012

Chip and Skim: cloning EMV cards with the pre-play attack

EMV, also known as "Chip and PIN", is the leading system for card payments worldwide. It is used throughout Europe and much of Asia, and is starting to be introduced in North America too. Payment cards contain a chip so they can execute an authentication protocol. This protocol requires point-of-sale (POS) terminals or ATMs to generate a nonce, called the unpredictable number, for each transaction to ensure it is fresh. We have discovered that some EMV implementers have merely used counters, timestamps or home-grown algorithms to supply this number. This exposes them to a "pre-play" attack which is indistinguishable from card cloning from the standpoint of the logs available to the card-issuing bank, and can be carried out even if it is impossible to clone a card physically (in the sense of extracting the key material and loading it into another card). Card cloning is the very type of fraud that EMV was supposed to prevent. We describe how we detected the vulnerability, a survey methodology we developed to chart the scope of the weakness, evidence from ATM and terminal experiments in the field, and our implementation of proof-of-concept attacks. We found flaws in widely-used ATMs from the largest manufacturers. We can now explain at least some of the increasing number of frauds in which victims are refused refunds by banks which claim that EMV cards cannot be cloned and that a customer involved in a dispute must therefore be mistaken or complicit. Pre-play attacks may also be carried out by malware in an ATM or POS terminal, or by a man-in-the-middle between the terminal and the acquirer. We explore the design and implementation mistakes that enabled the flaw to evade detection until now: shortcomings of the EMV specification, of the EMV kernel certification process, of implementation testing, formal analysis, or monitoring customer complaints. Finally we discuss countermeasures.

cs.CY↗