arXiv subjects

Neeraj Suri

Publications and source records attributed to Neeraj Suri.

18 recordsLinked to original sources

arXiv2025

Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems

Large Language Models (LLMs) guardrail systems are designed to protect against prompt injection and jailbreak attacks. However, they remain vulnerable to evasion techniques. We demonstrate two approaches for bypassing LLM prompt injection and jailbreak detection systems via traditional character injection methods and algorithmic Adversarial Machine Learning (AML) evasion techniques. Through testing against six prominent protection systems, including Microsoft's Azure Prompt Shield and Meta's Prompt Guard, we show that both methods can be used to evade detection while maintaining adversarial utility achieving in some instances up to 100% evasion success. Furthermore, we demonstrate that adversaries can enhance Attack Success Rates (ASR) against black-box targets by leveraging word importance ranking computed by offline white-box models. Our findings reveal vulnerabilities within current LLM protection mechanisms and highlight the need for more robust guardrail systems.

cs.CR↗

arXiv2025

Beyond Security-by-design: Securing a compromised system

Digital infrastructures are seeing convergence and connectivity at unprecedented scale. This is true for both current critical national infrastructures and emerging future systems that are highly cyber-physical in nature with complex intersections between humans and technologies, e.g., smart cities, intelligent transportation, high-value manufacturing and Industry 4.0. Diverse legacy and non-legacy software systems underpinned by heterogeneous hardware compose on-the-fly to deliver services to millions of users with varying requirements and unpredictable actions. This complexity is compounded by intricate and complicated supply-chains with many digital assets and services outsourced to third parties. The reality is that, at any particular point in time, there will be untrusted, partially-trusted or compromised elements across the infrastructure. Given this reality, and the societal scale of digital infrastructures, delivering secure and resilient operations is a major challenge. We argue that this requires us to move beyond the paradigm of security-by-design and embrace the challenge of securing-a-compromised-system.

cs.CR↗

arXiv2024

Self-Supervised Representation Learning for Adversarial Attack Detection

Supervised learning-based adversarial attack detection methods rely on a large number of labeled data and suffer significant performance degradation when applying the trained model to new domains. In this paper, we propose a self-supervised representation learning framework for the adversarial attack detection task to address this drawback. Firstly, we map the pixels of augmented input images into an embedding space. Then, we employ the prototype-wise contrastive estimation loss to cluster prototypes as latent variables. Additionally, drawing inspiration from the concept of memory banks, we introduce a discrimination bank to distinguish and learn representations for each individual instance that shares the same or a similar prototype, establishing a connection between instances and their associated prototypes. We propose a parallel axial-attention (PAA)-based encoder to facilitate the training process by parallel training over height- and width-axis of attention maps. Experimental results show that, compared to various benchmark self-supervised vision learning models and supervised adversarial attack detection methods, the proposed model achieves state-of-the-art performance on the adversarial attack detection task across a wide range of images.

cs.CV↗

arXiv2024

UNICAD: A Unified Approach for Attack Detection, Noise Reduction and Novel Class Identification

As the use of Deep Neural Networks (DNNs) becomes pervasive, their vulnerability to adversarial attacks and limitations in handling unseen classes poses significant challenges. The state-of-the-art offers discrete solutions aimed to tackle individual issues covering specific adversarial attack scenarios, classification or evolving learning. However, real-world systems need to be able to detect and recover from a wide range of adversarial attacks without sacrificing classification accuracy and to flexibly act in {\bf unseen} scenarios. In this paper, UNICAD, is proposed as a novel framework that integrates a variety of techniques to provide an adaptive solution. For the targeted image classification, UNICAD achieves accurate image classification, detects unseen classes, and recovers from adversarial attacks using Prototype and Similarity-based DNNs with denoising autoencoders. Our experiments performed on the CIFAR-10 dataset highlight UNICAD's effectiveness in adversarial mitigation and unseen class classification, outperforming traditional models.

cs.CV↗

arXiv2024

Federated Adversarial Learning for Robust Autonomous Landing Runway Detection

As the development of deep learning techniques in autonomous landing systems continues to grow, one of the major challenges is trust and security in the face of possible adversarial attacks. In this paper, we propose a federated adversarial learning-based framework to detect landing runways using paired data comprising of clean local data and its adversarial version. Firstly, the local model is pre-trained on a large-scale lane detection dataset. Then, instead of exploiting large instance-adaptive models, we resort to a parameter-efficient fine-tuning method known as scale and shift deep features (SSF), upon the pre-trained model. Secondly, in each SSF layer, distributions of clean local data and its adversarial version are disentangled for accurate statistics estimation. To the best of our knowledge, this marks the first instance of federated learning work that address the adversarial sample problem in landing runway detection. Our experimental evaluations over both synthesis and real images of Landing Approach Runway Detection (LARD) dataset consistently demonstrate good performance of the proposed federated adversarial learning and robust to adversarial attacks.

cs.CV↗

arXiv2023

Error Propagation Analysis for Multithreaded Programs: An Empirical Approach

Fault injection is a technique to measure the robustness of a program to errors by introducing faults into the program under test. Following a fault injection experiment, Error Propagation Analysis (EPA) is deployed to understand how errors affect a program's execution. EPA typically compares the traces of a fault-free (golden) run with those from a faulty run of the program. While this suffices for deterministic programs, EPA approaches are unsound for multithreaded programs with non-deterministic golden runs. In this paper, we propose Invariant Propagation Analysis (IPA) as the use of automatically inferred likely invariants ("invariants" in the following) in lieu of golden traces for conducting EPA in multithreaded programs. We evaluate the stability and fault coverage of invariants derived by IPA through fault injection experiments across six different fault types and six representative programs that can be executed with varying numbers of threads. We find that stable invariants can be inferred in all cases, but their fault coverage depends on the application and the fault type. We also find that fault coverage for multithreaded executions with IPA can be even higher than for traditional singlethreaded EPA, which emphasizes that IPA results cannot be trivially extrapolated from traditional EPA results.

cs.SE↗

arXiv2023

Compilation as a Defense: Enhancing DL Model Attack Robustness via Tensor Optimization

Adversarial Machine Learning (AML) is a rapidly growing field of security research, with an often overlooked area being model attacks through side-channels. Previous works show such attacks to be serious threats, though little progress has been made on efficient remediation strategies that avoid costly model re-engineering. This work demonstrates a new defense against AML side-channel attacks using model compilation techniques, namely tensor optimization. We show relative model attack effectiveness decreases of up to 43% using tensor optimization, discuss the implications, and direction of future work.

cs.LG↗

arXiv2023

Model Leeching: An Extraction Attack Targeting LLMs

Model Leeching is a novel extraction attack targeting Large Language Models (LLMs), capable of distilling task-specific knowledge from a target LLM into a reduced parameter model. We demonstrate the effectiveness of our attack by extracting task capability from ChatGPT-3.5-Turbo, achieving 73% Exact Match (EM) similarity, and SQuAD EM and F1 accuracy scores of 75% and 87%, respectively for only $50 in API cost. We further demonstrate the feasibility of adversarial attack transferability from an extracted model extracted via Model Leeching to perform ML attack staging against a target LLM, resulting in an 11% increase to attack success rate when applied to ChatGPT-3.5-Turbo.

cs.LG↗

arXiv2023

Security Challenges when Space Merges with Cyberspace

Spaceborne systems, such as communication satellites, sensory, surveillance, GPS and a multitude of other functionalities, form an integral part of global ICT cyberinfrastructures. However, a focussed discourse highlighting the distinctive threats landscape of these spaceborne assets is conspicuous by its absence. This position paper specifically considers the interplay of Space and Cyberspace to highlight security challenges that warrant dedicated attention in securing these complex infrastructures. The opinion piece additionally adds summary opinions on (a) emerging technology trends and (b) advocacy on technological and policy issues needed to support security responsiveness and mitigation.

cs.CR↗

arXiv2023

Specifying Autonomous System Behaviour

Specifying the intended behaviour of autonomous systems is becoming increasingly important but is fraught with many challenges. This technical report provides an overview of existing work on specifications of autonomous systems and places a particular emphasis on formal specification, i.e. mathematically rigorous approaches to specification that require an appropriate formalism. Given the breadth of this domain, our coverage is necessarily incomplete but serves to provide a brief introduction to some of the difficulties that specifying autonomous systems entails, as well as existing approaches to addressing these difficulties currently pursued in academia and industry.

eess.SY↗

arXiv2023

PINCH: An Adversarial Extraction Attack Framework for Deep Learning Models

Adversarial extraction attacks constitute an insidious threat against Deep Learning (DL) models in-which an adversary aims to steal the architecture, parameters, and hyper-parameters of a targeted DL model. Existing extraction attack literature have observed varying levels of attack success for different DL models and datasets, yet the underlying cause(s) behind their susceptibility often remain unclear, and would help facilitate creating secure DL systems. In this paper we present PINCH: an efficient and automated extraction attack framework capable of designing, deploying, and analyzing extraction attack scenarios across heterogeneous hardware platforms. Using PINCH, we perform extensive experimental evaluation of extraction attacks against 21 model architectures to explore new extraction attack scenarios and further attack staging. Our findings show (1) key extraction characteristics whereby particular model configurations exhibit strong resilience against specific attacks, (2) even partial extraction success enables further staging for other adversarial attacks, and (3) equivalent stolen models uncover differences in expressive power, yet exhibit similar captured knowledge.

cs.CR↗

arXiv2022

Privacy-preserving Decentralized Federated Learning over Time-varying Communication Graph

Establishing how a set of learners can provide privacy-preserving federated learning in a fully decentralized (peer-to-peer, no coordinator) manner is an open problem. We propose the first privacy-preserving consensus-based algorithm for the distributed learners to achieve decentralized global model aggregation in an environment of high mobility, where the communication graph between the learners may vary between successive rounds of model aggregation. In particular, in each round of global model aggregation, the Metropolis-Hastings method is applied to update the weighted adjacency matrix based on the current communication topology. In addition, the Shamir's secret sharing scheme is integrated to facilitate privacy in reaching consensus of the global model. The paper establishes the correctness and privacy properties of the proposed algorithm. The computational efficiency is evaluated by a simulation built on a federated learning framework with a real-word dataset.

cs.CR↗

arXiv2022

ThreatPro: Multi-Layer Threat Analysis in the Cloud

Many effective Threat Analysis (TA) techniques exist that focus on analyzing threats to targeted assets (e.g., components, services). These techniques consider static interconnections among the assets. However, in dynamic environments, such as the Cloud, resources can instantiate, migrate across physical hosts, or decommission to provide rapid resource elasticity to the users. It is evident that existing TA techniques cannot address all these requirements. In addition, there is an increasing number of complex multi-layer/multi-asset attacks on Cloud systems, such as the Equifax data breach. Hence, there is a need for threat analysis approaches that are designed to analyze threats in complex, dynamic, and multi-layer Cloud environments. In this paper, we propose ThreatPro that addresses the analysis of multi-layer attacks and supports dynamic interconnections in the Cloud. ThreatPro facilitates threat analysis by developing a technology-agnostic information flow model, which represents the Cloud's functionality through a set of conditional transitions. The model establishes the basis to capture the multi-layer and dynamic interconnections during the life-cycle of a Virtual Machine (VM). Specifically, ThreatPro contributes in (a) enabling the exploration of a threat's behavior and its propagation across the Cloud, and (b) assessing the security of the Cloud by analyzing the impact of multiple threats across various operational layers/assets. Using public information on threats from the National Vulnerability Database (NVD), we validate ThreatPro's capabilities, i.e., (a) identify and trace actual Cloud attacks and (b) speculatively postulate alternate potential attack paths.

cs.CR↗

arXiv2020

A Security Architecture for Railway Signalling

We present the proposed security architecture Deutsche Bahn plans to deploy to protect its trackside safety-critical signalling system against cyber-attacks. We first present the existing reference interlocking system that is built using standard components. Next, we present a taxonomy to help model the attack vectors relevant for the railway environment. Building upon this, we present the proposed "compartmentalized" defence concept for securing the upcoming signalling systems.

cs.CR↗

arXiv2020

Safe Execution of Concurrent Programs by Enforcement of Scheduling Constraints

Automated software verification of concurrent programs is challenging because of exponentially large state spaces with respect to the number of threads and number of events per thread. Verification techniques such as model checking need to explore a large number of possible executions that are possible under a non-deterministic scheduler. State space reduction techniques such as partial order reduction simplify the verification problem, however, the reduced state space may still be exponentially large and intractable. This paper discusses \emph{Iteratively Relaxed Scheduling}, a framework that uses scheduling constraints in order to simplify the verification problem and enable automated verification of programs which could not be handled with fully non-deterministic scheduling. Program executions are safe as long as the same scheduling constraints are enforced under which the program has been verified, e.g., by instrumenting a program with additional synchronization. As strict enforcement of scheduling constraints may induce a high execution time overhead, we present optimizations over a naive solution that reduce this overhead. Our evaluation of a prototype implementation on well-known benchmark programs shows the effect of scheduling constraints on the execution time overhead and how this overhead can be reduced by relaxing and choosing constraints.

cs.PL↗

arXiv2018

QRES: Quantitative Reasoning on Encrypted Security SLAs

While regulators advocate for higher cloud transparency, many Cloud Service Providers (CSPs) often do not provide detailed information regarding their security implementations in their Service Level Agreements (SLAs). In practice, CSPs are hesitant to release detailed information regarding their security posture for security and proprietary reasons. This lack of transparency hinders the adoption of cloud computing by enterprises and individuals. Unless CSPs share information regarding the technical details of their security proceedings and standards, customers cannot verify which cloud provider matched their needs in terms of security and privacy guarantees. To address this problem, we propose QRES, the first system that enables (a) CSPs to disclose detailed information about their offered security services in an encrypted form to ensure data confidentiality, and (b) customers to assess the CSPs' offered security services and find those satisfying their security requirements. Our system preserves each party's privacy by leveraging a novel evaluation method based on Secure Two Party Computation (2PC) and Searchable Encryption techniques. We implement QRES and highlight its usefulness by applying it to existing standardized SLAs. The real world tests illustrate that the system runs in acceptable time for practical application even when used with a multitude of CSPs. We formally prove the security requirements of the proposed system against a strong realistic adversarial model, using an automated cryptographic protocol verifier.

cs.CR↗

arXiv2018

Scaling Out Acid Applications with Operation Partitioning

OLTP applications with high workloads that cannot be served by a single server need to scale out to multiple servers. Typically, scaling out entails assigning a different partition of the application state to each server. But data partitioning is at odds with preserving the strong consistency guarantees of ACID transactions, a fundamental building block of many OLTP applications. The more we scale out and spread data across multiple servers, the more frequent distributed transactions accessing data at different servers will be. With a large number of servers, the high cost of distributed transactions makes scaling out ineffective or even detrimental. In this paper we propose Operation Partitioning, a novel paradigm to scale out OLTP applications that require ACID guarantees. Operation Partitioning indirectly partitions data across servers by partitioning the application's operations through static analysis. This partitioning of operations yields to a lock-free Conveyor Belt protocol for distributed coordination, which can scale out unmodified applications running on top of unmodified database management systems. We implement the protocol in a system called Elia and use it to scale out two applications, TPC-W and RUBiS. Our experiments show that Elia can increase maximum throughput by up to 4.2x and reduce latency by up to 58.6x compared to MySQL Cluster while at the same time providing a stronger isolation guarantee (serializability instead of read committed).

cs.DC↗

arXiv2012

Proofs of Writing for Efficient and Robust Storage

We present PoWerStore, the first efficient robust storage protocol that achieves optimal latency without using digital signatures. PoWerStore's robustness comprises tolerating asynchrony, maximum number of Byzantine storage servers, any number of Byzantine readers and crash-faulty writers, and guaranteeing wait-freedom and linearizability of read/write operations. PoWerStore's efficiency stems from combining lightweight authentication, erasure coding and metadata write-backs where readers write-back only metadata to achieve linearizability. At the heart of PoWerStore are Proofs of Writing (PoW): a novel storage technique based on lightweight cryptography. PoW enable reads and writes in the single-writer variant of PoWerStore to have latency of 2 rounds of communication between a client and storage servers in the worst-case (which we show optimal). We further present and implement a multi-writer PoWerStore variant featuring 3-round writes/reads where the third read round is invoked only under active attacks, and show that it outperforms existing robust storage protocols, including crash-tolerant ones.

cs.DC↗