arXiv subjects

Nicolas Küchler

Publications and source records attributed to Nicolas Küchler.

7 recordsLinked to original sources

arXiv2026

Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation

For nearly a decade the academic community has investigated backdoors in neural networks, primarily focusing on classification tasks where adversaries manipulate the model prediction. While demonstrably malicious, the immediate real-world impact of such prediction-altering attacks has remained unclear. In this paper we introduce a novel and significantly more potent class of backdoors that builds upon recent advancements in architectural backdoors. We demonstrate how these backdoors can be specifically engineered to exploit batched inference, a common technique for hardware utilization, enabling large-scale user data manipulation and theft. By targeting the batching process, these architectural backdoors facilitate information leakage between concurrent user requests and allow attackers to fully control model responses directed at other users within the same batch. In other words, an attacker who can change the model architecture can set and steal model inputs and outputs of other users within the same batch. We show that such attacks are not only feasible but also alarmingly effective, can be readily injected into prevalent model architectures, (e.g. Transformers), and represent a truly malicious threat to user privacy and system integrity. Critically, to counteract this new class of vulnerabilities, we propose a deterministic mitigation strategy that provides formal guarantees against this new attack vector, unlike prior work that relied on LLMs to find the backdoors. Our mitigation strategy employs a novel Information Flow Control mechanism that analyzes the model graph and proves non-interference between different user inputs within the same batch. Using our mitigation strategy we perform a large scale analysis of models hosted through Hugging Face and find over 200 models that introduce (unintended) information leakage between batch entries due to the use of dynamic quantization.

cs.CR↗

arXiv2025

Artemis: Efficient Commit-and-Prove SNARKs for zkML

Ensuring that AI models are both verifiable and privacy-preserving is important for trust, accountability, and compliance. To address these concerns, recent research has focused on developing zero-knowledge machine learning (zkML) techniques that enable the verification of various aspects of ML models without revealing sensitive information. However, while recent zkML advances have made significant improvements to the efficiency of proving ML computations, they have largely overlooked the costly consistency checks on committed model parameters and input data, which have become a dominant performance bottleneck. To address this gap, this paper introduces a new Commit-and-Prove SNARK (CP-SNARK) construction, Artemis, that effectively addresses the emerging challenge of commitment verification in zkML pipelines. In contrast to existing approaches, Artemis is compatible with any homomorphic polynomial commitment, including those without trusted setup. We present the first implementation of this CP-SNARK, evaluate its performance on a diverse set of ML models, and show substantial improvements over existing methods, achieving significant reductions in prover costs and maintaining efficiency even for large-scale models. For example, for the VGG model, we reduce the overhead associated with commitment checks from 11.5x to 1.1x. Our results indicate that Artemis provides a concrete step toward practical deployment of zkML, particularly in settings involving large-scale or complex models.

cs.CR↗

arXiv2025

DPolicy: Managing Privacy Risks Across Multiple Releases with Differential Privacy

Differential Privacy (DP) has emerged as a robust framework for privacy-preserving data releases and has been successfully applied in high-profile cases, such as the 2020 US Census. However, in organizational settings, the use of DP remains largely confined to isolated data releases. This approach restricts the potential of DP to serve as a framework for comprehensive privacy risk management at an organizational level. Although one might expect that the cumulative privacy risk of isolated releases could be assessed using DP's compositional property, in practice, individual DP guarantees are frequently tailored to specific releases, making it difficult to reason about their interaction or combined impact. At the same time, less tailored DP guarantees, which compose more easily, also offer only limited insight because they lead to excessively large privacy budgets that convey limited meaning. To address these limitations, we present DPolicy, a system designed to manage cumulative privacy risks across multiple data releases using DP. Unlike traditional approaches that treat each release in isolation or rely on a single (global) DP guarantee, our system employs a flexible framework that considers multiple DP guarantees simultaneously, reflecting the diverse contexts and scopes typical of real-world DP deployments. DPolicy introduces a high-level policy language to formalize privacy guarantees, making traditionally implicit assumptions on scopes and contexts explicit. By deriving the DP guarantees required to enforce complex privacy semantics from these high-level policies, DPolicy enables fine-grained privacy risk management on an organizational scale. We implement and evaluate DPolicy, demonstrating how it mitigates privacy risks that can emerge without comprehensive, organization-wide privacy risk management.

cs.CR↗

arXiv2024

Holding Secrets Accountable: Auditing Privacy-Preserving Machine Learning

Recent advancements in privacy-preserving machine learning are paving the way to extend the benefits of ML to highly sensitive data that, until now, have been hard to utilize due to privacy concerns and regulatory constraints. Simultaneously, there is a growing emphasis on enhancing the transparency and accountability of machine learning, including the ability to audit ML deployments. While ML auditing and PPML have both been the subjects of intensive research, they have predominately been examined in isolation. However, their combination is becoming increasingly important. In this work, we introduce Arc, an MPC framework for auditing privacy-preserving machine learning. At the core of our framework is a new protocol for efficiently verifying MPC inputs against succinct commitments at scale. We evaluate the performance of our framework when instantiated with our consistency protocol and compare it to hashing-based and homomorphic-commitment-based approaches, demonstrating that it is up to 10^4x faster and up to 10^6x more concise.

cs.CR↗

arXiv2023

Cohere: Managing Differential Privacy in Large Scale Systems

The need for a privacy management layer in today's systems started to manifest with the emergence of new systems for privacy-preserving analytics and privacy compliance. As a result, many independent efforts have emerged that try to provide system support for privacy. Recently, the scope of privacy solutions used in systems has expanded to encompass more complex techniques such as Differential Privacy (DP). The use of these solutions in large-scale systems imposes new challenges and requirements. Careful planning and coordination are necessary to ensure that privacy guarantees are maintained across a wide range of heterogeneous applications and data systems. This requires new solutions for managing and allocating scarce and non-replenishable privacy resources. In this paper, we introduce Cohere, a new system that simplifies the use of DP in large-scale systems. Cohere implements a unified interface that allows heterogeneous applications to operate on a unified view of users' data. In this work, we further address two pressing system challenges that arise in the context of real-world deployments: ensuring the continuity of privacy-based applications (i.e., preventing privacy budget depletion) and effectively allocating scarce shared privacy resources (i.e., budget) under complex preferences. Our experiments show that Cohere achieves a 6.4--28x improvement in utility compared to the state-of-the-art across a range of complex workloads.

cs.CR↗

arXiv2023

RoFL: Robustness of Secure Federated Learning

Even though recent years have seen many attacks exposing severe vulnerabilities in Federated Learning (FL), a holistic understanding of what enables these attacks and how they can be mitigated effectively is still lacking. In this work, we demystify the inner workings of existing (targeted) attacks. We provide new insights into why these attacks are possible and why a definitive solution to FL robustness is challenging. We show that the need for ML algorithms to memorize tail data has significant implications for FL integrity. This phenomenon has largely been studied in the context of privacy; our analysis sheds light on its implications for ML integrity. We show that certain classes of severe attacks can be mitigated effectively by enforcing constraints such as norm bounds on clients' updates. We investigate how to efficiently incorporate these constraints into secure FL protocols in the single-server setting. Based on this, we propose RoFL, a new secure FL system that extends secure aggregation with privacy-preserving input validation. Specifically, RoFL can enforce constraints such as $L_2$ and $L_\infty$ bounds on high-dimensional encrypted model updates.

cs.CR↗

arXiv2021

Zeph: Cryptographic Enforcement of End-to-End Data Privacy

As increasingly more sensitive data is being collected to gain valuable insights, the need to natively integrate privacy controls in data analytics frameworks is growing in importance. Today, privacy controls are enforced by data curators with full access to data in the clear. However, a plethora of recent data breaches show that even widely trusted service providers can be compromised. Additionally, there is no assurance that data processing and handling comply with the claimed privacy policies. This motivates the need for a new approach to data privacy that can provide strong assurance and control to users. This paper presents Zeph, a system that enables users to set privacy preferences on how their data can be shared and processed. Zeph enforces privacy policies cryptographically and ensures that data available to third-party applications complies with users' privacy policies. Zeph executes privacy-adhering data transformations in real-time and scales to thousands of data sources, allowing it to support large-scale low-latency data stream analytics. We introduce a hybrid cryptographic protocol for privacy-adhering transformations of encrypted data. We develop a prototype of Zeph on Apache Kafka to demonstrate that Zeph can perform large-scale privacy transformations with low overhead.

cs.CR↗