arXiv subjects

Tian Dong

Publications and source records attributed to Tian Dong.

34 records · Page 2Linked to original sources

arXiv2024

Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning

Machine unlearning has become a promising solution for fulfilling the "right to be forgotten", under which individuals can request the deletion of their data from machine learning models. However, existing studies of machine unlearning mainly focus on the efficacy and efficiency of unlearning methods, while neglecting the investigation of the privacy vulnerability during the unlearning process. With two versions of a model available to an adversary, that is, the original model and the unlearned model, machine unlearning opens up a new attack surface. In this paper, we conduct the first investigation to understand the extent to which machine unlearning can leak the confidential content of the unlearned data. Specifically, under the Machine Learning as a Service setting, we propose unlearning inversion attacks that can reveal the feature and label information of an unlearned sample by only accessing the original and unlearned model. The effectiveness of the proposed unlearning inversion attacks is evaluated through extensive experiments on benchmark datasets across various model architectures and on both exact and approximate representative unlearning approaches. The experimental results indicate that the proposed attack can reveal the sensitive information of the unlearned data. As such, we identify three possible defenses that help to mitigate the proposed attacks, while at the cost of reducing the utility of the unlearned model. The study in this paper uncovers an underexplored gap between machine unlearning and the privacy of unlearned data, highlighting the need for the careful design of mechanisms for implementing unlearning without leaking the information of the unlearned data.

cs.CR↗

arXiv2023

Mate! Are You Really Aware? An Explainability-Guided Testing Framework for Robustness of Malware Detectors

Numerous open-source and commercial malware detectors are available. However, their efficacy is threatened by new adversarial attacks, whereby malware attempts to evade detection, e.g., by performing feature-space manipulation. In this work, we propose an explainability-guided and model-agnostic testing framework for robustness of malware detectors when confronted with adversarial attacks. The framework introduces the concept of Accrued Malicious Magnitude (AMM) to identify which malware features could be manipulated to maximize the likelihood of evading detection. We then use this framework to test several state-of-the-art malware detectors' abilities to detect manipulated malware. We find that (i) commercial antivirus engines are vulnerable to AMM-guided test cases; (ii) the ability of a manipulated malware generated using one detector to evade detection by another detector (i.e., transferability) depends on the overlap of features with large AMM values between the different detectors; and (iii) AMM values effectively measure the fragility of features (i.e., capability of feature-space manipulation to flip the prediction results) and explain the robustness of malware detectors facing evasion attacks. Our findings shed light on the limitations of current malware detectors, as well as how they can be improved.

cs.CR↗

arXiv2022

Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing

Transforming off-the-shelf deep neural network (DNN) models into dynamic multi-exit architectures can achieve inference and transmission efficiency by fragmenting and distributing a large DNN model in edge computing scenarios (e.g., edge devices and cloud servers). In this paper, we propose a novel backdoor attack specifically on the dynamic multi-exit DNN models. Particularly, we inject a backdoor by poisoning one DNN model's shallow hidden layers targeting not this vanilla DNN model but only its dynamically deployed multi-exit architectures. Our backdoored vanilla model behaves normally on performance and cannot be activated even with the correct trigger. However, the backdoor will be activated when the victims acquire this model and transform it into a dynamic multi-exit architecture at their deployment. We conduct extensive experiments to prove the effectiveness of our attack on three structures (ResNet-56, VGG-16, and MobileNet) with four datasets (CIFAR-10, SVHN, GTSRB, and Tiny-ImageNet) and our backdoor is stealthy to evade multiple state-of-the-art backdoor detection or removal methods.

cs.CR↗

arXiv2022

Privacy for Free: How does Dataset Condensation Help Privacy?

To prevent unintentional data leakage, research community has resorted to data generators that can produce differentially private data for model training. However, for the sake of the data privacy, existing solutions suffer from either expensive training cost or poor generalization performance. Therefore, we raise the question whether training efficiency and privacy can be achieved simultaneously. In this work, we for the first time identify that dataset condensation (DC) which is originally designed for improving training efficiency is also a better solution to replace the traditional data generators for private data generation, thus providing privacy for free. To demonstrate the privacy benefit of DC, we build a connection between DC and differential privacy, and theoretically prove on linear feature extractors (and then extended to non-linear feature extractors) that the existence of one sample has limited impact ($O(m/n)$) on the parameter distribution of networks trained on $m$ samples synthesized from $n (n \gg m)$ raw samples by DC. We also empirically validate the visual privacy and membership privacy of DC-synthesized data by launching both the loss-based and the state-of-the-art likelihood-based membership inference attacks. We envision this work as a milestone for data-efficient and privacy-preserving machine learning.

cs.CR↗

arXiv2022

An Interpretable Federated Learning-based Network Intrusion Detection Framework

Learning-based Network Intrusion Detection Systems (NIDSs) are widely deployed for defending various cyberattacks. Existing learning-based NIDS mainly uses Neural Network (NN) as a classifier that relies on the quality and quantity of cyberattack data. Such NN-based approaches are also hard to interpret for improving efficiency and scalability. In this paper, we design a new local-global computation paradigm, FEDFOREST, a novel learning-based NIDS by combining the interpretable Gradient Boosting Decision Tree (GBDT) and Federated Learning (FL) framework. Specifically, FEDFOREST is composed of multiple clients that extract local cyberattack data features for the server to train models and detect intrusions. A privacy-enhanced technology is also proposed in FEDFOREST to further defeat the privacy of the FL systems. Extensive experiments on 4 cyberattack datasets of different tasks demonstrate that FEDFOREST is effective, efficient, interpretable, and extendable. FEDFOREST ranks first in the collaborative learning and cybersecurity competition 2021 for Chinese college students.

cs.CR↗

arXiv2021

Fingerprinting Multi-exit Deep Neural Network Models via Inference Time

Transforming large deep neural network (DNN) models into the multi-exit architectures can overcome the overthinking issue and distribute a large DNN model on resource-constrained scenarios (e.g. IoT frontend devices and backend servers) for inference and transmission efficiency. Nevertheless, intellectual property (IP) protection for the multi-exit models in the wild is still an unsolved challenge. Previous efforts to verify DNN model ownership mainly rely on querying the model with specific samples and checking the responses, e.g., DNN watermarking and fingerprinting. However, they are vulnerable to adversarial settings such as adversarial training and are not suitable for the IP verification for multi-exit DNN models. In this paper, we propose a novel approach to fingerprint multi-exit models via inference time rather than inference predictions. Specifically, we design an effective method to generate a set of fingerprint samples to craft the inference process with a unique and robust inference time cost as the evidence for model ownership. We conduct extensive experiments to prove the uniqueness and robustness of our method on three structures (ResNet-56, VGG-16, and MobileNet) and three datasets (CIFAR-10, CIFAR-100, and Tiny-ImageNet) under comprehensive adversarial settings.

cs.CR↗

arXiv2021

Hidden Backdoors in Human-Centric Language Models

Natural language processing (NLP) systems have been proven to be vulnerable to backdoor attacks, whereby hidden features (backdoors) are trained into a language model and may only be activated by specific inputs (called triggers), to trick the model into producing unexpected behaviors. In this paper, we create covert and natural triggers for textual backdoor attacks, \textit{hidden backdoors}, where triggers can fool both modern language models and human inspection. We deploy our hidden backdoors through two state-of-the-art trigger embedding methods. The first approach via homograph replacement, embeds the trigger into deep neural networks through the visual spoofing of lookalike character replacement. The second approach uses subtle differences between text generated by language models and real natural text to produce trigger sentences with correct grammar and high fluency. We demonstrate that the proposed hidden backdoors can be effective across three downstream security-critical NLP tasks, representative of modern human-centric NLP systems, including toxic comment detection, neural machine translation (NMT), and question answering (QA). Our two hidden backdoor attacks can achieve an Attack Success Rate (ASR) of at least $97\%$ with an injection rate of only $3\%$ in toxic comment detection, $95.1\%$ ASR in NMT with less than $0.5\%$ injected data, and finally $91.12\%$ ASR against QA updated with only 27 poisoning data samples on a model previously trained with 92,024 samples (0.029\%). We are able to demonstrate the adversary's high success rate of attacks, while maintaining functionality for regular users, with triggers inconspicuous by the human administrators.

cs.CL↗

arXiv2019

Towards Generation and Evaluation of Comprehensive Mapping Robot Datasets

This paper presents a fully hardware synchronized mapping robot with support for a hardware synchronized external tracking system, for super-precise timing and localization. We also employ a professional, static 3D scanner for ground truth map collection. Three datasets are generated to evaluate the performance of mapping algorithms within a room and between rooms. Based on these datasets we generate maps and trajectory data, which is then fed into evaluation algorithms. The mapping and evaluation procedures are made in a very easily reproducible manner for maximum comparability. In the end we can draw a couple of conclusions about the tested SLAM algorithms.

cs.RO↗

arXiv2016

A New Result on Packing Unit Squares into a Large Square

In their 2009 note: \emph{Packing equal squares into a large square}, Chung and Graham proved that the uncovered area of a large square of side length $x$ is $O\left(x^{(3+\sqrt{2})/7}\log x\right)$ after maximum number of non-overlapping unit squares are packed into it, which improved the earlier results of Erdős-Graham, Roth-Vaughan, and Karabash-Soifer. Here we further improve the result to $O(x^{5/8})$ that also helps to improve the bound for the dual problem: finding the minimum number of unit squares needed for covering the large square, from $x^2+O\left(x^{(3+\sqrt{2})/7}\log x\right)$ to $x^2+O(x^{5/8})$.

math.CO↗

arXiv2014

A Two-Dimensional Improvement for Farr-Gao Algorithm

Farr-Gao algorithm is a state-of-the-art algorithm for reduced Gröbner bases of vanishing ideals of finite points, which has been implemented in Maple$^\circledR$ as a build-in command. In this paper, we present a two-dimensional improvement for it that employs a preprocessing strategy for computing reduced Gröbner bases associated with tower subsets of given point sets. Experimental results show that the preprocessed Farr-Gao algorithm is more efficient than the classical one.

math.AC↗

arXiv2011

Ideal Projectors of Type Partial Derivative and Their Perturbations

In this paper, we verify Carl de Boor's conjecture on ideal projectors for real ideal projectors of type partial derivative by proving that there exists a positive $η\in \mathbb{R}$ such that a real ideal projector of type partial derivative $P$ is the pointwise limit of a sequence of Lagrange projectors which are perturbed from $P$ up to $η$ in magnitude. Furthermore, we present an algorithm for computing the value of such $η$ when the range of the Lagrange projectors is spanned by the Gröbner éscalier of their kernels w.r.t. lexicographic order.

math.NA↗

arXiv2011

Finite Sets of Affine Points with Unique Associated Monomial Order Quotient Bases

The quotient bases for zero-dimensional ideals are often of interest in the investigation of multivariate polynomial interpolation, algebraic coding theory, and computational molecular biology, etc. In this paper, we discuss the properties of zero-dimensional ideals with unique monomial quotient bases, and verify that the vanishing ideals of Cartesian sets have unique monomial quotient bases. Furthermore, we reveal the relation between Cartesian sets and the point sets with unique associated monomial quotient bases.

math.AC↗

arXiv2011

On a C. de Boor's Conjecture in a Particular Case and Related Perturbation

In this paper, we focus on two classes of D-invariant polynomial subspaces. The first is a classical type, while the second is a new class. With matrix computation, we prove that every ideal projector with each D-invariant subspace belonging to either the first class or the second is the pointwise limit of Lagrange projectors. This verifies a particular case of a C. de Boor's conjecture asserting that every complex ideal projector is the pointwise limit of Lagrange projectors. Specifically, we provide the concrete perturbation procedure for ideal projectors of this type.

math.NA↗

arXiv2011

On existence of certain error formulas for a special class of ideal projectors

In this paper, we focus on a special class of ideal projectors. With the aid of algebraic geometry, we prove that for this special class of ideal projectors, there exist "good" error formulas as defined by C. de Boor. Furthermore, we completely analyze the properties of the interpolation conditions matched by this special class of ideal projectors, and show that the ranges of this special class of ideal projectors are the minimal degree interpolation spaces with regard to their associated interpolation conditions.

math.NA↗

arXiv2010

Bivariate Quasi-Tower Sets and Their Associated Lagrange Interpolation Bases

As we all known, there is still a long way for us to solve arbitrary multivariate Lagrange interpolation in theory. Nevertheless, it is well accepted that theories about Lagrange interpolation on special point sets should cast important lights on the general solution. In this paper, we propose a new type of bivariate point sets, quasi-tower sets, whose geometry is more natural than some known point sets such as cartesian sets and tower sets. For bivariate Lagrange interpolation on quasi-tower sets, we construct the associated degree reducing interpolation monomial and Newton bases w.r.t. common monomial orderings theoretically. Moreover, by inputting these bases into Buchberger-Möller algorithm, we obtain the reduced Gröbner bases for vanishing ideals of quasi-tower sets much more efficiently than before.

math.AC↗

arXiv2010

A Bivariate Preprocessing Paradigm for Buchberger-Möller Algorithm

For the last almost three decades, since the famous Buchberger-Möller(BM) algorithm emerged, there has been wide interest in vanishing ideals of points and associated interpolation polynomials. Our paradigm is based on the theory of bivariate polynomial interpolation on cartesian point sets that gives us related degree reducing interpolation monomial and Newton bases directly. Since the bases are involved in the computation process as well as contained in the final output of BM algorithm, our paradigm obviously simplifies the computation and accelerates the BM process. The experiments show that the paradigm is best suited for the computation over finite prime fields that have many applications.

math.AC↗