arXiv subjects

Dong Yuan

Publications and source records attributed to Dong Yuan.

54 records · Page 3Linked to original sources

arXiv2023

Honest Score Client Selection Scheme: Preventing Federated Learning Label Flipping Attacks in Non-IID Scenarios

Federated Learning (FL) is a promising technology that enables multiple actors to build a joint model without sharing their raw data. The distributed nature makes FL vulnerable to various poisoning attacks, including model poisoning attacks and data poisoning attacks. Today, many byzantine-resilient FL methods have been introduced to mitigate the model poisoning attack, while the effectiveness when defending against data poisoning attacks still remains unclear. In this paper, we focus on the most representative data poisoning attack - "label flipping attack" and monitor its effectiveness when attacking the existing FL methods. The results show that the existing FL methods perform similarly in Independent and identically distributed (IID) settings but fail to maintain the model robustness in Non-IID settings. To mitigate the weaknesses of existing FL methods in Non-IID scenarios, we introduce the Honest Score Client Selection (HSCS) scheme and the corresponding HSCSFL framework. In the HSCSFL, The server collects a clean dataset for evaluation. Under each iteration, the server collects the gradients from clients and then perform HSCS to select aggregation candidates. The server first evaluates the performance of each class of the global model and generates the corresponding risk vector to indicate which class could be potentially attacked. Similarly, the server evaluates the client's model and records the performance of each class as the accuracy vector. The dot product of each client's accuracy vector and global risk vector is generated as the client's host score; only the top p\% host score clients are included in the following aggregation. Finally, server aggregates the gradients and uses the outcome to update the global model. The comprehensive experimental results show our HSCSFL effectively enhances the FL robustness and defends against the "label flipping attack."

cs.CR↗

arXiv2023

Handling Heavy Occlusion in Dense Crowd Tracking by Focusing on the Heads

With the rapid development of deep learning, object detection and tracking play a vital role in today's society. Being able to identify and track all the pedestrians in the dense crowd scene with computer vision approaches is a typical challenge in this field, also known as the Multiple Object Tracking (MOT) challenge. Modern trackers are required to operate on more and more complicated scenes. According to the MOT20 challenge result, the pedestrian is 4 times denser than the MOT17 challenge. Hence, improving the ability to detect and track in extremely crowded scenes is the aim of this work. In light of the occlusion issue with the human body, the heads are usually easier to identify. In this work, we have designed a joint head and body detector in an anchor-free style to boost the detection recall and precision performance of pedestrians in both small and medium sizes. Innovatively, our model does not require information on the statistical head-body ratio for common pedestrians detection for training. Instead, the proposed model learns the ratio dynamically. To verify the effectiveness of the proposed model, we evaluate the model with extensive experiments on different datasets, including MOT20, Crowdhuman, and HT21 datasets. As a result, our proposed method significantly improves both the recall and precision rate on small & medium sized pedestrians and achieves state-of-the-art results in these challenging datasets.

cs.CV↗

arXiv2023

Exact Quantum Many-Body Scars in Higher-Spin Kinetically Constrained Models

We discover a variety of exact quantum many-body scars in higher-spin kinetically constrained models, through the recently developed DMRG-S algorithm [Zhang et al., Phys. Rev. Lett. 131, 020402]. Specifically, for the higher-spin PXP model on arbitrary bipartite lattices of any spatial dimension, we find exact many-body scars that are equidistantly spaced in the energy spectrum and exhibit similar structures to the ground state of the Affleck-Kennedy-Lieb-Tasaki model. For the one-dimensional Fermi-Hubbard model with a tilted potential in a certain parameter regime, whose effective model is equivalent to a kinetically constrained spin model with four degrees of freedom on each site, we find several many-body scars at energy $E=0$ and $E=\pm \sqrt{2}$ that can be exactly represented as matrix product states with finite bond dimensions. Our results demonstrate that larger local degrees of freedom in the kinetically constrained models provide a much broader space for the emergence of quantum many-body scars and weak ergodicity breaking.

cond-mat.str-el↗

arXiv2023

Extracting Quantum Many-Body Scarred Eigenstates with Matrix Product States

Quantum many-body scarred systems host nonthermal excited eigenstates immersed in a sea of thermal ones. In cases where exact expressions for these special eigenstates are not known, it is computationally demanding to distinguish them from their exponentially many thermal neighbors. We propose a matrix-product-state (MPS) algorithm, dubbed DMRG-S, to extract such states at system sizes far beyond the scope of exact diagonalization. Using this technique, we obtain scarred eigenstates in Rydberg-blockaded chains of up to 80 sites and perform a finite-size scaling study to address the lingering question of the stability for the Néel state revivals in the thermodynamic limit. Our method also provides a systematic way to obtain exact MPS representations for scarred eigenstates near the target energy without a priori knowledge. In particular, we find several new scarred eigenstates with exact MPS representations in kinetically constrained spin and clock models. The combination of numerical and analytical investigations in our work provides a new methodology for future studies of quantum many-body scars.

cond-mat.str-el↗

arXiv2023

FedMAE: Federated Self-Supervised Learning with One-Block Masked Auto-Encoder

Latest federated learning (FL) methods started to focus on how to use unlabeled data in clients for training due to users' privacy concerns, high labeling costs, or lack of expertise. However, current Federated Semi-Supervised/Self-Supervised Learning (FSSL) approaches fail to learn large-scale images because of the limited computing resources of local clients. In this paper, we introduce a new framework FedMAE, which stands for Federated Masked AutoEncoder, to address the problem of how to utilize unlabeled large-scale images for FL. Specifically, FedMAE can pre-train one-block Masked AutoEncoder (MAE) using large images in lightweight client devices, and then cascades multiple pre-trained one-block MAEs in the server to build a multi-block ViT backbone for downstream tasks. Theoretical analysis and experimental results on image reconstruction and classification show that our FedMAE achieves superior performance compared to the state-of-the-art FSSL methods.

cs.LG↗

arXiv2023

FedIL: Federated Incremental Learning from Decentralized Unlabeled Data with Convergence Analysis

Most existing federated learning methods assume that clients have fully labeled data to train on, while in reality, it is hard for the clients to get task-specific labels due to users' privacy concerns, high labeling costs, or lack of expertise. This work considers the server with a small labeled dataset and intends to use unlabeled data in multiple clients for semi-supervised learning. We propose a new framework with a generalized model, Federated Incremental Learning (FedIL), to address the problem of how to utilize labeled data in the server and unlabeled data in clients separately in the scenario of Federated Learning (FL). FedIL uses the Iterative Similarity Fusion to enforce the server-client consistency on the predictions of unlabeled data and uses incremental confidence to establish a credible pseudo-label set in each client. We show that FedIL will accelerate model convergence by Cosine Similarity with normalization, proved by Banach Fixed Point Theorem. The code is available at https://anonymous.4open.science/r/fedil.

cs.LG↗

arXiv2023

Random Padding Data Augmentation

The convolutional neural network (CNN) learns the same object in different positions in images, which can improve the recognition accuracy of the model. An implication of this is that CNN may know where the object is. The usefulness of the features' spatial information in CNNs has not been well investigated. In this paper, we found that the model's learning of features' position information hindered the learning of the features' relationship. Therefore, we introduced Random Padding, a new type of padding method for training CNNs that impairs the architecture's capacity to learn position information by adding zero-padding randomly to half of the border of feature maps. Random Padding is parameter-free, simple to construct, and compatible with the majority of CNN-based recognition models. This technique is also complementary to data augmentations such as random cropping, rotation, flipping and erasing, and consistently improves the performance of image classification over strong baselines.

cs.CV↗

arXiv2022

Enhancing Quantum Adversarial Robustness by Randomized Encodings

The interplay between quantum physics and machine learning gives rise to the emergent frontier of quantum machine learning, where advanced quantum learning models may outperform their classical counterparts in solving certain challenging problems. However, quantum learning systems are vulnerable to adversarial attacks: adding tiny carefully-crafted perturbations on legitimate input samples can cause misclassifications. To address this issue, we propose a general scheme to protect quantum learning systems from adversarial attacks by randomly encoding the legitimate data samples through unitary or quantum error correction encoders. In particular, we rigorously prove that both global and local random unitary encoders lead to exponentially vanishing gradients (i.e. barren plateaus) for any variational quantum circuits that aim to add adversarial perturbations, independent of the input data and the inner structures of adversarial circuits and quantum classifiers. In addition, we prove a rigorous bound on the vulnerability of quantum classifiers under local unitary adversarial attacks. We show that random black-box quantum error correction encoders can protect quantum classifiers against local adversarial noises and their robustness increases as we concatenate error correction codes. To quantify the robustness enhancement, we adapt quantum differential privacy as a measure of the prediction stability for quantum classifiers. Our results establish versatile defense strategies for quantum classifiers against adversarial perturbations, which provide valuable guidance to enhance the reliability and security for both near-term and future quantum learning technologies.

quant-ph↗

arXiv2022

Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networks

In this paper, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of O(1/T). In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, {we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum}. We also propose HierOPT, which optimizes the aggregation periods (worker-edge and edge-cloud aggregation periods) to minimize the loss given a limited training time.

cs.LG↗

arXiv2022

Federated Learning with Nesterov Accelerated Gradient

Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL (FedAvg) employs gradient descent algorithm, which may not be efficient enough. Momentum is able to improve the situation by adding an additional momentum step to accelerate the convergence and has demonstrated its benefits in both centralized and FL environments. It is well-known that Nesterov Accelerated Gradient (NAG) is a more advantageous form of momentum, but it is not clear how to quantify the benefits of NAG in FL so far. This motives us to propose FedNAG, which employs NAG in each worker as well as NAG momentum and model aggregation in the aggregator. We provide a detailed convergence analysis of FedNAG and compare it with FedAvg. Extensive experiments based on real-world datasets and trace-driven simulation are conducted, demonstrating that FedNAG increases the learning accuracy by 3-24% and decreases the total training time by 11-70% compared with the benchmarks under a wide range of settings.

cs.LG↗

arXiv2022

Quantum Information Scrambling in Quantum Many-body Scarred Systems

Quantum many-body scarred systems host special non-thermal eigenstates that support periodic revival dynamics and weakly break the ergodicity. Here, we study the quantum information scrambling dynamics in quantum many-body scarred systems, with a focus on the "PXP" model. We use the out-of-time-ordered correlator (OTOC) and Holevo information as measures of the information scrambling, and apply an efficient numerical method based on matrix product operators to compute them up to 41 spins. We find that both the OTOC and Holevo information exhibit a linear light cone and periodic oscillations inside the light cone for initial states within the scarred subspace, which is in sharp contrast to thermal or many-body localized systems. The periodic revivals of OTOCs and Holevo information signify unusual breakdown of quantum chaos and are not equivalent to the revival dynamics of state fidelity or local observables studied in the previous literature. To explain the formation of the linear light cone structure, we provide a perturbation-type calculation based on a phenomenological model. In addition, we demonstrate that the OTOC and Holevo information dynamics of the "PXP" model can be measured using the Rydberg-atom quantum simulators with current experimental technologies, and numerically identify the measurable signatures using experimental parameters.

quant-ph↗

arXiv2021

Fast Multi-label Learning

Embedding approaches have become one of the most pervasive techniques for multi-label classification. However, the training process of embedding methods usually involves a complex quadratic or semidefinite programming problem, or the model may even involve an NP-hard problem. Thus, such methods are prohibitive on large-scale applications. More importantly, much of the literature has already shown that the binary relevance (BR) method is usually good enough for some applications. Unfortunately, BR runs slowly due to its linear dependence on the size of the input data. The goal of this paper is to provide a simple method, yet with provable guarantees, which can achieve competitive performance without a complex training process. To achieve our goal, we provide a simple stochastic sketch strategy for multi-label classification and present theoretical results from both algorithmic and statistical learning perspectives. Our comprehensive empirical studies corroborate our theoretical findings and demonstrate the superiority of the proposed methods.

cs.LG↗

arXiv2021

Solving the Liouvillian Gap with Artificial Neural Networks

We propose a machine-learning inspired variational method to obtain the Liouvillian gap, which plays a crucial role in characterizing the relaxation time and dissipative phase transitions of open quantum systems. By using the "spin bi-base mapping", we map the density matrix to a pure restricted-Boltzmann-machine (RBM) state and transform the Liouvillian superoperator to a rank-two non-Hermitian operator. The Liouvillian gap can be obtained by a variational real-time evolution algorithm under this non-Hermitian operator. We apply our method to the dissipative Heisenberg model in both one and two dimensions. For the isotropic case, we find that the Liouvillian gap can be analytically obtained and in one dimension even the whole Liouvillian spectrum can be exactly solved using the Bethe ansatz method. By comparing our numerical results with their analytical counterparts, we show that the Liouvillian gap could be accessed by the RBM approach efficiently to a desirable accuracy, regardless of the dimensionality and entanglement properties.

quant-ph↗

arXiv2021

A Novel Graph-based Computation Offloading Strategy for Workflow Applications in Mobile Edge Computing

With the fast development of mobile edge computing (MEC), there is an increasing demand for running complex applications on the edge. These complex applications can be represented as workflows where task dependencies are explicitly specified. To achieve better Quality of Service (QoS), for instance, faster response time and lower energy consumption, computation offloading is widely used in the MEC environment. However, many existing computation offloading strategies only focus on independent computation tasks but overlook the task dependencies. Meanwhile, most of these strategies are based on search algorithms such as particle swarm optimization (PSO), genetic algorithm (GA) which are often time-consuming and hence not suitable for many delay-sensitive complex applications in MEC. Therefore, a highly efficient graph-based strategy was proposed in our recent work but it can only deal with simple workflow applications with linear (namely sequential) structure. For solving these problems, a novel graph-based strategy is proposed for workflow applications in MEC. Specifically, this strategy can deal with complex workflow applications with nonlinear (viz. parallel, selective and iterative) structures. Meanwhile, the offloading decision plan with the lowest energy consumption of the end-device under the deadline constraint can be found by using the graph-based partition technique. We have comprehensively evaluated our strategy using both a real-world case study on a MEC based UAV (Unmanned Aerial Vehicle) delivery system and extensive simulation experiments on the FogWorkflowSim platform for MEC based workflow applications. The evaluation results successfully demonstrate the effectiveness of our proposed strategy and its overall better performance than other representative strategies.

cs.DC↗

arXiv2020

A Fast and Scalable Authentication Scheme in IoT for Smart Living

Numerous resource-limited smart objects (SOs) such as sensors and actuators have been widely deployed in smart environments, opening new attack surfaces to intruders. The severe security flaw discourages the adoption of the Internet of things in smart living. In this paper, we leverage fog computing and microservice to push certificate authority (CA) functions to the proximity of data sources. Through which, we can minimize attack surfaces and authentication latency, and result in a fast and scalable scheme in authenticating a large volume of resource-limited devices. Then, we design lightweight protocols to implement the scheme, where both a high level of security and low computation workloads on SO (no bilinear pairing requirement on the client-side) is accomplished. Evaluations demonstrate the efficiency and effectiveness of our scheme in handling authentication and registration for a large number of nodes, meanwhile protecting them against various threats to smart living. Finally, we showcase the success of computing intelligence movement towards data sources in handling complicated services.

cs.CR↗

arXiv2020

ServiceNet: A P2P Service Network

Given a large number of online services on the Internet, from time to time, people are still struggling to find out the services that they need. On the other hand, when there are considerable research and development on service discovery and service recommendation, most of the related work are centralized and thus suffers inherent shortages of the centralized systems, e.g., adv-driven, lack at trust, transparence and fairness. In this paper, we propose a ServiceNet - a peer-to-peer (P2P) service network for service discovery and service recommendation. ServiceNet is inspired by blockchain technology and aims at providing an open, transparent and self-growth, and self-management service ecosystem. The paper will present the basic idea, an architecture design of the prototype, and an initial implementation and performance evaluation the prototype design.

cs.DC↗

arXiv2020

Online Metric Learning for Multi-Label Classification

Existing research into online multi-label classification, such as online sequential multi-label extreme learning machine (OSML-ELM) and stochastic gradient descent (SGD), has achieved promising performance. However, these works do not take label dependencies into consideration and lack a theoretical analysis of loss functions. Accordingly, we propose a novel online metric learning paradigm for multi-label classification to fill the current research gap. Generally, we first propose a new metric for multi-label classification which is based on $k$-Nearest Neighbour ($k$NN) and combined with large margin principle. Then, we adapt it to the online settting to derive our model which deals with massive volume ofstreaming data at a higher speed online. Specifically, in order to learn the new $k$NN-based metric, we first project instances in the training dataset into the label space, which make it possible for the comparisons of instances and labels in the same dimension. After that, we project both of them into a new lower dimension space simultaneously, which enables us to extract the structure of dependencies between instances and labels. Finally, we leverage the large margin and $k$NN principle to learn the metric with an efficient optimization algorithm. Moreover, we provide theoretical analysis on the upper bound of the cumulative loss for our method. Comprehensive experiments on a number of benchmark multi-label datasets validate our theoretical approach and illustrate that our proposed online metric learning (OML) algorithm outperforms state-of-the-art methods.

cs.LG↗

arXiv2016

A Cost-Effective Strategy for Storing Scientific Datasets with Multiple Service Providers in the Cloud

Cloud computing provides scientists a platform that can deploy computation and data intensive applications without infrastructure investment. With excessive cloud resources and a decision support system, large generated data sets can be flexibly 1 stored locally in the current cloud, 2 deleted and regenerated whenever reused or 3 transferred to cheaper cloud service for storage. However, due to the pay for use model, the total application cost largely depends on the usage of computation, storage and bandwidth resources, hence cutting the cost of cloud based data storage becomes a big concern for deploying scientific applications in the cloud. In this paper, we propose a novel strategy that can cost effectively store large generated data sets with multiple cloud service providers. The strategy is based on a novel algorithm that finds the trade off among computation, storage and bandwidth costs in the cloud, which are three key factors for the cost of data storage. Both general (random) simulations conducted with popular cloud service providers pricing models and three specific case studies on real world scientific applications show that the proposed storage strategy is highly cost effective and practical for run time utilization in the cloud.

cs.DC↗