arXiv subjects

Xu Yuan

Publications and source records attributed to Xu Yuan.

At least 55 records · Page 3Linked to original sources

arXiv2024

Towards Robust Vision Transformer via Masked Adaptive Ensemble

Adversarial training (AT) can help improve the robustness of Vision Transformers (ViT) against adversarial attacks by intentionally injecting adversarial examples into the training data. However, this way of adversarial injection inevitably incurs standard accuracy degradation to some extent, thereby calling for a trade-off between standard accuracy and robustness. Besides, the prominent AT solutions are still vulnerable to adaptive attacks. To tackle such shortcomings, this paper proposes a novel ViT architecture, including a detector and a classifier bridged by our newly developed adaptive ensemble. Specifically, we empirically discover that detecting adversarial examples can benefit from the Guided Backpropagation technique. Driven by this discovery, a novel Multi-head Self-Attention (MSA) mechanism is introduced to enhance our detector to sniff adversarial examples. Then, a classifier with two encoders is employed for extracting visual representations respectively from clean images and adversarial examples, with our adaptive ensemble to adaptively adjust the proportion of visual representations from the two encoders for accurate classification. This design enables our ViT architecture to achieve a better trade-off between standard accuracy and robustness. Besides, our adaptive ensemble technique allows us to mask off a random subset of image patches within input data, boosting our ViT's robustness against adaptive attacks, while maintaining high standard accuracy. Experimental results exhibit that our ViT architecture, on CIFAR-10, achieves the best standard accuracy and adversarial robustness of 90.3% and 49.8%, respectively.

cs.CV↗

arXiv2024

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering

Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative training of machine learning models over decentralized devices without exposing their local data. One of the major challenges in FL is the presence of uneven data distributions across client devices, violating the well-known assumption of independent-and-identically-distributed (IID) training samples in conventional machine learning. To address the performance degradation issue incurred by such data heterogeneity, clustered federated learning (CFL) shows its promise by grouping clients into separate learning clusters based on the similarity of their local data distributions. However, state-of-the-art CFL approaches require a large number of communication rounds to learn the distribution similarities during training until the formation of clusters is stabilized. Moreover, some of these algorithms heavily rely on a predefined number of clusters, thus limiting their flexibility and adaptability. In this paper, we propose {\em FedClust}, a novel approach for CFL that leverages the correlation between local model weights and the data distribution of clients. {\em FedClust} groups clients into clusters in a one-shot manner by measuring the similarity degrees among clients based on the strategically selected partial weights of locally trained models. We conduct extensive experiments on four benchmark datasets with different non-IID data settings. Experimental results demonstrate that {\em FedClust} achieves higher model accuracy up to $\sim$45\% as well as faster convergence with a significantly reduced communication cost up to 2.7$\times$ compared to its state-of-the-art counterparts.

cs.DC↗

arXiv2024

On the near soliton dynamics for the 2D cubic Zakharov-Kuznetsov equations

In this article, we consider the Cauchy problem for the cubic (mass-critical) Zakharov-Kuznetsov equations in dimension two: $$\partial_t u+\partial_{x_1}(Δu+u^3)=0,\quad (t,x)\in [0,\infty)\times \mathbb{R}^{2}.$$ For initial data in $H^1$ close to the soliton with a suitable space-decay property, we fully describe the asymptotic behavior of the corresponding solution. More precisely, for such initial data, we show that only three possible behaviors can occur: 1) The solution leaves a tube near soliton in finite time; 2) the solution blows up in finite time; 3) the solution is global and locally converges to a soliton. In addition, we show that for initial data near a soliton with non-positive energy and above the threshold mass, the corresponding solution will blow up as described in Case 2. Our proof is inspired by the techniques developed for mass-critical generalized Korteweg-de Vries equation (gKdV) equation in a similar context by Martel-Merle-Raphaël. More precisely, our proof relies on refined modulation estimates and a modified energy-virial Lyapunov functional. The primary challenge in our problem is the lack of coercivity of the Schrödinger operator which appears in the virial-type estimate. To overcome the difficulty, we apply a transform, which was first introduced in Kenig-Martel [13], to perform the virial computations after converting the original problem to the adjoint one. Th coercivity of the Schrödinger operator in the adjoint problem has been numerically verified by Farah-Holmer-Roudenko-Yang [9].

math.AP↗

arXiv2024

An Open and Large-Scale Dataset for Multi-Modal Climate Change-aware Crop Yield Predictions

Precise crop yield predictions are of national importance for ensuring food security and sustainable agricultural practices. While AI-for-science approaches have exhibited promising achievements in solving many scientific problems such as drug discovery, precipitation nowcasting, etc., the development of deep learning models for predicting crop yields is constantly hindered by the lack of an open and large-scale deep learning-ready dataset with multiple modalities to accommodate sufficient information. To remedy this, we introduce the CropNet dataset, the first terabyte-sized, publicly available, and multi-modal dataset specifically targeting climate change-aware crop yield predictions for the contiguous United States (U.S.) continent at the county level. Our CropNet dataset is composed of three modalities of data, i.e., Sentinel-2 Imagery, WRF-HRRR Computed Dataset, and USDA Crop Dataset, for over 2200 U.S. counties spanning 6 years (2017-2022), expected to facilitate researchers in developing versatile deep learning models for timely and precisely predicting crop yields at the county-level, by accounting for the effects of both short-term growing season weather variations and long-term climate change on crop yields. Besides, we develop the CropNet package, offering three types of APIs, for facilitating researchers in downloading the CropNet data on the fly over the time and region of interest, and flexibly building their deep learning models for accurate crop yield predictions. Extensive experiments have been conducted on our CropNet dataset via employing various types of deep learning solutions, with the results validating the general applicability and the efficacy of the CropNet dataset in climate change-aware crop yield predictions.

cs.LG↗

arXiv2024

Higher-order Structure Based Anomaly Detection on Attributed Networks

Anomaly detection (such as telecom fraud detection and medical image detection) has attracted the increasing attention of people. The complex interaction between multiple entities widely exists in the network, which can reflect specific human behavior patterns. Such patterns can be modeled by higher-order network structures, thus benefiting anomaly detection on attributed networks. However, due to the lack of an effective mechanism in most existing graph learning methods, these complex interaction patterns fail to be applied in detecting anomalies, hindering the progress of anomaly detection to some extent. In order to address the aforementioned issue, we present a higher-order structure based anomaly detection (GUIDE) method. We exploit attribute autoencoder and structure autoencoder to reconstruct node attributes and higher-order structures, respectively. Moreover, we design a graph attention layer to evaluate the significance of neighbors to nodes through their higher-order structure differences. Finally, we leverage node attribute and higher-order structure reconstruction errors to find anomalies. Extensive experiments on five real-world datasets (i.e., ACM, Citation, Cora, DBLP, and Pubmed) are implemented to verify the effectiveness of GUIDE. Experimental results in terms of ROC-AUC, PR-AUC, and Recall@K show that GUIDE significantly outperforms the state-of-art methods.

cs.LG↗

arXiv2024

Backdoor Federated Learning by Poisoning Backdoor-Critical Layers

Federated learning (FL) has been widely deployed to enable machine learning training on sensitive data across distributed devices. However, the decentralized learning paradigm and heterogeneity of FL further extend the attack surface for backdoor attacks. Existing FL attack and defense methodologies typically focus on the whole model. None of them recognizes the existence of backdoor-critical (BC) layers-a small subset of layers that dominate the model vulnerabilities. Attacking the BC layers achieves equivalent effects as attacking the whole model but at a far smaller chance of being detected by state-of-the-art (SOTA) defenses. This paper proposes a general in-situ approach that identifies and verifies BC layers from the perspective of attackers. Based on the identified BC layers, we carefully craft a new backdoor attack methodology that adaptively seeks a fundamental balance between attacking effects and stealthiness under various defense strategies. Extensive experiments show that our BC layer-aware backdoor attacks can successfully backdoor FL under seven SOTA defenses with only 10% malicious clients and outperform the latest backdoor attack methods.

cs.CR↗

arXiv2024

Quantum Vulnerability Analysis to Accurate Estimate the Quantum Algorithm Success Rate

While quantum computers provide exciting opportunities for information processing, they currently suffer from noise during computation that is not fully understood. Incomplete noise models have led to discrepancies between quantum program success rate (SR) estimates and actual machine outcomes. For example, the estimated probability of success (ESP) is the state-of-the-art metric used to gauge quantum program performance. The ESP suffers poor prediction since it fails to account for the unique combination of circuit structure, quantum state, and quantum computer properties specific to each program execution. Thus, an urgent need exists for a systematic approach that can elucidate various noise impacts and accurately and robustly predict quantum computer success rates, emphasizing application and device scaling. In this article, we propose quantum vulnerability analysis (QVA) to systematically quantify the error impact on quantum applications and address the gap between current success rate (SR) estimators and real quantum computer results. The QVA determines the cumulative quantum vulnerability (CQV) of the target quantum computation, which quantifies the quantum error impact based on the entire algorithm applied to the target quantum machine. By evaluating the CQV with well-known benchmarks on three 27-qubit quantum computers, the CQV success estimation outperforms the estimated probability of success state-of-the-art prediction technique by achieving on average six times less relative prediction error, with best cases at 30 times, for benchmarks with a real SR rate above 0.1%. Direct application of QVA has been provided that helps researchers choose a promising compiling strategy at compile time.

quant-ph↗

arXiv2024

FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering

Federated learning (FL) is an emerging distributed machine learning paradigm enabling collaborative model training on decentralized devices without exposing their local data. A key challenge in FL is the uneven data distribution across client devices, violating the well-known assumption of independent-and-identically-distributed (IID) training samples in conventional machine learning. Clustered federated learning (CFL) addresses this challenge by grouping clients based on the similarity of their data distributions. However, existing CFL approaches require a large number of communication rounds for stable cluster formation and rely on a predefined number of clusters, thus limiting their flexibility and adaptability. This paper proposes FedClust, a novel CFL approach leveraging correlations between local model weights and client data distributions. FedClust groups clients into clusters in a one-shot manner using strategically selected partial model weights and dynamically accommodates newcomers in real-time. Experimental results demonstrate FedClust outperforms baseline approaches in terms of accuracy and communication costs.

cs.DC↗

arXiv2023

MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer

Precise crop yield prediction provides valuable information for agricultural planning and decision-making processes. However, timely predicting crop yields remains challenging as crop growth is sensitive to growing season weather variation and climate change. In this work, we develop a deep learning-based solution, namely Multi-Modal Spatial-Temporal Vision Transformer (MMST-ViT), for predicting crop yields at the county level across the United States, by considering the effects of short-term meteorological variations during the growing season and the long-term climate change on crops. Specifically, our MMST-ViT consists of a Multi-Modal Transformer, a Spatial Transformer, and a Temporal Transformer. The Multi-Modal Transformer leverages both visual remote sensing data and short-term meteorological data for modeling the effect of growing season weather variations on crop growth. The Spatial Transformer learns the high-resolution spatial dependency among counties for accurate agricultural tracking. The Temporal Transformer captures the long-range temporal dependency for learning the impact of long-term climate change on crops. Meanwhile, we also devise a novel multi-modal contrastive learning technique to pre-train our model without extensive human supervision. Hence, our MMST-ViT captures the impacts of both short-term weather variations and long-term climate change on crops by leveraging both satellite images and meteorological data. We have conducted extensive experiments on over 200 counties in the United States, with the experimental results exhibiting that our MMST-ViT outperforms its counterparts under three performance metrics of interest.

cs.CV↗

arXiv2022

Generically sharp decay for quasilinear wave equations with null condition

We are interested in the three-dimensional quasilinear wave equations with null condition. Global existence and pointwise decay for this model have been proved in the celebrated works of Klainerman \cite{Klainerman86} and Christodoulou \cite{Christodoulou86} for small smooth initial data. In this work, we illustrate the precise pointwise asymptotic behavior of the solutions for initial data posed on a hyperboloid and show that the decay rate $v^{-1}u^{-1}$ is optimal for a generic set of initial data.

math.AP↗

arXiv2022

Global solution to the 3D Dirac--Klein-Gordon system with uniform energy bounds

On the (1+3) dimensional Minkowski spacetime, for small, regular initial data, it is well-known that the Dirac-Klein-Gordon system admits a global solution. In the present paper, we aim to establish the uniform boundedness of the total energy of the solution for this system. The proof relies on Klainerman's vector field and Alinhac's ghost weight methods. The main difficulty originates from the slow decay nature of the Dirac and wave components in three space dimensions. To overcome the difficulty, a sharp understanding of the structure for this system, and a new weighted conformal energy estimate are required. In addition, we also provide a few scattering results for the system.

math.AP↗

arXiv2022

Global Behavior of Small Data Solutions for The 2D Dirac-Klein-Gordon Equations

In this paper, we are interested in the two-dimensional Dirac-Klein-Gordon system, which is a basic model in particle physics. We investigate the global behaviors of small data solutions to this system in the case of a massive scalar field and a massless Dirac field. More precisely, our main result is twofold: 1) we show sharp time decay for the pointwise estimates of the solutions which imply the asymptotic stability of this system; 2) we show the linear scattering result of this system which is a fundamental problem when it is viewed as dispersive equations. Our result is valid for general small, high-regular initial data, in particular, there is no restriction on the support of the initial data.

math.AP↗

arXiv2022

Accelerating Serverless Computing by Harvesting Idle Resources

Serverless computing automates fine-grained resource scaling and simplifies the development and deployment of online services with stateless functions. However, it is still non-trivial for users to allocate appropriate resources due to various function types, dependencies, and input sizes. Misconfiguration of resource allocations leaves functions either under-provisioned or over-provisioned and leads to continuous low resource utilization. This paper presents Freyr, a new resource manager (RM) for serverless platforms that maximizes resource efficiency by dynamically harvesting idle resources from over-provisioned functions to under-provisioned functions. Freyr monitors each function's resource utilization in real-time, detects over-provisioning and under-provisioning, and learns to harvest idle resources safely and accelerates functions efficiently by applying deep reinforcement learning algorithms along with a safeguard mechanism. We have implemented and deployed a Freyr prototype in a 13-node Apache OpenWhisk cluster. Experimental results show that 38.8% of function invocations have idle resources harvested by Freyr, and 39.2% of invocations are accelerated by the harvested resources. Freyr reduces the 99th-percentile function response latency by 32.1% compared to the baseline RMs.

cs.DC↗

arXiv2022

Asymptotic behavior of 2D Wave-Klein-Gordon coupled system under null condition

We study the 2D coupled wave-Klein-Gordon systems with semi-linear null nonlinearities $Q_0$ and $Q_{αβ}$. The main result states that the solution to the 2D coupled systems exists globally provided that the initial data are small in some weighted Sobolev space, which do not necessarily have compact support, and we also show the optimal time decay of the solution. The major difficulties lie in the slow decay nature of the wave and the Klein-Gordon components in two space dimensions, in addition, extra difficulties arise due to the presence of the null form $Q_0$ which is not of divergence form and is not compatible with the Klein-Gordon equations. To overcome the difficulties, a new observation for the structure of the null form $Q_0$ is required.

math.AP↗

arXiv2021

Improving Sequential Recommendation Consistency with Self-Supervised Imitation

Most sequential recommendation models capture the features of consecutive items in a user-item interaction history. Though effective, their representation expressiveness is still hindered by the sparse learning signals. As a result, the sequential recommender is prone to make inconsistent predictions. In this paper, we propose a model, SSI, to improve sequential recommendation consistency with Self-Supervised Imitation. Precisely, we extract the consistency knowledge by utilizing three self-supervised pre-training tasks, where temporal consistency and persona consistency capture user-interaction dynamics in terms of the chronological order and persona sensitivities, respectively. Furthermore, to provide the model with a global perspective, global session consistency is introduced by maximizing the mutual information among global and local interaction sequences. Finally, to comprehensively take advantage of all three independent aspects of consistency-enhanced knowledge, we establish an integrated imitation learning framework. The consistency knowledge is effectively internalized and transferred to the student model by imitating the conventional prediction logit as well as the consistency-enhanced item representations. In addition, the flexible self-supervised imitation framework can also benefit other student recommenders. Experiments on four real-world datasets show that SSI effectively outperforms the state-of-the-art sequential recommendation methods.

cs.IR↗

arXiv2021

Construction of excited multi-solitons for the focusing 4D cubic wave equation

Consider the focusing 4D cubic wave equation \[ \partial_{tt}u-Δu-u^{3}=0,\quad \mbox{on}\ (t,x)\in [0,\infty)\times \mathbb{R}^{4}.\] The main result states the existence in energy space $\dot{H}^{1}\times L^{2}$ of multi-solitary waves where each traveling wave is generated by Lorentz transform from a specific excited state, with different but collinear Lorentz speeds. The specific excited state is deduced from the non-degenerate sign-changing state constructed in Musso-Wei [34]. The proof is inspired by the techniques developed for the 5D energy-critical wave equation and the nonlinear Klein-Gordon equation in a similar context by Martel-Merle [30] and Côte-Martel [6]. The main difficulty originates from the strong interactions between solutions in the 4D case compared to other dispersive and wave-type models. To overcome the difficulty, a sharp understanding of the asymptotic behavior of the excited states involved and of the kernel of their linearized operator is needed.

math.AP↗

arXiv2021

Asymptotics of solutions with a compactness property for the nonlinear damped Klein-Gordon equation

We consider the nonlinear damped Klein-Gordon equation \[ \partial_{tt}u+2α\partial_{t}u-Δu+u-|u|^{p-1}u=0 \quad \text{on} \ \ [0,\infty)\times \mathbb{R}^N \] with $α>0$, $2 \le N\le 5$ and energy subcritical exponents $p>2$. We study the behavior of solutions for which it is supposed that only one nonlinear object appears asymptotically for large times, at least for a sequence of times. We first prove that the nonlinear object is necessarily a bound state. Next, we show that when the nonlinear object is a non-degenerate state or a degenerate excited state satisfying a simplicity condition, the convergence holds for all positive times, with an exponential or algebraic rate respectively. Last, we provide an example where the solution converges exactly at the rate $t^{-1}$ to the excited state.

math.AP↗

arXiv2020

Construction of excited multi-solitons for the 5D energy-critical wave equation

For the 5D energy-critical wave equation, we construct excited $N$-solitons with collinear speeds, i.e. solutions $u$ of the equation such that \begin{equation*} \lim_{t\to+\infty}\bigg\|\nabla_{t,x}u(t)-\nabla_{t,x}\bigg(\sum_{n=1}^{N}Q_{n}(t)\bigg)\bigg\|_{L^{2}}=0, \end{equation*} where for $n=1,\ldots,N$, $Q_n(t,x)$ is the Lorentz transform of a non-degenerate and sufficiently decaying excited state, each with different but collinear speeds. The existence proof follows the ideas of Martel-Merle and Côte-Martel developed for the energy-critical wave and nonlinear Klein-Gordon equations. In particular, we rely on an energy method and on a general coercivity property for the linearized operator.

math.AP↗