arXiv subjects

Yong Zhang

Publications and source records attributed to Yong Zhang.

At least 415 records · Page 23Linked to original sources

arXiv2022

Prior-Guided Adversarial Initialization for Fast Adversarial Training

Fast adversarial training (FAT) effectively improves the efficiency of standard adversarial training (SAT). However, initial FAT encounters catastrophic overfitting, i.e.,the robust accuracy against adversarial attacks suddenly and dramatically decreases. Though several FAT variants spare no effort to prevent overfitting, they sacrifice much calculation cost. In this paper, we explore the difference between the training processes of SAT and FAT and observe that the attack success rate of adversarial examples (AEs) of FAT gets worse gradually in the late training stage, resulting in overfitting. The AEs are generated by the fast gradient sign method (FGSM) with a zero or random initialization. Based on the observation, we propose a prior-guided FGSM initialization method to avoid overfitting after investigating several initialization strategies, improving the quality of the AEs during the whole training process. The initialization is formed by leveraging historically generated AEs without additional calculation cost. We further provide a theoretical analysis for the proposed initialization method. We also propose a simple yet effective regularizer based on the prior-guided initialization,i.e., the currently generated perturbation should not deviate too much from the prior-guided initialization. The regularizer adopts both historical and current adversarial perturbations to guide the model learning. Evaluations on four datasets demonstrate that the proposed method can prevent catastrophic overfitting and outperform state-of-the-art FAT methods. The code is released at https://github.com/jiaxiaojunQAQ/FGSM-PGI.

cs.CV↗

arXiv2022

A phase transition driven by subtle distortion without broken symmetry on spin, charge and lattice in Layered LnCu4-δP2(Ln=Eu, Sr)

In the scenario of Landau phase transition theory in condensed matter physics, any thermal dynamic phase transition must be subject to some kind of broken symmetries, that are relative to its spin, charge, orbital and lattice. Here we report a rare phase transition at Tp ~120 K or 140 K in layered materials LnCu4-δP2 (Ln=Eu, Sr) driven by a subtle structural-distortion without any broken symmetry on charge, spin and lattice. The variations of the lattice parameters, (ΔLc/Lc) ~ 0.013% or 0.062%, verified by thermal expansion, is much less than that for a typical crystalline phase transition (~0.5-1%), but the significant anomaly in heat capacity provides clear evidence of its intrinsic nature of thermodynamic transition.

cond-mat.str-el↗

arXiv2022

Robust Counterfactual Explanations on Graph Neural Networks

Massive deployment of Graph Neural Networks (GNNs) in high-stake applications generates a strong demand for explanations that are robust to noise and align well with human intuition. Most existing methods generate explanations by identifying a subgraph of an input graph that has a strong correlation with the prediction. These explanations are not robust to noise because independently optimizing the correlation for a single input can easily overfit noise. Moreover, they do not align well with human intuition because removing an identified subgraph from an input graph does not necessarily change the prediction result. In this paper, we propose a novel method to generate robust counterfactual explanations on GNNs by explicitly modelling the common decision logic of GNNs on similar input graphs. Our explanations are naturally robust to noise because they are produced from the common decision boundaries of a GNN that govern the predictions of many similar input graphs. The explanations also align well with human intuition because removing the set of edges identified by an explanation from the input graph changes the prediction significantly. Exhaustive experiments on many public datasets demonstrate the superior performance of our method.

cs.LG↗

arXiv2022

Multi-Camera View Based Proactive BS Selection and Beam Switching for V2X

Due to the short wavelength and large attenuation of millimeter-wave (mmWave), mmWave BSs are densely distributed and require beamforming with high directivity. When the user moves out of the coverage of the current BS or is severely blocked, the mmWave BS must be switched to ensure the communication quality. In this paper, we proposed a multi-camera view based proactive BS selection and beam switching that can predict the optimal BS of the user in the future frame and switch the corresponding beam pair. Specifically, we extract the features of multi-camera view images and a small part of channel state information (CSI) in historical frames, and dynamically adjust the weight of each modality feature. Then we design a multi-task learning module to guide the network to better understand the main task, thereby enhancing the accuracy and the robustness of BS selection and beam switching. Using the outputs of all tasks, a prior knowledge based fine tuning network is designed to further increase the BS switching accuracy. After the optimal BS is obtained, a beam pair switching network is proposed to directly predict the optimal beam pair of the corresponding BS. Simulation results in an outdoor intersection environment show the superior performance of our proposed solution under several metrics such as predicting accuracy, achievable rate, harmonic mean of precision and recall.

eess.SP↗

arXiv2022

Revealing Unfair Models by Mining Interpretable Evidence

The popularity of machine learning has increased the risk of unfair models getting deployed in high-stake applications, such as justice system, drug/vaccination design, and medical diagnosis. Although there are effective methods to train fair models from scratch, how to automatically reveal and explain the unfairness of a trained model remains a challenging task. Revealing unfairness of machine learning models in interpretable fashion is a critical step towards fair and trustworthy AI. In this paper, we systematically tackle the novel task of revealing unfair models by mining interpretable evidence (RUMIE). The key idea is to find solid evidence in the form of a group of data instances discriminated most by the model. To make the evidence interpretable, we also find a set of human-understandable key attributes and decision rules that characterize the discriminated data instances and distinguish them from the other non-discriminated data. As demonstrated by extensive experiments on many real-world data sets, our method finds highly interpretable and solid evidence to effectively reveal the unfairness of trained models. Moreover, it is much more scalable than all of the baseline methods.

cs.LG↗

arXiv2022

Global bifurcation structure and geometric properties for steady periodic water waves with vorticity

This paper studies the classical water wave problem with vorticity described by the Euler equations with a free surface under the influence of gravity over a flat bottom. Based on fundamental work \cite{ConstantinStrauss}, we first obtain two continuous bifurcation curves which meet the laminar flow only one time by using modified analytic bifurcation theorem. They are symmetric waves whose profiles are monotone between each crest and trough. Furthermore, we find that there is at least one inflection point on the wave profile between successive crests and troughs and the free surface is strictly concave at any crest and strictly convex at any trough. In addition, for favorable vorticity, we prove that the vertical displacement of water waves decreases with depth.

math.AP↗

arXiv2022

Self-Adaptive Real-Time Time-Dependent Density Functional Theory for X-ray Absorptions

Real-time time-dependent density functional theory (RT-TDDFT) can in principle access the whole absorption spectrum of a many-electron system exposed to a narrow pulse. However, this requires an accurate and efficient propagator for the numerical integration of the time-dependent Kohn-Sham equation. While a low-order time propagator is already sufficient for the low-lying valence absorption spectra, it is no longer the case for the X-ray absorption spectra (XAS) of systems composed even only of light elements, for which the use of a high-order propagator is indispensable. It is then crucial to choose a largest possible time step and a shortest possible simulation time, so as to minimize the computational cost. To this end, we propose here a robust AutoPST approach to determine automatically (Auto) the propagator (P), step (S), and time (T) for relativistic RT-TDDFT simulations of XAS.

physics.chem-ph↗

arXiv2022

ATOMS: ALMA Three-millimeter Observations of Massive Star-forming regions -- XI. From inflow to infall in hub-filament systems

We investigate the presence of hub-filament systems in a large sample of 146 active proto-clusters, using H$^{13}$CO$^{+}$ J=1-0 molecular line data obtained from the ATOMS survey. We find that filaments are ubiquitous in proto-clusters, and hub-filament systems are very common from dense core scales ($\sim$0.1 pc) to clump/cloud scales ($\sim$1-10 pc). The proportion of proto-clusters containing hub-filament systems decreases with increasing dust temperature ($T_d$) and luminosity-to-mass ratios ($L/M$) of clumps, indicating that stellar feedback from H{\sc ii} regions gradually destroys the hub-filament systems as proto-clusters evolve. Clear velocity gradients are seen along the longest filaments with a mean velocity gradient of 8.71 km s$^{-1}$pc$^{-1}$ and a median velocity gradient of 5.54 km s$^{-1}$pc$^{-1}$. We find that velocity gradients are small for filament lengths larger than $\sim$1~pc, probably hinting at the existence of inertial inflows, although we cannot determine whether the latter are driven by large-scale turbulence or large-scale gravitational contraction. In contrast, velocity gradients below $\sim$1~pc dramatically increase as filament lengths decrease, indicating that the gravity of the hubs or cores starts to dominate gas infall at small scales. We suggest that self-similar hub-filament systems and filamentary accretion at all scales may play a key role in high-mass star formation.

astro-ph.GA↗

arXiv2022

Boosting Fast Adversarial Training with Learnable Adversarial Initialization

Adversarial training (AT) has been demonstrated to be effective in improving model robustness by leveraging adversarial examples for training. However, most AT methods are in face of expensive time and computational cost for calculating gradients at multiple steps in generating adversarial examples. To boost training efficiency, fast gradient sign method (FGSM) is adopted in fast AT methods by calculating gradient only once. Unfortunately, the robustness is far from satisfactory. One reason may arise from the initialization fashion. Existing fast AT generally uses a random sample-agnostic initialization, which facilitates the efficiency yet hinders a further robustness improvement. Up to now, the initialization in fast AT is still not extensively explored. In this paper, we boost fast AT with a sample-dependent adversarial initialization, i.e., an output from a generative network conditioned on a benign image and its gradient information from the target network. As the generative network and the target network are optimized jointly in the training phase, the former can adaptively generate an effective initialization with respect to the latter, which motivates gradually improved robustness. Experimental evaluations on four benchmark databases demonstrate the superiority of our proposed method over state-of-the-art fast AT methods, as well as comparable robustness to advanced multi-step AT methods. The code is released at https://github.com//jiaxiaojunQAQ//FGSM-SDI.

cs.CV↗

arXiv2022

AuxMix: Semi-Supervised Learning with Unconstrained Unlabeled Data

Semi-supervised learning (SSL) has seen great strides when labeled data is scarce but unlabeled data is abundant. Critically, most recent work assume that such unlabeled data is drawn from the same distribution as the labeled data. In this work, we show that state-of-the-art SSL algorithms suffer a degradation in performance in the presence of unlabeled auxiliary data that does not necessarily possess the same class distribution as the labeled set. We term this problem as Auxiliary-SSL and propose AuxMix, an algorithm that leverages self-supervised learning tasks to learn generic features in order to mask auxiliary data that are not semantically similar to the labeled set. We also propose to regularize learning by maximizing the predicted entropy for dissimilar auxiliary samples. We show an improvement of 5% over existing baselines on a ResNet-50 model when trained on CIFAR10 dataset with 4k labeled samples and all unlabeled data is drawn from the Tiny-ImageNet dataset. We report competitive results on several datasets and conduct ablation studies.

cs.CV↗

arXiv2022

Deep Reinforcement Learning for Exact Combinatorial Optimization: Learning to Branch

Branch-and-bound is a systematic enumerative method for combinatorial optimization, where the performance highly relies on the variable selection strategy. State-of-the-art handcrafted heuristic strategies suffer from relatively slow inference time for each selection, while the current machine learning methods require a significant amount of labeled data. We propose a new approach for solving the data labeling and inference latency issues in combinatorial optimization based on the use of the reinforcement learning (RL) paradigm. We use imitation learning to bootstrap an RL agent and then use Proximal Policy Optimization (PPO) to further explore global optimal actions. Then, a value network is used to run Monte-Carlo tree search (MCTS) to enhance the policy network. We evaluate the performance of our method on four different categories of combinatorial optimization problems and show that our approach performs strongly compared to the state-of-the-art machine learning and heuristics based methods.

cs.LG↗

arXiv2022

Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection

Recent high-performing Human-Object Interaction (HOI) detection techniques have been highly influenced by Transformer-based object detector (i.e., DETR). Nevertheless, most of them directly map parametric interaction queries into a set of HOI predictions through vanilla Transformer in a one-stage manner. This leaves rich inter- or intra-interaction structure under-exploited. In this work, we design a novel Transformer-style HOI detector, i.e., Structure-aware Transformer over Interaction Proposals (STIP), for HOI detection. Such design decomposes the process of HOI set prediction into two subsequent phases, i.e., an interaction proposal generation is first performed, and then followed by transforming the non-parametric interaction proposals into HOI predictions via a structure-aware Transformer. The structure-aware Transformer upgrades vanilla Transformer by encoding additionally the holistically semantic structure among interaction proposals as well as the locally spatial structure of human/object within each interaction proposal, so as to strengthen HOI predictions. Extensive experiments conducted on V-COCO and HICO-DET benchmarks have demonstrated the effectiveness of STIP, and superior results are reported when comparing with the state-of-the-art HOI detectors. Source code is available at \url{https://github.com/zyong812/STIP}.

cs.CV↗

arXiv2022

Extending Momentum Contrast with Cross Similarity Consistency Regularization

Contrastive self-supervised representation learning methods maximize the similarity between the positive pairs, and at the same time tend to minimize the similarity between the negative pairs. However, in general the interplay between the negative pairs is ignored as they do not put in place special mechanisms to treat negative pairs differently according to their specific differences and similarities. In this paper, we present Extended Momentum Contrast (XMoCo), a self-supervised representation learning method founded upon the legacy of the momentum-encoder unit proposed in the MoCo family configurations. To this end, we introduce a cross consistency regularization loss, with which we extend the transformation consistency to dissimilar images (negative pairs). Under the cross consistency regularization rule, we argue that semantic representations associated with any pair of images (positive or negative) should preserve their cross-similarity under pretext transformations. Moreover, we further regularize the training loss by enforcing a uniform distribution of similarity over the negative pairs across a batch. The proposed regularization can easily be added to existing self-supervised learning algorithms in a plug-and-play fashion. Empirically, we report a competitive performance on the standard Imagenet-1K linear head classification benchmark. In addition, by transferring the learned representations to common downstream tasks, we show that using XMoCo with the prevalently utilized augmentations can lead to improvements in the performance of such tasks. We hope the findings of this paper serve as a motivation for researchers to take into consideration the important interplay among the negative examples in self-supervised learning.

cs.LG↗

arXiv2022

Spatial Cross-Attention Improves Self-Supervised Visual Representation Learning

Unsupervised representation learning methods like SwAV are proved to be effective in learning visual semantics of a target dataset. The main idea behind these methods is that different views of a same image represent the same semantics. In this paper, we further introduce an add-on module to facilitate the injection of the knowledge accounting for spatial cross correlations among the samples. This in turn results in distilling intra-class information including feature level locations and cross similarities between same-class instances. The proposed add-on can be added to existing methods such as the SwAV. We can later remove the add-on module for inference without any modification of the learned weights. Through an extensive set of empirical evaluations, we verify that our method yields an improved performance in detecting the class activation maps, top-1 classification accuracy, and down-stream tasks such as object detection, with different configuration settings.

cs.CV↗

arXiv2022

Radiative heat transfer in low-symmetry Bravais crystal

Over the last few years, broken symmetry within crystals has attracted extensive attention since it can improve the control of light propagation. In particular, low-symmetry Bravais crystal can support shear polaritons which has great potential in thermal photonics. In this work, we report a twist-induced near-field thermal control system based on the low-symmetry Bravais crystal medium (\b{eta}-Ga2O3). The near-field thermal radiation (NFTR) between such crystal slabs is nearly four orders of magnitude larger than the blackbody limit, exceeding the NFTR from other traditional dielectric materials. Moreover, we show that this crystal can serve as an excellent platform for twist-induced near-field thermal control. Due to the intrinsic shear effect, the twist-induced modulation supported by low-symmetry Bravais crystal exceeds that by high-symmetry crystal. We further clarify how the shear effect affects the twist-induced thermal-radiation modulation supported by hyperbolic and elliptical polaritons and show that the shear effect significantly enhances the twist-induced thermal control induced by the elliptical polariton mode. These results open new directions for thermal-radiation control in low-symmetry materials, including geological minerals, common oxides, and organic crystals.

physics.app-ph↗

arXiv2022

Fast Adversarial Training with Adaptive Step Size

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet. The key idea of recent works to accelerate adversarial training is to substitute multi-step attacks (e.g., PGD) with single-step attacks (e.g., FGSM). However, these single-step methods suffer from catastrophic overfitting, where the accuracy against PGD attack suddenly drops to nearly 0% during training, destroying the robustness of the networks. In this work, we study the phenomenon from the perspective of training instances. We show that catastrophic overfitting is instance-dependent and fitting instances with larger gradient norm is more likely to cause catastrophic overfitting. Based on our findings, we propose a simple but effective method, Adversarial Training with Adaptive Step size (ATAS). ATAS learns an instancewise adaptive step size that is inversely proportional to its gradient norm. The theoretical analysis shows that ATAS converges faster than the commonly adopted non-adaptive counterparts. Empirically, ATAS consistently mitigates catastrophic overfitting and achieves higher robust accuracy on CIFAR10, CIFAR100 and ImageNet when evaluated on various adversarial budgets.

cs.LG↗

arXiv2022

A Real-time Critical-scenario-generation Framework for Testing Autonomous Driving System

In order to find the most likely failure scenarios which may occur under certain given operation domain, critical-scenario-based test is supposed as an effective and widely used method, which gives suggestions for designers to improve the developing algorithm. However, for the state of art, critical-scenario generation approaches commonly utilize random-search or reinforcement learning methods to generate series of scenarios for a specific algorithm, which takes amounts of computing resource for testing a developing target that is always changing, and inapplicable for testing a real-time system. In this paper, we proposed a real-time critical-scenario-generation (RTCSG) framework to address the above challenges. In our framework, an aggressive-driving algorithm is proposed in controlling the virtual agent vehicles, a specially designed cost function is presented to guide scenarios to evolve towards critical conditions, and a self-adaptive coefficient iteration is designed that enable the approach to operate successfully in different conditions. With our proposed method, the critical-scenarios can be directly generated for the target under test which is a black-box system, and the real-time critical-scenario test can be brought into reality. The simulation results show that our approach is able to obtain more critical scenarios in most conditions than current methods, with a higher stability of success. For a real-time testing, our approach improves the efficiency around 16 times.

cs.RO↗

arXiv2022

Improving Fairness for Data Valuation in Horizontal Federated Learning

Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired properties for data valuation. However, there are still factors of potential unfairness in the design of federated Shapley value because two data owners with the same local data may not receive the same evaluation. We propose a new measure called completed federated Shapley value to improve the fairness of federated Shapley value. The design depends on completing a matrix consisting of all the possible contributions by different subsets of the data owners. It is shown under mild conditions that this matrix is approximately low-rank by leveraging concepts and tools from optimization. Both theoretical analysis and empirical evaluation verify that the proposed measure does improve fairness in many circumstances.

cs.LG↗