Searcharxiv⌕ Search

arXiv subjects

Xiao Yang

Publications and source records attributed to Xiao Yang.

At least 181 records · Page 10Linked to original sources
arXiv2023

How Robust is Google's Bard to Adversarial Image Attacks?

Multimodal Large Language Models (MLLMs) that integrate text and other modalities (especially vision) have achieved unprecedented performance in various multimodal tasks. However, due to the unsolved adversarial robustness problem of vision models, MLLMs can have more severe safety and security risks by introducing the vision inputs. In this work, we study the adversarial robustness of Google's Bard, a competitive chatbot to ChatGPT that released its multimodal capability recently, to better understand the vulnerabilities of commercial MLLMs. By attacking white-box surrogate vision encoders or MLLMs, the generated adversarial examples can mislead Bard to output wrong image descriptions with a 22% success rate based solely on the transferability. We show that the adversarial examples can also attack other MLLMs, e.g., a 26% attack success rate against Bing Chat and a 86% attack success rate against ERNIE bot. Moreover, we identify two defense mechanisms of Bard, including face detection and toxicity detection of images. We design corresponding attacks to evade these defenses, demonstrating that the current defenses of Bard are also vulnerable. We hope this work can deepen our understanding on the robustness of MLLMs and facilitate future research on defenses. Our code is available at https://github.com/thu-ml/Attack-Bard. Update: GPT-4V is available at October 2023. We further evaluate its robustness under the same set of adversarial examples, achieving a 45% attack success rate.

cs.CV↗
arXiv2023

An unprecedented synergy of high-temperature tensile strength and ductility in a NiCoCrAlTi high-entropy alloy

The present work reported a novel L12-strengthening NiCoCrAlTi high entropy alloy (HEA) with an outstanding synergy of tensile strength and ductility at both ambient and high temperatures. Transmission electron microscopy (TEM) characterization revealed a high density of rod-like and spheroidal L12 precipitates distributing in the micro/nanograins and non-recrystallized regions in the annealed specimens. The tremendously high yield stress, ultimate tensile stress (UTS), and ductility of the HEA at 600 C were ~1060 MPa, 1271 MPa, and 25%, respectively, which were significantly superior to most reported HEAs and Co- and Ni-based superalloys to date. Systematic TEM analysis unveiled that the cooperation among L12 precipitation, extensive stacking faults (SFs), deformation twins (DTs), immobile Lomer-Cottrell (L-C) locks formed from interactions between SFs and SFs/DTs, hierarchical SFs/DTs networks, as well as hetero-deformation-induced strengthening dominated the plastic deformation at 600 C. Such a unique deformation mechanism enabled extremely high tensile strength and sustained ductility of the HEA at a high temperature.

cond-mat.mtrl-sci↗
arXiv2023

Knowledge-Driven Multi-Agent Reinforcement Learning for Computation Offloading in Cybertwin-Enabled Internet of Vehicles

By offloading computation-intensive tasks of vehicles to roadside units (RSUs), mobile edge computing (MEC) in the Internet of Vehicles (IoV) can relieve the onboard computation burden. However, existing model-based task offloading methods suffer from heavy computational complexity with the increase of vehicles and data-driven methods lack interpretability. To address these challenges, in this paper, we propose a knowledge-driven multi-agent reinforcement learning (KMARL) approach to reduce the latency of task offloading in cybertwin-enabled IoV. Specifically, in the considered scenario, the cybertwin serves as a communication agent for each vehicle to exchange information and make offloading decisions in the virtual space. To reduce the latency of task offloading, a KMARL approach is proposed to select the optimal offloading option for each vehicle, where graph neural networks are employed by leveraging domain knowledge concerning graph-structure communication topology and permutation invariance into neural networks. Numerical results show that our proposed KMARL yields higher rewards and demonstrates improved scalability compared with other methods, benefitting from the integration of domain knowledge.

cs.LG↗
arXiv2023

Transition metal single-atom anchored on MoSi2N4 monolayer as highly efficient electrocatalyst for hydrogen evolution reaction

Single-atom catalysts are considered as a promising method for efficient energy conversion, owing to their advantages of high atom utilization and low catalyst cost. However, finding a stable two-dimensional structure and high hydrogen evolution reaction (HER) performance is a current research hotspot. Herein, based on the first-principles calculations, we identify the HER properties of six catalysts (TM@MoSi2N4, TM = Sc, Ti, V, Fe, Co, and Ni) comprising transition metal atoms anchored on MoSi2N4 monolayer. The results show that the spin-polarized states appear around the Fermi level after anchoring TM atoms. Therefore, the energy level of the first available unoccupied state for accommodating hydrogen drops, regulating the bonding strength of hydrogen. Thus, the single transition metal atom activates the active site of the MoSi2N4 inert base plane, becoming a quite suitable site for the HER. Based on ΔGH*, the exchange current density and volcano diagram of the corresponding catalytic system were also calculated. Among them, V@MoSi2N4 (ΔGH* = -0.07 eV) and Ni@MoSi2N4 (ΔGH* = 0.06 eV) systems show efficient the HER property. Our study confirms that the transition metal atom anchoring is an effective means to improve the performance of electrocatalysis, and TM@MoSi2N4 has practical application potential as a high efficiency HER electrocatalyst.

cond-mat.mtrl-sci↗
arXiv2023

Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videos

This work focuses on the 3D reconstruction of non-rigid objects based on monocular RGB video sequences. Concretely, we aim at building high-fidelity models for generic object categories and casually captured scenes. To this end, we do not assume known root poses of objects, and do not utilize category-specific templates or dense pose priors. The key idea of our method, Root Pose Decomposition (RPD), is to maintain a per-frame root pose transformation, meanwhile building a dense field with local transformations to rectify the root pose. The optimization of local transformations is performed by point registration to the canonical space. We also adapt RPD to multi-object scenarios with object occlusions and individual differences. As a result, RPD allows non-rigid 3D reconstruction for complicated scenarios containing objects with large deformations, complex motion patterns, occlusions, and scale diversities of different individuals. Such a pipeline potentially scales to diverse sets of objects in the wild. We experimentally show that RPD surpasses state-of-the-art methods on the challenging DAVIS, OVIS, and AMA datasets.

cs.CV↗
arXiv2023

Change ratios of magnetic helicity and magnetic free energy during major solar flares

Magnetic helicity is an important concept in solar physics, with a number of theoretical statements pointing out the important role of magnetic helicity in solar flares and coronal mass ejections (CMEs). Here we construct a sample of 47 solar flares, which contains 18 no-CME-associated confined flares and 29 CME-associated eruptive flares. We calculate the change ratios of magnetic helicity and magnetic free energy before and after these 47 flares. Our calculations show that the change ratios of magnetic helicity and magnetic free energy show distinct different distributions in confined flares and eruptive flares. The median value of the change ratios of magnetic helicity in confined flares is $-0.8$%, while this number is $-14.5$% for eruptive flares. For the magnetic free energy, the median value of the change ratios is $-4.3$% for confined flares, whereas this number is $-14.6$% for eruptive flares. This statistical result, using observational data, is well consistent with the theoretical understandings that magnetic helicity is approximately conserved in the magnetic reconnection, as shown by confined flares, and the CMEs take away magnetic helicity from the corona, as shown by eruptive flares.

astro-ph.SR↗
arXiv2023

AdvFAS: A robust face anti-spoofing framework against adversarial examples

Ensuring the reliability of face recognition systems against presentation attacks necessitates the deployment of face anti-spoofing techniques. Despite considerable advancements in this domain, the ability of even the most state-of-the-art methods to defend against adversarial examples remains elusive. While several adversarial defense strategies have been proposed, they typically suffer from constrained practicability due to inevitable trade-offs between universality, effectiveness, and efficiency. To overcome these challenges, we thoroughly delve into the coupled relationship between adversarial detection and face anti-spoofing. Based on this, we propose a robust face anti-spoofing framework, namely AdvFAS, that leverages two coupled scores to accurately distinguish between correctly detected and wrongly detected face images. Extensive experiments demonstrate the effectiveness of our framework in a variety of settings, including different attacks, datasets, and backbones, meanwhile enjoying high accuracy on clean examples. Moreover, we successfully apply the proposed method to detect real-world adversarial examples.

cs.CV↗
arXiv2023

Improving Opinion-based Question Answering Systems Through Label Error Detection and Overwrite

Label error is a ubiquitous problem in annotated data. Large amounts of label error substantially degrades the quality of deep learning models. Existing methods to tackle the label error problem largely focus on the classification task, and either rely on task specific architecture or require non-trivial additional computations, which is undesirable or even unattainable for industry usage. In this paper, we propose LEDO: a model-agnostic and computationally efficient framework for Label Error Detection and Overwrite. LEDO is based on Monte Carlo Dropout combined with uncertainty metrics, and can be easily generalized to multiple tasks and data sets. Applying LEDO to an industry opinion-based question answering system demonstrates it is effective at improving accuracy in all the core models. Specifically, LEDO brings 1.1% MRR gain for the retrieval model, 1.5% PR AUC improvement for the machine reading comprehension model, and 0.9% rise in the Average Precision for the ranker, on top of the strong baselines with a large-scale social media dataset. Importantly, LEDO is computationally efficient compared to methods that require loss function change, and cost-effective as the resulting data can be used in the same continuous training pipeline for production. Further analysis shows that these gains come from an improved decision boundary after cleaning the label errors existed in the training data.

cs.CL↗
arXiv2023

Unifying gradient regularization for Heterogeneous Graph Neural Networks

Heterogeneous Graph Neural Networks (HGNNs) are a class of powerful deep learning methods widely used to learn representations of heterogeneous graphs. Despite the fast development of HGNNs, they still face some challenges such as over-smoothing, and non-robustness. Previous studies have shown that these problems can be reduced by using gradient regularization methods. However, the existing gradient regularization methods focus on either graph topology or node features. There is no universal approach to integrate these features, which severely affects the efficiency of regularization. In addition, the inclusion of gradient regularization into HGNNs sometimes leads to some problems, such as an unstable training process, increased complexity and insufficient coverage regularized information. Furthermore, there is still short of a complete theoretical analysis of the effects of gradient regularization on HGNNs. In this paper, we propose a novel gradient regularization method called Grug, which iteratively applies regularization to the gradients generated by both propagated messages and the node features during the message-passing process. Grug provides a unified framework integrating graph topology and node features, based on which we conduct a detailed theoretical analysis of their effectiveness. Specifically, the theoretical analyses elaborate the advantages of Grug: 1) Decreasing sample variance during the training process (Stability); 2) Enhancing the generalization of the model (Universality); 3) Reducing the complexity of the model (Simplicity); 4) Improving the integrity and diversity of graph information utilization (Diversity). As a result, Grug has the potential to surpass the theoretical upper bounds set by DropMessage (AAAI-23 Distinguished Papers). In addition, we evaluate Grug on five public real-world datasets with two downstream tasks...

cs.LG↗
arXiv2023

Built-in electric field and strain tunable valley-related multiple topological phase transitions in VSiXN$_4$ (X= C, Si, Ge, Sn, Pb) monolayers

The valley-related multiple topological phase transitions attracted significant attention due to their providing significant opportunities for fundamental research and practical applications. However, unfortunately, to date there is no real material that can realize valley-related multiple topological phase transitions. Here, through first-principles calculations and model analysis, we investigate the structural, magnetic, electronic, and topological properties of VSiXN$_4$ (X = C, Si, Ge, Sn, Pb) monolayers. VSiXN$_4$ monolayers are stable and intrinsically ferrovalley materials. Intriguingly, we found that the built-in electric field and strain can induce valley-related multiple topological phase transitions in the materials from valley semiconductor to valley-half-semimetal, to valley quantum anomalous Hall insulator, to valley-half-semimetal, and to valley semiconductor (or to valley-metal). The nature of topological phase transition is the built-in electric field and strain induce band inversion between the d$_{xy}$/d$_{x2-y2}$ and d$_{z2}$ at obritals at K and K' valleys. Our findings not only reveal the mechanism of multiple topological phase transitions, but also provides an ideal platform for the multi-field manipulating the spin, valley, and topological physics. It will open new perspectives for spintronic, valleytronic, and topological nanoelectronic applications based on these materials.

cond-mat.mes-hall↗
arXiv2023

SAM for Poultry Science

In recent years, the agricultural industry has witnessed significant advancements in artificial intelligence (AI), particularly with the development of large-scale foundational models. Among these foundation models, the Segment Anything Model (SAM), introduced by Meta AI Research, stands out as a groundbreaking solution for object segmentation tasks. While SAM has shown success in various agricultural applications, its potential in the poultry industry, specifically in the context of cage-free hens, remains relatively unexplored. This study aims to assess the zero-shot segmentation performance of SAM on representative chicken segmentation tasks, including part-based segmentation and the use of infrared thermal images, and to explore chicken-tracking tasks by using SAM as a segmentation tool. The results demonstrate SAM's superior performance compared to SegFormer and SETR in both whole and part-based chicken segmentation. SAM-based object tracking also provides valuable data on the behavior and movement patterns of broiler birds. The findings of this study contribute to a better understanding of SAM's potential in poultry science and lay the foundation for future advancements in chicken segmentation and tracking.

cs.CV↗
arXiv2023

Solving a class of zero-sum stopping game with regime switching

This paper studies a class of zero-sum stopping game in a regime switching model. A verification theorem as a sufficient criterion for Nash equilibriums is established based on a set of variational inequalities (VIs). Under an appropriate regularity condition for solutions to the VIs, a suitable system of algebraic equations is derived via the so-called smooth-fit principle. Explicit Nash equilibrium stopping rules of threshold-type for the two players and the corresponding value function of the game in closed form are obtained. Numerical experiments are reported to demonstrate the dependence of the threshold levels on various model parameters. A reduction to the case with no regime switching is also presented as a comparison.

math.OC↗
arXiv2023

Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognition

Face recognition is a prevailing authentication solution in numerous biometric applications. Physical adversarial attacks, as an important surrogate, can identify the weaknesses of face recognition systems and evaluate their robustness before deployed. However, most existing physical attacks are either detectable readily or ineffective against commercial recognition systems. The goal of this work is to develop a more reliable technique that can carry out an end-to-end evaluation of adversarial robustness for commercial systems. It requires that this technique can simultaneously deceive black-box recognition models and evade defensive mechanisms. To fulfill this, we design adversarial textured 3D meshes (AT3D) with an elaborate topology on a human face, which can be 3D-printed and pasted on the attacker's face to evade the defenses. However, the mesh-based optimization regime calculates gradients in high-dimensional mesh space, and can be trapped into local optima with unsatisfactory transferability. To deviate from the mesh-based space, we propose to perturb the low-dimensional coefficient space based on 3D Morphable Model, which significantly improves black-box transferability meanwhile enjoying faster search efficiency and better visual quality. Extensive experiments in digital and physical scenarios show that our method effectively explores the security vulnerabilities of multiple popular commercial services, including three recognition APIs, four anti-spoofing APIs, two prevailing mobile phones and two automated access control systems.

cs.CV↗
arXiv2023

PAniC-3D: Stylized Single-view 3D Reconstruction from Portraits of Anime Characters

We propose PAniC-3D, a system to reconstruct stylized 3D character heads directly from illustrated (p)ortraits of (ani)me (c)haracters. Our anime-style domain poses unique challenges to single-view reconstruction; compared to natural images of human heads, character portrait illustrations have hair and accessories with more complex and diverse geometry, and are shaded with non-photorealistic contour lines. In addition, there is a lack of both 3D model and portrait illustration data suitable to train and evaluate this ambiguous stylized reconstruction task. Facing these challenges, our proposed PAniC-3D architecture crosses the illustration-to-3D domain gap with a line-filling model, and represents sophisticated geometries with a volumetric radiance field. We train our system with two large new datasets (11.2k Vroid 3D models, 1k Vtuber portrait illustrations), and evaluate on a novel AnimeRecon benchmark of illustration-to-3D pairs. PAniC-3D significantly outperforms baseline methods, and provides data to establish the task of stylized reconstruction from portrait illustrations.

cs.CV↗
arXiv2023

Shifted Diffusion for Text-to-image Generation

We present Corgi, a novel method for text-to-image generation. Corgi is based on our proposed shifted diffusion model, which achieves better image embedding generation from input text. Unlike the baseline diffusion model used in DALL-E 2, our method seamlessly encodes prior knowledge of the pre-trained CLIP model in its diffusion process by designing a new initialization distribution and a new transition step of the diffusion. Compared to the strong DALL-E 2 baseline, our method performs better in generating image embedding from the text in terms of both efficiency and effectiveness, resulting in better text-to-image generation. Extensive large-scale experiments are conducted and evaluated in terms of both quantitative measures and human evaluation, indicating a stronger generation ability of our method compared to existing ones. Furthermore, our model enables semi-supervised and language-free training for text-to-image generation, where only part or none of the images in the training dataset have an associated caption. Trained with only 1.7% of the images being captioned, our semi-supervised model obtains FID results comparable to DALL-E 2 on zero-shot text-to-image generation evaluated on MS-COCO. Corgi also achieves new state-of-the-art results across different datasets on downstream language-free text-to-image generation tasks, outperforming the previous method, Lafite, by a large margin.

cs.CV↗
arXiv2023

Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving

3D object detection is an important task in autonomous driving to perceive the surroundings. Despite the excellent performance, the existing 3D detectors lack the robustness to real-world corruptions caused by adverse weathers, sensor noises, etc., provoking concerns about the safety and reliability of autonomous driving systems. To comprehensively and rigorously benchmark the corruption robustness of 3D detectors, in this paper we design 27 types of common corruptions for both LiDAR and camera inputs considering real-world driving scenarios. By synthesizing these corruptions on public datasets, we establish three corruption robustness benchmarks -- KITTI-C, nuScenes-C, and Waymo-C. Then, we conduct large-scale experiments on 24 diverse 3D object detection models to evaluate their corruption robustness. Based on the evaluation results, we draw several important findings, including: 1) motion-level corruptions are the most threatening ones that lead to significant performance drop of all models; 2) LiDAR-camera fusion models demonstrate better robustness; 3) camera-only models are extremely vulnerable to image corruptions, showing the indispensability of LiDAR point clouds. We release the benchmarks and codes at https://github.com/kkkcx/3D_Corruptions_AD. We hope that our benchmarks and findings can provide insights for future research on developing robust 3D object detection models.

cs.CV↗
arXiv2023

The Solar Upper Transition Region Imager (SUTRI) onboard the SATech-01 satellite

The Solar Upper Transition Region Imager (SUTRI) onboard the Space Advanced Technology demonstration satellite (SATech-01), which was launched to a sun-synchronous orbit at a height of 500 km in July 2022, aims to test the on-orbit performance of our newly developed Sc-Si multi-layer reflecting mirror and the 2kx2k EUV CMOS imaging camera and to take full-disk solar images at the Ne VII 46.5 nm spectral line with a filter width of 3 nm. SUTRI employs a Ritchey-Chretien optical system with an aperture of 18 cm. The on-orbit observations show that SUTRI images have a field of view of 41.6'x41.6' and a moderate spatial resolution of 8" without an image stabilization system. The normal cadence of SUTRI images is 30 s and the solar observation time is about 16 hours each day because the earth eclipse time accounts for about 1/3 of SATech-01's orbit period. Approximately 15 GB data is acquired each day and made available online after processing. SUTRI images are valuable as the Ne VII 46.5 nm line is formed at a temperature regime of 0.5 MK in the solar atmosphere, which has rarely been sampled by existing solar imagers. SUTRI observations will establish connections between structures in the lower solar atmosphere and corona, and advance our understanding of various types of solar activity such as flares, filament eruptions, coronal jets and coronal mass ejections.

astro-ph.SR↗
arXiv2023

A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking

The robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts, which becomes an important research problem in the development of deep learning. Although new deep learning methods and robustness improvement techniques have been constantly proposed, the robustness evaluations of existing methods are often inadequate due to their rapid development, diverse noise patterns, and simple evaluation metrics. Without thorough robustness evaluations, it is hard to understand the advances in the field and identify the effective methods. In this paper, we establish a comprehensive robustness benchmark called \textbf{ARES-Bench} on the image classification task. In our benchmark, we evaluate the robustness of 55 typical deep learning models on ImageNet with diverse architectures (e.g., CNNs, Transformers) and learning algorithms (e.g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets. Using robustness curves as the major evaluation criteria, we conduct large-scale experiments and draw several important findings, including: 1) there is an inherent trade-off between adversarial and natural robustness for the same model architecture; 2) adversarial training effectively improves adversarial robustness, especially when performed on Transformer architectures; 3) pre-training significantly improves natural robustness based on more training data or self-supervised learning. Based on ARES-Bench, we further analyze the training tricks in large-scale adversarial training on ImageNet. By designing the training settings accordingly, we achieve the new state-of-the-art adversarial robustness. We have made the benchmarking results and code platform publicly available.

cs.CV↗