arXiv subjects

Xiangyu Zhang

Publications and source records attributed to Xiangyu Zhang.

At least 307 records · Page 17Linked to original sources

arXiv2022

SA4U: Practical Static Analysis for Unit Type Error Detection

Unit type errors, where values with physical unit types (e.g., meters, hours) are used incorrectly in a computation, are common in today's unmanned aerial system (UAS) firmware. Recent studies show that unit type errors represent over 10% of bugs in UAS firmware. Moreover, the consequences of unit type errors are severe. Over 30% of unit type errors cause UAS crashes. This paper proposes SA4U: a practical system for detecting unit type errors in real-world UAS firmware. SA4U requires no modifications to firmware or developer annotations. It deduces the unit types of program variables by analyzing simulation traces and protocol definitions. SA4U uses the deduced unit types to identify when unit type errors occur. SA4U is effective: it identified 14 previously undetected bugs in two popular open-source firmware (ArduPilot & PX4.)

cs.SE↗

arXiv2022

Self-Supervised Visual Representation Learning with Semantic Grouping

In this paper, we tackle the problem of learning visual representations from unlabeled scene-centric data. Existing works have demonstrated the potential of utilizing the underlying complex structure within scene-centric data; still, they commonly rely on hand-crafted objectness priors or specialized pretext tasks to build a learning framework, which may harm generalizability. Instead, we propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning. The semantic grouping is performed by assigning pixels to a set of learnable prototypes, which can adapt to each sample by attentive pooling over the feature and form new slots. Based on the learned data-dependent slots, a contrastive objective is employed for representation learning, which enhances the discriminability of features, and conversely facilitates grouping semantically coherent pixels together. Compared with previous efforts, by simultaneously optimizing the two coupled objectives of semantic grouping and contrastive learning, our approach bypasses the disadvantages of hand-crafted priors and is able to learn object/group-level representations from scene-centric images. Experiments show our approach effectively decomposes complex scenes into semantic groups for feature learning and significantly benefits downstream tasks, including object detection, instance segmentation, and semantic segmentation. Code is available at: https://github.com/CVMI-Lab/SlotCon.

cs.CV↗

arXiv2022

Discovering IoT Physical Channel Vulnerabilities

Smart homes contain diverse sensors and actuators controlled by IoT apps that provide custom automation. Prior works showed that an adversary could exploit physical interaction vulnerabilities among apps and put the users and environment at risk, e.g., to break into a house, an adversary turns on the heater to trigger an app that opens windows when the temperature exceeds a threshold. Currently, the safe behavior of physical interactions relies on either app code analysis or dynamic analysis of device states with manually derived policies by developers. However, existing works fail to achieve sufficient breadth and fidelity to translate the app code into their physical behavior or provide incomplete security policies, causing poor accuracy and false alarms. In this paper, we introduce a new approach, IoTSeer, which efficiently combines app code analysis and dynamic analysis with new security policies to discover physical interaction vulnerabilities. IoTSeer works by first translating sensor events and actuator commands of each app into a physical execution model (PeM) and unifying PeMs to express composite physical execution of apps (CPeM). CPeM allows us to deploy IoTSeer in different smart homes by defining its execution parameters with minimal data collection. IoTSeer supports new security policies with intended/unintended physical channel labels. It then efficiently checks them on the CPeM via falsification, which addresses the undecidability of verification due to the continuous and discrete behavior of IoT devices. We evaluate IoTSeer in an actual house with 14 actuators, six sensors, and 39 apps. IoTSeer discovers 16 unique policy violations, whereas prior works identify only 2 out of 16 with 18 falsely flagged violations. IoTSeer only requires 30 mins of data collection for each actuator to set the CPeM parameters and is adaptive to newly added, removed, and relocated devices.

cs.CR↗

arXiv2022

Differentiable Architecture Search with Random Features

Differentiable architecture search (DARTS) has significantly promoted the development of NAS techniques because of its high search efficiency and effectiveness but suffers from performance collapse. In this paper, we make efforts to alleviate the performance collapse problem for DARTS from two aspects. First, we investigate the expressive power of the supernet in DARTS and then derive a new setup of DARTS paradigm with only training BatchNorm. Second, we theoretically find that random features dilute the auxiliary connection role of skip-connection in supernet optimization and enable search algorithm focus on fairer operation selection, thereby solving the performance collapse problem. We instantiate DARTS and PC-DARTS with random features to build an improved version for each named RF-DARTS and RF-PCDARTS respectively. Experimental results show that RF-DARTS obtains \textbf{94.36\%} test accuracy on CIFAR-10 (which is the nearest optimal result in NAS-Bench-201), and achieves the newest state-of-the-art top-1 test error of \textbf{24.0\%} on ImageNet when transferring from CIFAR-10. Moreover, RF-DARTS performs robustly across three datasets (CIFAR-10, CIFAR-100, and SVHN) and four search spaces (S1-S4). Besides, RF-PCDARTS achieves even better results on ImageNet, that is, \textbf{23.9\%} top-1 and \textbf{7.1\%} top-5 test error, surpassing representative methods like single-path, training-free, and partial-channel paradigms directly searched on ImageNet.

cs.CV↗

arXiv2022

Understanding Masked Image Modeling via Learning Occlusion Invariant Feature

Recently, Masked Image Modeling (MIM) achieves great success in self-supervised visual recognition. However, as a reconstruction-based framework, it is still an open question to understand how MIM works, since MIM appears very different from previous well-studied siamese approaches such as contrastive learning. In this paper, we propose a new viewpoint: MIM implicitly learns occlusion-invariant features, which is analogous to other siamese methods while the latter learns other invariance. By relaxing MIM formulation into an equivalent siamese form, MIM methods can be interpreted in a unified framework with conventional methods, among which only a) data transformations, i.e. what invariance to learn, and b) similarity measurements are different. Furthermore, taking MAE (He et al.) as a representative example of MIM, we empirically find the success of MIM models relates a little to the choice of similarity functions, but the learned occlusion invariant feature introduced by masked image -- it turns out to be a favored initialization for vision transformers, even though the learned feature could be less semantic. We hope our findings could inspire researchers to develop more powerful self-supervised methods in computer vision community.

cs.CV↗

arXiv2022

Simple Baselines for Image Restoration

Although there have been significant advances in the field of image restoration recently, the system complexity of the state-of-the-art (SOTA) methods is increasing as well, which may hinder the convenient analysis and comparison of methods. In this paper, we propose a simple baseline that exceeds the SOTA methods and is computationally efficient. To further simplify the baseline, we reveal that the nonlinear activation functions, e.g. Sigmoid, ReLU, GELU, Softmax, etc. are not necessary: they could be replaced by multiplication or removed. Thus, we derive a Nonlinear Activation Free Network, namely NAFNet, from the baseline. SOTA results are achieved on various challenging benchmarks, e.g. 33.69 dB PSNR on GoPro (for image deblurring), exceeding the previous SOTA 0.38 dB with only 8.4% of its computational costs; 40.30 dB PSNR on SIDD (for image denoising), exceeding the previous SOTA 0.28 dB with less than half of its computational costs. The code and the pre-trained models are released at https://github.com/megvii-research/NAFNet.

cs.CV↗

arXiv2022

Revisiting the Critical Factors of Augmentation-Invariant Representation Learning

We focus on better understanding the critical factors of augmentation-invariant representation learning. We revisit MoCo v2 and BYOL and try to prove the authenticity of the following assumption: different frameworks bring about representations of different characteristics even with the same pretext task. We establish the first benchmark for fair comparisons between MoCo v2 and BYOL, and observe: (i) sophisticated model configurations enable better adaptation to pre-training dataset; (ii) mismatched optimization strategies of pre-training and fine-tuning hinder model from achieving competitive transfer performances. Given the fair benchmark, we make further investigation and find asymmetry of network structure endows contrastive frameworks to work well under the linear evaluation protocol, while may hurt the transfer performances on long-tailed classification tasks. Moreover, negative samples do not make models more sensible to the choice of data augmentations, nor does the asymmetric network structure. We believe our findings provide useful information for future work.

cs.CV↗

arXiv2022

Chern numbers of topological phonon band crossing determined with inelastic neutron scattering

Topological invariants in the band structure, such as Chern numbers, are crucial for the classification of topological matters and dictate the occurrence of exotic properties, yet their direct spectroscopic determination has been largely limited to electronic bands. Here, we use inelastic neutron scattering in conjunction with ab initio calculations to identify a variety of topological phonon band crossings in MnSi and CoSi single crystals. We find a distinct relation between the Chern numbers of a band-crossing node and the scattering intensity modulation in momentum space around the node. Given sufficiently high resolution, our method can be used to determine arbitrarily large Chern numbers of topological phonon band-crossing nodes.

cond-mat.mes-hall↗

arXiv2022

OGLE-2018-BLG-0799Lb: a $q \sim 2.7 \times 10^{-3}$ Planet with Spitzer Parallax

We report the discovery and analysis of a planet in the microlensing event OGLE-2018-BLG-0799. The planetary signal was observed by several ground-based telescopes, and the planet-host mass ratio is $q = (2.65 \pm 0.16) \times 10^{-3}$. The ground-based observations yield a constraint on the angular Einstein radius $θ_{\rm E}$, and the microlensing parallax vector $\vecπ_{\rm E}$, is strongly constrained by the Spitzer data. However, the 2019 Spitzer baseline data reveal systematics in the Spitzer photometry, so there is ambiguity in the magnitude of the parallax. In our preferred interpretation, a full Bayesian analysis using a Galactic model indicates that the planetary system is composed of an $M_{\rm planet} = 0.26_{-0.11}^{+0.22}~M_{J}$ planet orbiting an $M_{\rm host} = 0.093_{-0.038}^{+0.082}~M_{\odot}$, at a distance of $D_{\rm L} = 3.71_{-1.70}^{+3.24}$ kpc. An alternate interpretation of the data shifts the localization of the minima along the arc-shaped microlens parallax constraints. This, in turn, yields a more massive host with median mass of $0.13 {M_{\odot}}$ at a distance of 6.3 kpc. This analysis demonstrates the robustness of the osculating circles formalism, but shows that further investigation is needed to assess how systematics affect the specific localization of the microlens parallax vector and, consequently, the inferred physical parameters.

astro-ph.EP↗

arXiv2022

Systematic KMTNet Planetary Anomaly Search. IV. Complete Sample of 2019 Prime-Field

We report the complete statistical planetary sample from the prime fields ($Γ\geq 2~{\rm hr}^{-1}$) of the 2019 Korea Microlensing Telescope Network (KMTNet) microlensing survey. We develop the optimized KMTNet AnomalyFinder algorithm and apply it to the 2019 KMTNet prime fields. We find a total of 14 homogeneously selected planets and report the analysis of three planetary events, KMT-2019-BLG-(1042,1552,2974). The planet-host mass ratios, $q$, for the three planetary events are $6.34 \times 10^{-4}, 4.89 \times 10^{-3}$ and $6.18 \times 10^{-4}$, respectively. A Bayesian analysis indicates the three planets are all cold giant planets beyond the snow line of their host stars. The 13 planets are basically uniform in $\log q$ over the range $-5.0 < \log q < -1.5$. This result suggests that the planets below $q_{\rm break} = 1.7 \times 10^{-4}$ proposed by the MOA-II survey may be more common than previously believed. This work is an early component of a large project to determine the KMTNet mass-ratio function, and the whole sample of 2016--2019 KMTNet events should contain about 120 planets.

astro-ph.EP↗

arXiv2022

MOTR: End-to-End Multiple-Object Tracking with Transformer

Temporal modeling of objects is a key challenge in multiple object tracking (MOT). Existing methods track by associating detections through motion-based and appearance-based similarity heuristics. The post-processing nature of association prevents end-to-end exploitation of temporal variations in video sequence. In this paper, we propose MOTR, which extends DETR and introduces track query to model the tracked instances in the entire video. Track query is transferred and updated frame-by-frame to perform iterative prediction over time. We propose tracklet-aware label assignment to train track queries and newborn object queries. We further propose temporal aggregation network and collective average loss to enhance temporal relation modeling. Experimental results on DanceTrack show that MOTR significantly outperforms state-of-the-art method, ByteTrack by 6.5% on HOTA metric. On MOT17, MOTR outperforms our concurrent works, TrackFormer and TransTrack, on association performance. MOTR can serve as a stronger baseline for future research on temporal modeling and Transformer-based trackers. Code is available at https://github.com/megvii-research/MOTR.

cs.CV↗

arXiv2022

PETR: Position Embedding Transformation for Multi-View 3D Object Detection

In this paper, we develop position embedding transformation (PETR) for multi-view 3D object detection. PETR encodes the position information of 3D coordinates into image features, producing the 3D position-aware features. Object query can perceive the 3D position-aware features and perform end-to-end object detection. PETR achieves state-of-the-art performance (50.4% NDS and 44.1% mAP) on standard nuScenes dataset and ranks 1st place on the benchmark. It can serve as a simple yet strong baseline for future research. Code is available at \url{https://github.com/megvii-research/PETR}.

cs.CV↗

arXiv2022

Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches

Deep learning has substantially boosted the performance of Monocular Depth Estimation (MDE), a critical component in fully vision-based autonomous driving (AD) systems (e.g., Tesla and Toyota). In this work, we develop an attack against learning-based MDE. In particular, we use an optimization-based method to systematically generate stealthy physical-object-oriented adversarial patches to attack depth estimation. We balance the stealth and effectiveness of our attack with object-oriented adversarial design, sensitive region localization, and natural style camouflage. Using real-world driving scenarios, we evaluate our attack on concurrent MDE models and a representative downstream task for AD (i.e., 3D object detection). Experimental results show that our method can generate stealthy, effective, and robust adversarial patches for different target objects and models and achieves more than 6 meters mean depth estimation error and 93% attack success rate (ASR) in object detection with a patch of 1/9 of the vehicle's rear area. Field tests on three different driving routes with a real vehicle indicate that we cause over 6 meters mean depth estimation error and reduce the object detection rate from 90.70% to 5.16% in continuous video frames.

cs.CV↗

arXiv2022

Robust Watermarking for Video Forgery Detection with Improved Imperceptibility and Robustness

Videos are prone to tampering attacks that alter the meaning and deceive the audience. Previous video forgery detection schemes find tiny clues to locate the tampered areas. However, attackers can successfully evade supervision by destroying such clues using video compression or blurring. This paper proposes a video watermarking network for tampering localization. We jointly train a 3D-UNet-based watermark embedding network and a decoder that predicts the tampering mask. The perturbation made by watermark embedding is close to imperceptible. Considering that there is no off-the-shelf differentiable video codec simulator, we propose to mimic video compression by ensembling simulation results of other typical attacks, e.g., JPEG compression and blurring, as an approximation. Experimental results demonstrate that our method generates watermarked videos with good imperceptibility and robustly and accurately locates tampered areas within the attacked version.

cs.CV↗

arXiv2022

Magnetic molecular orbitals in MnSi

A large body of knowledge about magnetism is attained from models of interacting spins, which usually reside on magnetic ions. Proposals beyond the ionic picture are uncommon and seldom verified by direct observations in conjunction with microscopic theory. Here, using inelastic neutron scattering to study the itinerant near-ferromagnet MnSi, we find that the system's fundamental magnetic units are interconnected, extended molecular orbitals consisting of three Mn atoms each, rather than individual Mn atoms. This result is further corroborated by magnetic Wannier orbitals obtained by ab initio calculations. It contrasts the ionic picture with a concrete example, and presents a novel regime of the spin waves where the wavelength is comparable to the spatial extent of the molecular orbitals. Our discovery brings important insights into not only the magnetism of MnSi, but also a broad range of magnetic quantum materials where structural symmetry, electron itinerancy and correlations act in concert.

cond-mat.str-el↗

arXiv2022

DECK: Model Hardening for Defending Pervasive Backdoors

Pervasive backdoors are triggered by dynamic and pervasive input perturbations. They can be intentionally injected by attackers or naturally exist in normally trained models. They have a different nature from the traditional static and localized backdoors that can be triggered by perturbing a small input area with some fixed pattern, e.g., a patch with solid color. Existing defense techniques are highly effective for traditional backdoors. However, they may not work well for pervasive backdoors, especially regarding backdoor removal and model hardening. In this paper, we propose a novel model hardening technique against pervasive backdoors, including both natural and injected backdoors. We develop a general pervasive attack based on an encoder-decoder architecture enhanced with a special transformation layer. The attack can model a wide range of existing pervasive backdoor attacks and quantify them by class distances. As such, using the samples derived from our attack in adversarial training can harden a model against these backdoor vulnerabilities. Our evaluation on 9 datasets with 15 model structures shows that our technique can enlarge class distances by 59.65% on average with less than 1% accuracy degradation and no robustness loss, outperforming five hardening techniques such as adversarial training, universal adversarial training, MOTH, etc. It can reduce the attack success rate of six pervasive backdoor attacks from 99.06% to 1.94%, surpassing seven state-of-the-art backdoor removal techniques.

cs.CR↗

arXiv2022

RWN: Robust Watermarking Network for Image Cropping Localization

Image cropping can be maliciously used to manipulate the layout of an image and alter the underlying meaning. Previous image crop detection schemes only predicts whether an image has been cropped, ignoring which part of the image is cropped. This paper presents a novel robust watermarking network (RWN) for image crop localization. We train an anti-crop processor (ACP) that embeds a watermark into a target image. The visually indistinguishable protected image is then posted on the social network instead of the original image. At the recipient's side, ACP extracts the watermark from the attacked image, and we conduct feature matching on the original and extracted watermark to locate the position of the crop in the original image plane. We further extend our scheme to detect tampering attack on the attacked image. Besides, we explore a simple yet efficient method (JPEG-Mixup) to improve the generalization of JPEG robustness. According to our comprehensive experiments, RWN is the first to provide high-accuracy and robust image crop localization. Besides, the accuracy of tamper detection is comparable with many state-of-the-art passive-based methods.

cs.CV↗

arXiv2022

Systematic KMTNet Planetary Anomaly Search, Paper I: OGLE-2019-BLG-1053Lb, A Buried Terrestrial Planet

In order to exhume the buried signatures of "missing planetary caustics" in the KMTNet data, we conducted a systematic anomaly search to the residuals from point-source point-lens fits, based on a modified version of the KMTNet EventFinder algorithm. This search reveals the lowest mass-ratio planetary caustic to date in the microlensing event OGLE-2019-BLG-1053, for which the planetary signal had not been noticed before. The planetary system has a planet-host mass ratio of $q = (1.25 \pm 0.13) \times 10^{-5}$. A Bayesian analysis yields estimates of the mass of the host star, $M_{\rm host} = 0.61_{-0.24}^{+0.29}~M_\odot$, the mass of its planet, $M_{\rm planet} = 2.48_{-0.98}^{+1.19}~M_{\oplus}$, the projected planet-host separation, $a_\perp = 3.4_{-0.5}^{+0.5}$ au, and the lens distance of $D_{\rm L} = 6.8_{-0.9}^{+0.6}$ kpc. The discovery of this very low mass-ratio planet illustrates the utility of our method and opens a new window for a large and homogeneous sample to study the microlensing planet-host mass-ratio function down to $q \sim 10^{-5}$.

astro-ph.EP↗