arXiv subjects

He Wang

Publications and source records attributed to He Wang.

At least 307 records · Page 17Linked to original sources

arXiv2022

Defending Black-box Skeleton-based Human Activity Classifiers

Skeletal motions have been heavily replied upon for human activity recognition (HAR). Recently, a universal vulnerability of skeleton-based HAR has been identified across a variety of classifiers and data, calling for mitigation. To this end, we propose the first black-box defense method for skeleton-based HAR to our best knowledge. Our method is featured by full Bayesian treatments of the clean data, the adversaries and the classifier, leading to (1) a new Bayesian Energy-based formulation of robust discriminative classifiers, (2) a new adversary sampling scheme based on natural motion manifolds, and (3) a new post-train Bayesian strategy for black-box defense. We name our framework Bayesian Energy-based Adversarial Training or BEAT. BEAT is straightforward but elegant, which turns vulnerable black-box classifiers into robust ones without sacrificing accuracy. It demonstrates surprising and universal effectiveness across a wide range of skeletal HAR classifiers and datasets, under various attacks. Code is available at https://github.com/realcrane/RobustActionRecogniser.

cs.CV↗

arXiv2022

Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects

Commercial depth sensors usually generate noisy and missing depths, especially on specular and transparent objects, which poses critical issues to downstream depth or point cloud-based tasks. To mitigate this problem, we propose a powerful RGBD fusion network, SwinDRNet, for depth restoration. We further propose Domain Randomization-Enhanced Depth Simulation (DREDS) approach to simulate an active stereo depth system using physically based rendering and generate a large-scale synthetic dataset that contains 130K photorealistic RGB images along with their simulated depths carrying realistic sensor noises. To evaluate depth restoration methods, we also curate a real-world dataset, namely STD, that captures 30 cluttered scenes composed of 50 objects with different materials from specular, transparent, to diffuse. Experiments demonstrate that the proposed DREDS dataset bridges the sim-to-real domain gap such that, trained on DREDS, our SwinDRNet can seamlessly generalize to other real depth datasets, e.g. ClearGrasp, and outperform the competing methods on depth restoration with a real-time speed. We further show that our depth restoration effectively boosts the performance of downstream tasks, including category-level pose estimation and grasping tasks. Our data and code are available at https://github.com/PKU-EPIC/DREDS

cs.CV↗

arXiv2022

LoopDA: Constructing Self-loops to Adapt Nighttime Semantic Segmentation

Due to the lack of training labels and the difficulty of annotating, dealing with adverse driving conditions such as nighttime has posed a huge challenge to the perception system of autonomous vehicles. Therefore, adapting knowledge from a labelled daytime domain to an unlabelled nighttime domain has been widely researched. In addition to labelled daytime datasets, existing nighttime datasets usually provide nighttime images with corresponding daytime reference images captured at nearby locations for reference. The key challenge is to minimize the performance gap between the two domains. In this paper, we propose LoopDA for domain adaptive nighttime semantic segmentation. It consists of self-loops that result in reconstructing the input data using predicted semantic maps, by rendering them into the encoded features. In a warm-up training stage, the self-loops comprise of an inner-loop and an outer-loop, which are responsible for intra-domain refinement and inter-domain alignment, respectively. To reduce the impact of day-night pose shifts, in the later self-training stage, we propose a co-teaching pipeline that involves an offline pseudo-supervision signal and an online reference-guided signal `DNA' (Day-Night Agreement), bringing substantial benefits to enhance nighttime segmentation. Our model outperforms prior methods on Dark Zurich and Nighttime Driving datasets for semantic segmentation. Code and pretrained models are available at https://github.com/fy-vision/LoopDA.

cs.CV↗

arXiv2022

Probing the gravitational wave background from cosmic strings with Alternative LISA-TAIJI network

As one of the detection targets of all gravitational wave detectors at present, stochastic gravitational wave background (SGWB) provides us an important way to understand the evolution of our universe. In this paper, we explore the feasibility of detecting the SGWB generated by the loops, which arose throughout the cosmological evolution of the cosmic string network, by individual space detectors (e.g.~LISA, TAIJI) and joint space detectors (LISA-TAIJI). For joint detectors, we choose three different configurations of TAIJI (e.g.~TAIJIm, TAIJIp, TAIJIc) to form the LISA-TAIJI networks. And we investigate the performance of them to detect the SGWB. Though comparing the power-law sensitivity (PLS) curves of individual space detectors and joint detectors with energy density spectrum of SGWB. We find that LISA-TAIJIc has the best sensitivity for detecting the SGWB from cosmic string loops and is promising to further constrains the tension of cosmic sting $Gμ=O(10^{-17})$.

gr-qc↗

arXiv2022

Region Proposal Rectification Towards Robust Instance Segmentation of Biological Images

Top-down instance segmentation framework has shown its superiority in object detection compared to the bottom-up framework. While it is efficient in addressing over-segmentation, top-down instance segmentation suffers from over-crop problem. However, a complete segmentation mask is crucial for biological image analysis as it delivers important morphological properties such as shapes and volumes. In this paper, we propose a region proposal rectification (RPR) module to address this challenging incomplete segmentation problem. In particular, we offer a progressive ROIAlign module to introduce neighbor information into a series of ROIs gradually. The ROI features are fed into an attentive feed-forward network (FFN) for proposal box regression. With additional neighbor information, the proposed RPR module shows significant improvement in correction of region proposal locations and thereby exhibits favorable instance segmentation performances on three biological image datasets compared to state-of-the-art baseline methods. Experimental results demonstrate that the proposed RPR module is effective in both anchor-based and anchor-free top-down instance segmentation approaches, suggesting the proposed method can be applied to general top-down instance segmentation of biological images.

cs.CV↗

arXiv2022

The Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CMRxMotion)

The quality of cardiac magnetic resonance (CMR) imaging is susceptible to respiratory motion artifacts. The model robustness of automated segmentation techniques in face of real-world respiratory motion artifacts is unclear. This manuscript describes the design of extreme cardiac MRI analysis challenge under respiratory motion (CMRxMotion Challenge). The challenge aims to establish a public benchmark dataset to assess the effects of respiratory motion on image quality and examine the robustness of segmentation models. The challenge recruited 40 healthy volunteers to perform different breath-hold behaviors during one imaging visit, obtaining paired cine imaging with artifacts. Radiologists assessed the image quality and annotated the level of respiratory motion artifacts. For those images with diagnostic quality, radiologists further segmented the left ventricle, left ventricle myocardium and right ventricle. The images of training set (20 volunteers) along with the annotations are released to the challenge participants, to develop an automated image quality assessment model (Task 1) and an automated segmentation model (Task 2). The images of validation set (5 volunteers) are released to the challenge participants but the annotations are withheld for online evaluation of submitted predictions. Both the images and annotations of the test set (15 volunteers) were withheld and only used for offline evaluation of submitted containerized dockers. The image quality assessment task is quantitatively evaluated by the Cohen's kappa statistics and the segmentation task is evaluated by the Dice scores and Hausdorff distances.

eess.IV↗

arXiv2022

Shape Completion with Points in the Shadow

Single-view point cloud completion aims to recover the full geometry of an object based on only limited observation, which is extremely hard due to the data sparsity and occlusion. The core challenge is to generate plausible geometries to fill the unobserved part of the object based on a partial scan, which is under-constrained and suffers from a huge solution space. Inspired by the classic shadow volume technique in computer graphics, we propose a new method to reduce the solution space effectively. Our method considers the camera a light source that casts rays toward the object. Such light rays build a reasonably constrained but sufficiently expressive basis for completion. The completion process is then formulated as a point displacement optimization problem. Points are initialized at the partial scan and then moved to their goal locations with two types of movements for each point: directional movements along the light rays and constrained local movement for shape refinement. We design neural networks to predict the ideal point movements to get the completion results. We demonstrate that our method is accurate, robust, and generalizable through exhaustive evaluation and comparison. Moreover, it outperforms state-of-the-art methods qualitatively and quantitatively on MVP datasets.

cs.CV↗

arXiv2022

VFLens: Co-design the Modeling Process for Efficient Vertical Federated Learning via Visualization

As a decentralized training approach, federated learning enables multiple organizations to jointly train a model without exposing their private data. This work investigates vertical federated learning (VFL) to address scenarios where collaborating organizations have the same set of users but with different features, and only one party holds the labels. While VFL shows good performance, practitioners often face uncertainty when preparing non-transparent, internal/external features and samples for the VFL training phase. Moreover, to balance the prediction accuracy and the resource consumption of model inference, practitioners require to know which subset of prediction instances is genuinely needed to invoke the VFL model for inference. To this end, we co-design the VFL modeling process by proposing an interactive real-time visualization system, VFLens, to help practitioners with feature engineering, sample selection, and inference. A usage scenario, a quantitative experiment, and expert feedback suggest that VFLens helps practitioners boost VFL efficiency at a lower cost with sufficient confidence.

cs.HC↗

arXiv2022

Spatio-temporal Keyframe Control of Traffic Simulation using Coarse-to-Fine Optimization

We present a novel traffic trajectory editing method which uses spatio-temporal keyframes to control vehicles during the simulation to generate desired traffic trajectories. By taking self-motivation, path following and collision avoidance into account, the proposed force-based traffic simulation framework updates vehicle's motions in both the Frenet coordinates and the Cartesian coordinates. With the way-points from users, lane-level navigation can be generated by reference path planning. With a given keyframe, the coarse-to-fine optimization is proposed to efficiently generate the plausible trajectory which can satisfy the spatio-temporal constraints. At first, a directed state-time graph constructed along the reference path is used to search for a coarse-grained trajectory by mapping the keyframe as the goal. Then, using the information extracted from the coarse trajectory as initialization, adjoint-based optimization is applied to generate a finer trajectory with smooth motions based on our force-based simulation. We validate our method with extensive experiments.

cs.GR↗

arXiv2022

Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild

In this work, we tackle the challenging task of jointly tracking hand object pose and reconstructing their shapes from depth point cloud sequences in the wild, given the initial poses at frame 0. We for the first time propose a point cloud based hand joint tracking network, HandTrackNet, to estimate the inter-frame hand joint motion. Our HandTrackNet proposes a novel hand pose canonicalization module to ease the tracking task, yielding accurate and robust hand joint tracking. Our pipeline then reconstructs the full hand via converting the predicted hand joints into a template-based parametric hand model MANO. For object tracking, we devise a simple yet effective module that estimates the object SDF from the first frame and performs optimization-based tracking. Finally, a joint optimization step is adopted to perform joint hand and object reasoning, which alleviates the occlusion-induced ambiguity and further refines the hand pose. During training, the whole pipeline only sees purely synthetic data, which are synthesized with sufficient variations and by depth simulation for the ease of generalization. The whole pipeline is pertinent to the generalization gaps and thus directly transferable to real in-the-wild data. We evaluate our method on two real hand object interaction datasets, e.g. HO3D and DexYCB, without any finetuning. Our experiments demonstrate that the proposed method significantly outperforms the previous state-of-the-art depth-based hand and object pose estimation and tracking methods, running at a frame rate of 9 FPS.

cs.CV↗

arXiv2022

Quantum correlations of a two-qubit system and the Aubry-André chain in bosonic environments

In this research, we analyze two models using the tensor network algorithm. The quantum correlations of a two-qubit system are first studied in different bosonic reservoirs. Both equilibrium and nonequilibrium scenarios are discussed. Non-Markovian effects can improve the survival time of the quantum correlations significantly and weaken the decoherence effect. Non-Markovian dynamics with existing memory can lead to entanglement rebirth in specific scenarios instead of the eventual entanglement decay or death seen in memoryless Markovian cases. The system reaches a steady state quickest in sub-Ohmic reservoirs and shows the most apparent non-Markovian behavior in super-Ohmic reservoirs. We not only study the impact of the environment on quantum correlations but also how to protect quantum correlations. Starting from a state in which the two ends are maximally entangled, a one-dimensional AA chain model is also studied. We identify distinct phases by monitoring the imbalance dynamics. When the chain is closed, the imbalance dynamics behave differently in various phases, and so does the entanglement evolution between the chain's ends. When the first site couples to a bath, we found the imbalance dynamics can still be an effective indicator to differentiate various phases in an early evolution stage since the imbalance dynamics is only remarkably affected at relatively high temperatures. The distribution of the eigenenergy of the system can account for it. The entanglement of the chain ends decays rapidly in all phases due to one of the ends being coupled to the bath directly. However, the entanglement of the chain ends will persist for a perceptible amount of time in the localization phase if the bath is coupled to the middle site of the chain. Our research shows that one can utilize the disordered environment as a buffer to protect quantum correlations.

quant-ph↗

arXiv2022

MLGWSC-1: The first Machine Learning Gravitational-Wave Search Mock Data Challenge

We present the results of the first Machine Learning Gravitational-Wave Search Mock Data Challenge (MLGWSC-1). For this challenge, participating groups had to identify gravitational-wave signals from binary black hole mergers of increasing complexity and duration embedded in progressively more realistic noise. The final of the 4 provided datasets contained real noise from the O3a observing run and signals up to a duration of 20 seconds with the inclusion of precession effects and higher order modes. We present the average sensitivity distance and runtime for the 6 entered algorithms derived from 1 month of test data unknown to the participants prior to submission. Of these, 4 are machine learning algorithms. We find that the best machine learning based algorithms are able to achieve up to 95% of the sensitive distance of matched-filtering based production analyses for simulated Gaussian noise at a false-alarm rate (FAR) of one per month. In contrast, for real noise, the leading machine learning search achieved 70%. For higher FARs the differences in sensitive distance shrink to the point where select machine learning submissions outperform traditional search algorithms at FARs $\geq 200$ per month on some datasets. Our results show that current machine learning search algorithms may already be sensitive enough in limited parameter regions to be useful for some production settings. To improve the state-of-the-art, machine learning algorithms need to reduce the false-alarm rates at which they are capable of detecting signals and extend their validity to regions of parameter space where modeled searches are computationally expensive to run. Based on our findings we compile a list of research areas that we believe are the most important to elevate machine learning searches to an invaluable tool in gravitational-wave signal detection.

astro-ph.IM↗

arXiv2022

Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations

Generalizable object manipulation skills are critical for intelligent and multi-functional robots to work in real-world complex scenes. Despite the recent progress in reinforcement learning, it is still very challenging to learn a generalizable manipulation policy that can handle a category of geometrically diverse articulated objects. In this work, we tackle this category-level object manipulation policy learning problem via imitation learning in a task-agnostic manner, where we assume no handcrafted dense rewards but only a terminal reward. Given this novel and challenging generalizable policy learning problem, we identify several key issues that can fail the previous imitation learning algorithms and hinder the generalization to unseen instances. We then propose several general but critical techniques, including generative adversarial self-imitation learning from demonstrations, progressive growing of discriminator, and instance-balancing for expert buffer, that accurately pinpoints and tackles these issues and can benefit category-level manipulation policy learning regardless of the tasks. Our experiments on ManiSkill benchmarks demonstrate a remarkable improvement on all tasks and our ablation studies further validate the contribution of each proposed technique.

cs.RO↗

arXiv2022

Talking Head from Speech Audio using a Pre-trained Image Generator

We propose a novel method for generating high-resolution videos of talking-heads from speech audio and a single 'identity' image. Our method is based on a convolutional neural network model that incorporates a pre-trained StyleGAN generator. We model each frame as a point in the latent space of StyleGAN so that a video corresponds to a trajectory through the latent space. Training the network is in two stages. The first stage is to model trajectories in the latent space conditioned on speech utterances. To do this, we use an existing encoder to invert the generator, mapping from each video frame into the latent space. We train a recurrent neural network to map from speech utterances to displacements in the latent space of the image generator. These displacements are relative to the back-projection into the latent space of an identity image chosen from the individuals depicted in the training dataset. In the second stage, we improve the visual quality of the generated videos by tuning the image generator on a single image or a short video of any chosen identity. We evaluate our model on standard measures (PSNR, SSIM, FID and LMD) and show that it significantly outperforms recent state-of-the-art methods on one of two commonly used datasets and gives comparable performance on the other. Finally, we report on ablation experiments that validate the components of the model. The code and videos from experiments can be found at https://mohammedalghamdi.github.io/talking-heads-acm-mm

cs.CV↗

arXiv2022

Functionally Reconfigurable Integrated Structure of Fabry-Perot Antenna and Wideband Liquid Absorber

This paper proposes a functionally reconfigurable integrated structure of a Fabry-Perot (FP) antenna and wideband liquid absorber. The partial reflecting surface (PRS) is elaborately tailored to serve as both a component of the FP antenna and the metal ground of the broadband liquid absorber. Then the integrated structure is realized by combining the FP antenna with the liquid absorber. The PRS is composed of patches on the top layer of the substrate and the square loop on the bottom and the microstrip patch antenna is used as a source. The liquid absorber is composed of a 3-D printed container, 45% ethanol layer, and a metal ground. By this design, the structure can serve as the FP antenna when the ethanol is extracted or the liquid absorber when the ethanol is injected. This can be used to switch between detection and stealth mode which can better adapt to the complex electromagnetic environment. The simulation results show that the gain of the antenna reaches to 19.7 dBi and the operating band is 100 MHz when the ethanol is extracted. When the ethanol is injected, the absorption band of the absorber ranges from 4 GHz to 20 GHz and the absorption bandwidth is over 133%. The monostatic RCS reduction bands of the structure with ethanol range from 4 GHz to 20GHz and the average RCS reduction is 28.4 dB. The measured and simulated results agree well.

physics.app-ph↗

arXiv2022

Design of Switchable Frequency-Selective Rasorber with A-R-A-T or A-T-A-R Operating Modes

This work presents a switchable frequency-selective rasorber (SFSR) with two operating modes. Switching from transmission to reflection can be achieved by appropriately adding feeders and PIN diodes based on cascaded two-dimensional lossy and lossless frequency-selective surface (FSS) screens. The proposed SFSR can realize out-of-band absorption. Analysis of the equivalent circuit model (ECM) can be useful for achieving a switchable rasorber. As the state of the PIN diodes changes, the working state of the SFSR can be switched from low-frequency reflection and high-frequency transmission to low-frequency transmission and high-frequency reflection, respectively. In the final-revision simulation of the working band of the SFSR, in the ON state, reflection and transmission peaks of -0.55 dB at 4.06 GHz and -0.52 dB at 5.97 GHz, respectively, are achieved; in the OFF state, the transmission and reflection peaks of -0.31 dB at 4.04GHz and -0.26 dB at 6.01 GHz, respectively, are obtained. A prototype sample is developed and validated. The results are in good agreement with those of the full-wave simulation. The proposed design can be used in intelligent anti-jamming communication.

physics.app-ph↗

arXiv2022

Efficient characterization of quantum nondemolition qubit readout

We study the quantitative characterization of the performance of qubit measurements in this paper. In particular, the back-action evading nature of quantum nondemolition (QND) readout of qubits is fully quantified by quantum trace distance. Only computational basis states are necessary to be taken into consideration. Most importantly, we propose an experimentally efficient method to evaluate the QND fidelity based on the classical trace distance, which uses an experimental scheme with two consecutive measurements. The three key quantifiers of a measurement, i.e., QND fidelity, readout fidelity, and projectivity, can be derived directly from the same experimental scheme. Besides, we present the relationships among these three factors. Theoretical simulation results for the dispersive readout of superconducting qubits show the validity of the proposed QND fidelity. Efficient quantification of measurement performance is of practical significance for the diagnosis, performance improvement, and design of measurement apparatuses.

quant-ph↗

arXiv2022

FishFuzz: Throwing Larger Nets to Catch Deeper Bugs

Greybox fuzzing is the de-facto standard to discover bugs during development. Fuzzers execute many inputs to maximize the amount of reached code. Recently, Directed Greybox Fuzzers (DGFs) propose an alternative strategy that goes beyond "just" coverage: driving testing toward specific code targets by selecting "closer" seeds. DGFs go through different phases: exploration (i.e., reaching interesting locations) and exploitation (i.e., triggering bugs). In practice, DGFs leverage coverage to directly measure exploration, while exploitation is, at best, measured indirectly by alternating between different targets. Specifically, we observe two limitations in existing DGFs: (i) they lack precision in their distance metric, i.e., averaging multiple paths and targets into a single score (to decide which seeds to prioritize), and (ii) they assign energy to seeds in a round-robin fashion without adjusting the priority of the targets (exhaustively explored targets should be dropped). We propose FishFuzz, which draws inspiration from trawl fishing: first casting a wide net, scraping for high coverage, then slowly pulling it in to maximize the harvest. The core of our fuzzer is a novel seed selection strategy that builds on two concepts: (i) a novel multi-distance metric whose precision is independent of the number of targets, and (ii) a dynamic target ranking to automatically discard exhausted targets. This strategy allows FishFuzz to seamlessly scale to tens of thousands of targets and dynamically alternate between exploration and exploitation phases. We evaluate FishFuzz by leveraging all sanitizer labels as targets. Extensively comparing FishFuzz against modern DGFs and coverage-guided fuzzers shows that FishFuzz reached higher coverage compared to the direct competitors, reproduces existing bugs (70.2% faster), and finally discovers 25 new bugs (18 CVEs) in 44 programs.

cs.CR↗