arXiv subjects

Qian He

Publications and source records attributed to Qian He.

90 records · Page 5Linked to original sources

arXiv2022

SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection

Convolutional neural networks (CNNs) are good at extracting contexture features within certain receptive fields, while transformers can model the global long-range dependency features. By absorbing the advantage of transformer and the merit of CNN, Swin Transformer shows strong feature representation ability. Based on it, we propose a cross-modality fusion model SwinNet for RGB-D and RGB-T salient object detection. It is driven by Swin Transformer to extract the hierarchical features, boosted by attention mechanism to bridge the gap between two modalities, and guided by edge information to sharp the contour of salient object. To be specific, two-stream Swin Transformer encoder first extracts multi-modality features, and then spatial alignment and channel re-calibration module is presented to optimize intra-level cross-modality features. To clarify the fuzzy boundary, edge-guided decoder achieves inter-level cross-modality fusion under the guidance of edge features. The proposed model outperforms the state-of-the-art models on RGB-D and RGB-T datasets, showing that it provides more insight into the cross-modality complementarity task.

cs.CV↗

arXiv2022

Region-Aware Face Swapping

This paper presents a novel Region-Aware Face Swapping (RAFSwap) network to achieve identity-consistent harmonious high-resolution face generation in a local-global manner: \textbf{1)} Local Facial Region-Aware (FRA) branch augments local identity-relevant features by introducing the Transformer to effectively model misaligned cross-scale semantic interaction. \textbf{2)} Global Source Feature-Adaptive (SFA) branch further complements global identity-relevant cues for generating identity-consistent swapped faces. Besides, we propose a \textit{Face Mask Predictor} (FMP) module incorporated with StyleGAN2 to predict identity-relevant soft facial masks in an unsupervised manner that is more practical for generating harmonious high-resolution faces. Abundant experiments qualitatively and quantitatively demonstrate the superiority of our method for generating more identity-consistent high-resolution swapped faces over SOTA methods, \eg, obtaining 96.70 ID retrieval that outperforms SOTA MegaFS by 5.87$\uparrow$.

cs.CV↗

arXiv2022

CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP

Training a text-to-image generator in the general domain (e.g., Dall.e, CogView) requires huge amounts of paired text-image data, which is too expensive to collect. In this paper, we propose a self-supervised scheme named as CLIP-GEN for general text-to-image generation with the language-image priors extracted with a pre-trained CLIP model. In our approach, we only require a set of unlabeled images in the general domain to train a text-to-image generator. Specifically, given an image without text labels, we first extract the embedding of the image in the united language-vision embedding space with the image encoder of CLIP. Next, we convert the image into a sequence of discrete tokens in the VQGAN codebook space (the VQGAN model can be trained with the unlabeled image dataset in hand). Finally, we train an autoregressive transformer that maps the image tokens from its unified language-vision representation. Once trained, the transformer can generate coherent image tokens based on the text embedding extracted from the text encoder of CLIP upon an input text. Such a strategy enables us to train a strong and general text-to-image generator with large text-free image dataset such as ImageNet. Qualitative and quantitative evaluations verify that our method significantly outperforms optimization-based text-to-image methods in terms of image quality while not compromising the text-image matching. Our method can even achieve comparable performance as flagship supervised models like CogView.

cs.CV↗

arXiv2022

Weakly Supervised Nuclei Segmentation via Instance Learning

Weakly supervised nuclei segmentation is a critical problem for pathological image analysis and greatly benefits the community due to the significant reduction of labeling cost. Adopting point annotations, previous methods mostly rely on less expressive representations for nuclei instances and thus have difficulty in handling crowded nuclei. In this paper, we propose to decouple weakly supervised semantic and instance segmentation in order to enable more effective subtask learning and to promote instance-aware representation learning. To achieve this, we design a modular deep network with two branches: a semantic proposal network and an instance encoding network, which are trained in a two-stage manner with an instance-sensitive loss. Empirical results show that our approach achieves the state-of-the-art performance on two public benchmarks of pathological images from different types of organs.

eess.IV↗

arXiv2021

FaceEraser: Removing Facial Parts for Augmented Reality

Our task is to remove all facial parts (e.g., eyebrows, eyes, mouth and nose), and then impose visual elements onto the ``blank'' face for augmented reality. Conventional object removal methods rely on image inpainting techniques (e.g., EdgeConnect, HiFill) that are trained in a self-supervised manner with randomly manipulated image pairs. Specifically, given a set of natural images, randomly masked images are used as inputs and the raw images are treated as ground truths. Whereas, this technique does not satisfy the requirements of facial parts removal, as it is hard to obtain ``ground-truth'' images with real ``blank'' faces. To address this issue, we propose a novel data generation technique to produce paired training data that well mimic the ``blank'' faces. In the mean time, we propose a novel network architecture for improved inpainting quality for our task. Finally, we demonstrate various face-oriented augmented reality applications on top of our facial parts removal model. The source codes are released at \href{https://github.com/duxingren14/FaceEraser}{duxingren14/FaceEraser} on github for research purposes.

cs.CV↗

arXiv2021

Single Image 3D Object Estimation with Primitive Graph Networks

Reconstructing 3D object from a single image (RGB or depth) is a fundamental problem in visual scene understanding and yet remains challenging due to its ill-posed nature and complexity in real-world scenes. To address those challenges, we adopt a primitive-based representation for 3D object, and propose a two-stage graph network for primitive-based 3D object estimation, which consists of a sequential proposal module and a graph reasoning module. Given a 2D image, our proposal module first generates a sequence of 3D primitives from input image with local feature attention. Then the graph reasoning module performs joint reasoning on a primitive graph to capture the global shape context for each primitive. Such a framework is capable of taking into account rich geometry and semantic constraints during 3D structure recovery, producing 3D objects with more coherent structure even under challenging viewing conditions. We train the entire graph neural network in a stage-wise strategy and evaluate it on three benchmarks: Pix3D, ModelNet and NYU Depth V2. Extensive experiments show that our approach outperforms the previous state of the arts with a considerable margin.

cs.CV↗

arXiv2021

Weakly Supervised Volumetric Segmentation via Self-taught Shape Denoising Model

Weakly supervised segmentation is an important problem in medical image analysis due to the high cost of pixelwise annotation. Prior methods, while often focusing on weak labels of 2D images, exploit few structural cues of volumetric medical images. To address this, we propose a novel weakly-supervised segmentation strategy capable of better capturing 3D shape prior in both model prediction and learning. Our main idea is to extract a self-taught shape representation by leveraging weak labels, and then integrate this representation into segmentation prediction for shape refinement. To this end, we design a deep network consisting of a segmentation module and a shape denoising module, which are trained by an iterative learning strategy. Moreover, we introduce a weak annotation scheme with a hybrid label design for volumetric images, which improves model learning without increasing the overall annotation cost. The empirical experiments show that our approach outperforms existing SOTA strategies on three organ segmentation benchmarks with distinctive shape properties. Notably, we can achieve strong performance with even 10\% labeled slices, which is significantly superior to other methods.

cs.CV↗

arXiv2019

An Empirical Study of Propagation-based Methods for Video Object Segmentation

While propagation-based approaches have achieved state-of-the-art performance for video object segmentation, the literature lacks a fair comparison of different methods using the same settings. In this paper, we carry out an empirical study for propagation-based methods. We view these approaches from a unified perspective and conduct detailed ablation study for core methods, input cues, multi-object combination and training strategies. With careful designs, our improved end-to-end memory networks achieve a global mean of 76.1 on DAVIS 2017 val set.

cs.CV↗

arXiv2019

A Study on the Characteristics of Douyin Short Videos and Implications for Edge Caching

Douyin, internationally known as TikTok, has become one of the most successful short-video platforms. To maintain its popularity, Douyin has to provide better Quality of Experience (QoE) to its growing user base. Understanding the characteristics of Douyin videos is thus critical to its service improvement and system design. In this paper, we present an initial study on the fundamental characteristics of Douyin videos based on a dataset of over 260 thousand short videos collected across three months. The characteristics of Douyin videos are found to be significantly different from traditional online videos, ranging from video bitrate, size, to popularity. In particular, the distributions of the bitrate and size of videos follow Weibull distribution. We further observe that the most popular Douyin videos follow Zifp's law on video popularity, but the rest of the videos do not. We also investigate the correlation between popularity metrics used for Douyin videos. It is found that the correlation between the number of views and the number of likes are strong, while other correlations are relatively low. Finally, by using a case study, we demonstrate that the above findings can provide important guidance on designing an efficient edge caching system.

cs.MM↗

arXiv2019

Amplitude-modulated pulses based phase-sensitive OTDR in distributed sensing

The detected frequency response range of phase-sensitive optical time domain reflectometry is limited by the sensing fiber length. In this paper, we demonstrate an amplitude modulated pulses based phase-sensitive OTDR to detect vibrations with high spatial resolution and wide frequency response range. The amplitude modulated pulses consisting of narrow pulses and background light are injected into the sensing fiber as probe light. The Rayleigh backscattering generated by the background light interferes at the detection end, presenting a phase sensitive response to external vibrations. The Rayleigh backscattering of narrow pulses are tracked to locate vibrations. The frequency information is obtained from the frequency domain analysis of the interference signal. The system can realize high frequency response with low repetition rate of probe pulses by one-end detection.

physics.app-ph↗

arXiv2019

Reconstruction of 3-D Atomic Distortions from Electron Microscopy with Deep Learning

Deep learning has demonstrated superb efficacy in processing imaging data, yet its suitability in solving challenging inverse problems in scientific imaging has not been fully explored. Of immense interest is the determination of local material properties from atomically-resolved imaging, such as electron microscopy, where such information is encoded in subtle and complex data signatures, and whose recovery and interpretation necessitate intensive numerical simulations subject to the requirement of near-perfect knowledge of the experimental setup. We demonstrate that an end-to-end deep learning model can successfully recover 3-dimensional atomic distortions of a variety of oxide perovskite materials from a single 2-dimensional experimental scanning transmission electron (STEM) micrograph, in the process resolving a longstanding question in the recovery of 3-D atomic distortions from STEM experiments. Our results indicate that deep learning is a promising approach to efficiently address unsolved inverse problems in scientific imaging and to underpin novel material investigations at atomic resolution.

cond-mat.mtrl-sci↗

arXiv2019

The detection of non-Gaussian vibrations with improved spatial resolution and signal-to-noise ratio in distributed sensing

In fiber-optic distributed sensing, vibration signals are mostly assumed to follow Gaussian distribution for the simplicity of signal processing. However, in real applications, vibration signals often behave as non-Gaussian processes, which have rarely been highly considered. In this paper, a higher-order cumulants algorithm based phase-sensitive optical time-domain reflectometry (OTDR) is proposed to detect and analyze non-Gaussian vibration signals accompanied with noises. When disturbances are applied on the sensing fiber, the distribution probability of Rayleigh backscattering signals will deviate from the ideal Gaussian distribution. The non-Gaussian vibration is then extracted from Gaussian noises based on the probability density distribution. Simulations and experiments are carried out. The experimental results show that the demonstrated method can measure non-Gaussian vibrations with improved signal-to-noise ratio and spatial resolution.

eess.SP↗

arXiv2017

Manipulating Multiple Order Parameters via Oxygen Vacancies: The case of Eu0.5Ba0.5TiO3-δ

Controlling functionalities, such as magnetism or ferroelectricity, by means of oxygen vacancies (VO) is a key issue for the future development of transition metal oxides. Progress in this field is currently addressed through VO variations and their impact on mainly one order parameter. Here we reveal a new mechanism for tuning both magnetism and ferroelectricity simultaneously by using VO. Combined experimental and density-functional theory studies of Eu0.5Ba0.5TiO3-δ, we demonstrate that oxygen vacancies create Ti3+ 3d1 defect states, mediating the ferromagnetic coupling between the localized Eu 4f7 spins, and increase an off-center displacement of Ti ions, enhancing the ferroelectric Curie temperature. The dual function of Ti sites also promises a magnetoelectric coupling in the Eu0.5Ba0.5TiO3-δ.

cond-mat.mtrl-sci↗

arXiv2015

Generalized Cramer-Rao Bound for Joint Estimation of Target Position and Velocity for Active and Passive Radar Networks

In this paper, we derive the Cramer-Rao bound (CRB) for joint target position and velocity estimation using an active or passive distributed radar network under more general, and practically occurring, conditions than assumed in previous work. In particular, the presented results allow nonorthogonal signals, spatially dependent Gaussian reflection coefficients, and spatially dependent Gaussian clutter-plus-noise. These bounds allow designers to compare the performance of their developed approaches, which are deemed to be of acceptable complexity, to the best achievable performance. If their developed approaches lead to performance close to the bounds, these developed approaches can be deemed "good enough". A particular recent study where algorithms have been developed for a practical radar application which must involve nonorthogonal signals, for which the best performance is unknown, is a great example. The presented results in our paper do not make any assumptions about the approximate location of the target being known from previous target detection signal processing. In addition, for situations in which we do not know some parameters accurately, we also derive the mismatched CRB. Numerical investigations of the mean squared error of the maximum likelihood estimation are employed to support the validity of the CRBs. In order to demonstrate the utility of the provided results to a topic of great current interest, the numerical results focus on a passive radar system using the Global System for Mobile communication (GSM) cellar system.

math.ST↗

arXiv2015

HAADF-STEM Study of Mo/V Distributions in Mo-V-Te-Ta-O M1 Phases and Their Correlations with Surface Reactivity

The MoVTeTaO M1 phases were prepared by conventional hydrothermal (HT) and microwave-assisted HT synthesis methods (MW) employing two different Ta precursors, Ta ethoxide and a custom-made Ta oxalate complex. The profile intensity analysis of the HAADF-STEM image of M1 phases oriented along [hk0] directions from the surface to bulk region of HAADF-STEM images indicated that the chemical composition of surface ab planes is very similar to their composition in the bulk. The HAADF-STEM image analysis showed that synthesis methods have a significant impact on the Mo/V distribution in the MoVTeTaO M1 phases and their reactivity in propane ammoxidation. Enhanced acrylonitrile (ACN) yield and 1st order irreversible reaction rate constants for propane consumption, normalized to the estimated surface ab plane areas, correlated with increased V content in the proposed catalytic center (S2-S4-S4-S7-S7). These observations lend further support to the idea that multiple VOx sites present in the surface ab planes may be responsible for the activity and selectivity of the M1 phase in propane ammoxidation.

cond-mat.mtrl-sci↗

arXiv2012

On the relationship between cyclic and hierarchical three-species predator-prey systems and the two-species Lotka-Volterra model

We aim to clarify the relationship between interacting three-species models and the two-species Lotka-Volterra (LV) model. We utilize mean-field theory and Monte Carlo simulations on two-dimensional square lattices to explore the temporal evolution characteristics of two different interacting three-species predator-prey systems: (1) a cyclic rock-paper-scissors (RPS) model with conserved total particle number but strongly asymmetric reaction rates that lets the system evolve towards one corner of configuration space; (2) a hierarchical food chain where an additional intermediate species is inserted between the predator and prey in the LV model. For model variant (1), we demonstrate that the evolutionary properties of both minority species in the steady state of this stochastic spatial three-species corner RPS model are well approximated by the LV system, with its emerging characteristic features of localized population clustering, persistent oscillatory dynamics, correlated spatio-temporal patterns, and fitness enhancement through quenched spatial disorder in the predation rates. In contrast, we could not identify any regime where the hierarchical model (2) would reduce to the two-species LV system. In the presence of pair exchange processes, the system remains essentially well-mixed, and we generally find the Monte Carlo simulation results for the spatially extended model (2) to be consistent with the predictions from the corresponding mean-field rate equations. If spreading occurs only through nearest-neighbor hopping, small population clusters emerge; yet the requirement of an intermediate species cluster obviously disrupts spatio-temporal correlations between predator and prey, and correspondingly eliminates many of the intriguing fluctuation phenomena that characterize the stochastic spatial LV system.

q-bio.PE↗

arXiv2011

Co-existence in the two-dimensional May-Leonard model with random rates

We employ Monte Carlo simulations to numerically study the temporal evolution and transient oscillations of the population densities, the associated frequency power spectra, and the spatial correlation functions in the (quasi-)steady state in two-dimensional stochastic May--Leonard models of mobile individuals, allowing for particle exchanges with nearest-neighbors and hopping onto empty sites. We therefore consider a class of four-state three-species cyclic predator-prey models whose total particle number is not conserved. We demonstrate that quenched disorder in either the reaction or in the mobility rates hardly impacts the dynamical evolution, the emergence and structure of spiral patterns, or the mean extinction time in this system. We also show that direct particle pair exchange processes promote the formation of regular spiral structures. Moreover, upon increasing the rates of mobility, we observe a remarkable change in the extinction properties in the May--Leonard system (for small system sizes): (1) As the mobility rate exceeds a threshold that separates a species coexistence (quasi-)steady state from an absorbing state, the mean extinction time as function of system size N crosses over from a functional form ~ e^{cN} / N (where c is a constant) to a linear dependence; (2) the measured histogram of extinction times displays a corresponding crossover from an (approximately) exponential to a Gaussian distribution. The latter results are found to hold true also when the mobility rates are randomly distributed.

q-bio.PE↗

arXiv2010

Spatial Rock-Paper-Scissors Models with Inhomogeneous Reaction Rates

We study several variants of the stochastic four-state rock-paper-scissors game or, equivalently, cyclic three-species predator-prey models with conserved total particle density, by means of Monte Carlo simulations on one- and two-dimensional lattices. Specifically, we investigate the influence of spatial variability of the reaction rates and site occupancy restrictions on the transient oscillations of the species densities and on spatial correlation functions in the quasi-stationary coexistence state. For small systems, we also numerically determine the dependence of typical extinction times on the number of lattice sites. In stark contrast with two-species stochastic Lotka-Volterra systems, we find that for our three-species models with cyclic competition quenched disorder in the reaction rates has very little effect on the dynamics and the long-time properties of the coexistence state. Similarly, we observe that site restriction only has a minor influence on the system's dynamical properties. Our results therefore demonstrate that the features of the spatial rock-paper-scissors system are remarkably robust with respect to model variations, and stochastic fluctuations as well as spatial correlations play a comparatively minor role.

q-bio.PE↗