arXiv subjects

Xinyu Zhang

Publications and source records attributed to Xinyu Zhang.

At least 433 records · Page 24Linked to original sources

arXiv2021

Early Exiting with Ensemble Internal Classifiers

As a simple technique to accelerate inference of large-scale pre-trained models, early exiting has gained much attention in the NLP community. It allows samples to exit early at internal classifiers without passing through the entire model. Most existing work usually trains the internal classifiers independently and employs an exiting strategy to decide whether or not to exit based on the confidence of the current internal classifier. However, none of these works takes full advantage of the fact that the internal classifiers are trained to solve the same task therefore can be used to construct an ensemble. In this paper, we show that a novel objective function for the training of the ensemble internal classifiers can be naturally induced from the perspective of ensemble learning and information theory. The proposed training objective consists of two terms: one for accuracy and the other for the diversity of the internal classifiers. In contrast, the objective used in prior work is exactly the accuracy term of our training objective therefore only optimizes the accuracy but not diversity. Further, we propose a simple voting-based strategy that considers predictions of all the past internal classifiers to infer the correct label and decide whether to exit. Experimental results on various NLP tasks show that our proposed objective function and voting-based strategy can achieve better accuracy-speed trade-offs.

cs.CL↗

arXiv2021

Emotion Eliciting Machine: Emotion Eliciting Conversation Generation based on Dual Generator

Recent years have witnessed great progress on building emotional chatbots. Tremendous methods have been proposed for chatbots to generate responses with given emotions. However, the emotion changes of the user during the conversation has not been fully explored. In this work, we study the problem of positive emotion elicitation, which aims to generate responses that can elicit positive emotion of the user, in human-machine conversation. We propose a weakly supervised Emotion Eliciting Machine (EEM) to address this problem. Specifically, we first collect weak labels of user emotion status changes in a conversion based on a pre-trained emotion classifier. Then we propose a dual encoder-decoder structure to model the generation of responses in both positive and negative side based on the changes of the user's emotion status in the conversation. An emotion eliciting factor is introduced on top of the dual structure to balance the positive and negative emotional impacts on the generated response during emotion elicitation. The factor also provides a fine-grained controlling manner for emotion elicitation. Experimental results on a large real-world dataset show that EEM outperforms the existing models in generating responses with positive emotion elicitation.

cs.CL↗

arXiv2021

A novel multimodal fusion network based on a joint coding model for lane line segmentation

There has recently been growing interest in utilizing multimodal sensors to achieve robust lane line segmentation. In this paper, we introduce a novel multimodal fusion architecture from an information theory perspective, and demonstrate its practical utility using Light Detection and Ranging (LiDAR) camera fusion networks. In particular, we develop, for the first time, a multimodal fusion network as a joint coding model, where each single node, layer, and pipeline is represented as a channel. The forward propagation is thus equal to the information transmission in the channels. Then, we can qualitatively and quantitatively analyze the effect of different fusion approaches. We argue the optimal fusion architecture is related to the essential capacity and its allocation based on the source and channel. To test this multimodal fusion hypothesis, we progressively determine a series of multimodal models based on the proposed fusion methods and evaluate them on the KITTI and the A2D2 datasets. Our optimal fusion network achieves 85%+ lane line accuracy and 98.7%+ overall. The performance gap among the models will inform continuing future research into development of optimal fusion algorithms for the deep multimodal learning community.

cs.CV↗

arXiv2021

S-AT GCN: Spatial-Attention Graph Convolution Network based Feature Enhancement for 3D Object Detection

3D object detection plays a crucial role in environmental perception for autonomous vehicles, which is the prerequisite of decision and control. This paper analyses partition-based methods' inherent drawbacks. In the partition operation, a single instance such as a pedestrian is sliced into several pieces, which we call it the partition effect. We propose the Spatial-Attention Graph Convolution (S-AT GCN), forming the Feature Enhancement (FE) layers to overcome this drawback. The S-AT GCN utilizes the graph convolution and the spatial attention mechanism to extract local geometrical structure features. This allows the network to have more meaningful features for the foreground. Our experiments on the KITTI 3D object and bird's eye view detection show that S-AT Conv and FE layers are effective, especially for small objects. FE layers boost the pedestrian class performance by 3.62\% and cyclist class by 4.21\% 3D mAP. The time cost of these extra FE layers are limited. PointPillars with FE layers can achieve 48 PFS, satisfying the real-time requirement.

cs.CV↗

arXiv2021

DNA: Deeply-supervised Nonlinear Aggregation for Salient Object Detection

Recent progress on salient object detection mainly aims at exploiting how to effectively integrate multi-scale convolutional features in convolutional neural networks (CNNs). Many popular methods impose deep supervision to perform side-output predictions that are linearly aggregated for final saliency prediction. In this paper, we theoretically and experimentally demonstrate that linear aggregation of side-output predictions is suboptimal, and it only makes limited use of the side-output information obtained by deep supervision. To solve this problem, we propose Deeply-supervised Nonlinear Aggregation (DNA) for better leveraging the complementary information of various side-outputs. Compared with existing methods, it i) aggregates side-output features rather than predictions, and ii) adopts nonlinear instead of linear transformations. Experiments demonstrate that DNA can successfully break through the bottleneck of current linear approaches. Specifically, the proposed saliency detector, a modified U-Net architecture with DNA, performs favorably against state-of-the-art methods on various datasets and evaluation metrics without bells and whistles.

cs.CV↗

arXiv2021

A New Window Loss Function for Bone Fracture Detection and Localization in X-ray Images with Point-based Annotation

Object detection methods are widely adopted for computer-aided diagnosis using medical images. Anomalous findings are usually treated as objects that are described by bounding boxes. Yet, many pathological findings, e.g., bone fractures, cannot be clearly defined by bounding boxes, owing to considerable instance, shape and boundary ambiguities. This makes bounding box annotations, and their associated losses, highly ill-suited. In this work, we propose a new bone fracture detection method for X-ray images, based on a labor effective and flexible annotation scheme suitable for abnormal findings with no clear object-level spatial extents or boundaries. Our method employs a simple, intuitive, and informative point-based annotation protocol to mark localized pathology information. To address the uncertainty in the fracture scales annotated via point(s), we convert the annotations into pixel-wise supervision that uses lower and upper bounds with positive, negative, and uncertain regions. A novel Window Loss is subsequently proposed to only penalize the predictions outside of the uncertain regions. Our method has been extensively evaluated on 4410 pelvic X-ray images of unique patients. Experiments demonstrate that our method outperforms previous state-of-the-art image classification and object detection baselines by healthy margins, with an AUROC of 0.983 and FROC score of 89.6%.

cs.CV↗

arXiv2020

Diverse Knowledge Distillation for End-to-End Person Search

Person search aims to localize and identify a specific person from a gallery of images. Recent methods can be categorized into two groups, i.e., two-step and end-to-end approaches. The former views person search as two independent tasks and achieves dominant results using separately trained person detection and re-identification (Re-ID) models. The latter performs person search in an end-to-end fashion. Although the end-to-end approaches yield higher inference efficiency, they largely lag behind those two-step counterparts in terms of accuracy. In this paper, we argue that the gap between the two kinds of methods is mainly caused by the Re-ID sub-networks of end-to-end methods. To this end, we propose a simple yet strong end-to-end network with diverse knowledge distillation to break the bottleneck. We also design a spatial-invariant augmentation to assist model to be invariant to inaccurate detection results. Experimental results on the CUHK-SYSU and PRW datasets demonstrate the superiority of our method against existing approaches -- it achieves on par accuracy with state-of-the-art two-step methods while maintaining high efficiency due to the single joint model. Code is available at: https://git.io/DKD-PersonSearch.

cs.CV↗

arXiv2020

Peer-to-Peer Localization for Single-Antenna Devices

Some important indoor localization applications, such as localizing a lost kid in a shopping mall, call for a new peer-to-peer localization technique that can localize an individual's smartphone or wearables by directly using another's on-body devices in unknown indoor environments. However, current localization solutions either require pre-deployed infrastructures or multiple antennas in both transceivers, impending their wide-scale application. In this paper, we present P2PLocate, a peer-to-peer localization system that enables a single-antenna device co-located with a batteryless backscatter tag to localize another single-antenna device with decimeter-level accuracy. P2PLocate leverages the multipath variations intentionally created by an on-body backscatter tag, coupled with spatial information offered by user movements, to accomplish this objective without relying on any pre-deployed infrastructures or pre-training. P2PLocate incorporates novel algorithms to address two major challenges: (i) interference with strong direct-path signal while extracting multipath variations, and (ii) lack of direction information while using single-antenna transceivers. We implement P2PLocate on commercial off-the-shelf Google Nexus 6p, Intel 5300 WiFi card, and Raspberry Pi B4. Real-world experiments reveal that P2PLocate can localize both static and mobile targets with a median accuracy of 0.88 m.

eess.SP↗

arXiv2020

Part-Guided Attention Learning for Vehicle Instance Retrieval

Vehicle instance retrieval often requires one to recognize the fine-grained visual differences between vehicles. Besides the holistic appearance of vehicles which is easily affected by the viewpoint variation and distortion, vehicle parts also provide crucial cues to differentiate near-identical vehicles. Motivated by these observations, we introduce a Part-Guided Attention Network (PGAN) to pinpoint the prominent part regions and effectively combine the global and part information for discriminative feature learning. PGAN first detects the locations of different part components and salient regions regardless of the vehicle identity, which serve as the bottom-up attention to narrow down the possible searching regions. To estimate the importance of detected parts, we propose a Part Attention Module (PAM) to adaptively locate the most discriminative regions with high-attention weights and suppress the distraction of irrelevant parts with relatively low weights. The PAM is guided by the instance retrieval loss and therefore provides top-down attention that enables attention to be calculated at the level of car parts and other salient regions. Finally, we aggregate the global appearance and part features to improve the feature performance further. The PGAN combines part-guided bottom-up and top-down attention, global and part visual features in an end-to-end framework. Extensive experiments demonstrate that the proposed method achieves new state-of-the-art vehicle instance retrieval performance on four large-scale benchmark datasets.

cs.CV↗

arXiv2020

Electronic structure of pristine and Ni-substituted LaFeO$_3$ from near edge x-ray absorption fine structure experiments and first-principles simulations

We present a joint theoretical and experimental study of the oxygen $K$-edge spectra for LaFeO$_3$ and homovalent Ni-substituted LaFeO$_3$ (LaFe$_{0.75}$Ni$_{0.25}$O$_3$), using first-principles simulations based on density-functional theory with extended Hubbard functionals and x-ray absorption near edge structure (XANES) measurements. Ground-state and excited-state XANES calculations employ Hubbard on-site $U$ and inter-site $V$ parameters determined from first principles and the Lanczos recursive method to obtain absorption cross sections, which allows for a reliable description of XANES spectra in transition-metal compounds in a very broad energy range, with an accuracy comparable to that of hybrid functionals but at a substantially lower cost. We show that standard gradient-corrected exchange-correlation functionals fail in capturing accurately the electronic properties of both materials. In particular, for LaFe$_{0.75}$Ni$_{0.25}$O$_3$ they do not reproduce its semiconducting behaviour and provide a poor description of the pre-edge features at the O $K$ edge. The inclusion of Hubbard interactions leads to a drastic improvement, accounting for the semiconducting ground state of LaFe$_{0.75}$Ni$_{0.25}$O$_3$ and for a good agreement between calculated and measured XANES spectra. We show that the partial substitution of Fe for Ni affects the conduction-band bottom by generating a strongly hybridized O($2p$)-Ni($3d$) minority-spin empty electronic state. The present work, based on a consistent correction of self-interaction errors, outlines the crucial role of extended Hubbard functionals to describe the electronic structure of complex transition-metal oxides such as LaFeO$_3$ and LaFe$_{0.75}$Ni$_{0.25}$O$_3$ and paves the way to future studies on similar systems.

cond-mat.str-el↗

arXiv2020

Error Bounds for Generalized Group Sparsity

In high-dimensional statistical inference, sparsity regularizations have shown advantages in consistency and convergence rates for coefficient estimation. We consider a generalized version of Sparse-Group Lasso which captures both element-wise sparsity and group-wise sparsity simultaneously. We state one universal theorem which is proved to obtain results on consistency and convergence rates for different forms of double sparsity regularization. The universality of the results lies in an generalization of various convergence rates for single regularization cases such as LASSO and group LASSO and also double regularization cases such as sparse-group LASSO. Our analysis identifies a generalized norm of $ε$-norm, which provides a dual formulation for our double sparsity regularization.

stat.ML↗

arXiv2020

Towards Embodied Scene Description

Embodiment is an important characteristic for all intelligent agents (creatures and robots), while existing scene description tasks mainly focus on analyzing images passively and the semantic understanding of the scenario is separated from the interaction between the agent and the environment. In this work, we propose the Embodied Scene Description, which exploits the embodiment ability of the agent to find an optimal viewpoint in its environment for scene description tasks. A learning framework with the paradigms of imitation learning and reinforcement learning is established to teach the intelligent agent to generate corresponding sensorimotor activities. The proposed framework is tested on both the AI2Thor dataset and a real world robotic platform demonstrating the effectiveness and extendability of the developed method.

cs.RO↗

arXiv2020

Effective gravitational couplings of four-dimensional $\mathcal{N}=2$ supersymmetric gauge theories

The low energy effective couplings of a four-dimensional $\mathcal{N}=2$ supersymmetric gauge theory to topological invariants of the background gravitational field are described by two functions $A$ and $B$. These two functions play an important role in the study of topologically twisted four-dimensional $\mathcal{N}=2$ supersymmetric gauge theories and in the computation of central charges of $\mathcal{N}=2$ superconformal theories. In this paper, we compute $A$ and $B$ from the partition function in the $Ω$-background for $\mathrm{SU}(2)$ gauge theories. Our results not only confirm the predicted expressions of the effective gravitational couplings, but also give the previously undetermined overall multiplicative factors. We also analyze $A$ and $B$ for the $\mathrm{SU}(N)$ super-Yang-Mills theory, and confirm all the previous predictions.

hep-th↗

arXiv2020

Thermal conductivity and Lorenz ratio of metals at intermediate temperature: a first-principles analysis

Electronic and phononic thermal conductivity are involved in the thermal conduction for metals and Wiedemann-Franz law is usually employed to predict them separately. However, Wiedemann-Franz law is shown to be invalid at intermediate temperatures. Here, to obtain the accurate thermal conductivity and Lorenz ratio for metals, the momentum relaxation time is used for electrical conductivity and energy relaxation time for electronic thermal conductivity. The mode-level first-principles calculation is conducted on two representative metals copper and aluminum. It is shown that the method can correctly predict electrical transport coefficients from 6 to 300 K. Also, the anomalous Lorenz ratio is observed within the present scheme, which has significant departure from the Sommerfeld value. The calculation scheme can be expanded to other metallic systems and is valuable in a better understanding of the electron dynamics and transport properties of metals.

physics.app-ph↗

arXiv2020

A note on chiral trace relations from qq-characters

We study chiral operators in four-dimensional N=2 supersymmetric gauge theories. We provide a general procedure to derive the exact relations among the vacuum expectation values of chiral operators in the Omega-background using the non-perturbative Dyson-Schwinger equations of qq-characters. We demonstrate our procedure using several examples. For pure SU(N) gauge theory and SU(N) gauge theory with 2N fundamental hypermultiplets, we verify the relations conjectured previously in the literature. We also briefly discuss the relations in SU(N) linear superconformal quiver gauge theories.

hep-th↗

arXiv2020

Dual-discriminator GAN: A GAN way of profile face recognition

A wealth of angle problems occur when facial recognition is performed: At present, the feature extraction network presents eigenvectors with large differences between the frontal face and profile face recognition of the same person in many cases. For this reason, the state-of-the-art facial recognition network will use multiple samples for the same target to ensure that eigenvector differences caused by angles are ignored during training. However, there is another solution available, which is to generate frontal face images with profile face images before recognition. In this paper, we proposed a method of generating frontal faces with image-to-image profile faces based on Generative Adversarial Network (GAN).

cs.CV↗

arXiv2020

DeepDualMapper: A Gated Fusion Network for Automatic Map Extraction using Aerial Images and Trajectories

Automatic map extraction is of great importance to urban computing and location-based services. Aerial image and GPS trajectory data refer to two different data sources that could be leveraged to generate the map, although they carry different types of information. Most previous works on data fusion between aerial images and data from auxiliary sensors do not fully utilize the information of both modalities and hence suffer from the issue of information loss. We propose a deep convolutional neural network called DeepDualMapper which fuses the aerial image and trajectory data in a more seamless manner to extract the digital map. We design a gated fusion module to explicitly control the information flows from both modalities in a complementary-aware manner. Moreover, we propose a novel densely supervised refinement decoder to generate the prediction in a coarse-to-fine way. Our comprehensive experiments demonstrate that DeepDualMapper can fuse the information of images and trajectories much more effectively than existing approaches, and is able to generate maps with higher accuracy.

cs.CV↗

arXiv2020

Memorizing Comprehensively to Learn Adaptively: Unsupervised Cross-Domain Person Re-ID with Multi-level Memory

Unsupervised cross-domain person re-identification (Re-ID) aims to adapt the information from the labelled source domain to an unlabelled target domain. Due to the lack of supervision in the target domain, it is crucial to identify the underlying similarity-and-dissimilarity relationships among the unlabelled samples in the target domain. In order to use the whole data relationships efficiently in mini-batch training, we apply a series of memory modules to maintain an up-to-date representation of the entire dataset. Unlike the simple exemplar memory in previous works, we propose a novel multi-level memory network (MMN) to discover multi-level complementary information in the target domain, relying on three memory modules, i.e., part-level memory, instance-level memory, and domain-level memory. The proposed memory modules store multi-level representations of the target domain, which capture both the fine-grained differences between images and the global structure for the holistic target domain. The three memory modules complement each other and systematically integrate multi-level supervision from bottom to up. Experiments on three datasets demonstrate that the multi-level memory modules cooperatively boost the unsupervised cross-domain Re-ID task, and the proposed MMN achieves competitive results.

cs.CV↗