arXiv subjects

Tao Wang

Publications and source records attributed to Tao Wang.

At least 685 records · Page 38Linked to original sources

arXiv2021

Revisiting Knowledge Distillation via Label Smoothing Regularization

Knowledge Distillation (KD) aims to distill the knowledge of a cumbersome teacher model into a lightweight student model. Its success is generally attributed to the privileged information on similarities among categories provided by the teacher model, and in this sense, only strong teacher models are deployed to teach weaker students in practice. In this work, we challenge this common belief by following experimental observations: 1) beyond the acknowledgment that the teacher can improve the student, the student can also enhance the teacher significantly by reversing the KD procedure; 2) a poorly-trained teacher with much lower accuracy than the student can still improve the latter significantly. To explain these observations, we provide a theoretical analysis of the relationships between KD and label smoothing regularization. We prove that 1) KD is a type of learned label smoothing regularization and 2) label smoothing regularization provides a virtual teacher model for KD. From these results, we argue that the success of KD is not fully due to the similarity information between categories from teachers, but also to the regularization of soft targets, which is equally or even more important. Based on these analyses, we further propose a novel Teacher-free Knowledge Distillation (Tf-KD) framework, where a student model learns from itself or manuallydesigned regularization distribution. The Tf-KD achieves comparable performance with normal KD from a superior teacher, which is well applied when a stronger teacher model is unavailable. Meanwhile, Tf-KD is generic and can be directly deployed for training deep neural networks. Without any extra computation cost, Tf-KD achieves up to 0.65\% improvement on ImageNet over well-established baseline models, which is superior to label smoothing regularization.

cs.CV↗

arXiv2021

A Universal Model for Cross Modality Mapping by Relational Reasoning

With the aim of matching a pair of instances from two different modalities, cross modality mapping has attracted growing attention in the computer vision community. Existing methods usually formulate the mapping function as the similarity measure between the pair of instance features, which are embedded to a common space. However, we observe that the relationships among the instances within a single modality (intra relations) and those between the pair of heterogeneous instances (inter relations) are insufficiently explored in previous approaches. Motivated by this, we redefine the mapping function with relational reasoning via graph modeling, and further propose a GCN-based Relational Reasoning Network (RR-Net) in which inter and intra relations are efficiently computed to universally resolve the cross modality mapping problem. Concretely, we first construct two kinds of graph, i.e., Intra Graph and Inter Graph, to respectively model intra relations and inter relations. Then RR-Net updates all the node features and edge features in an iterative manner for learning intra and inter relations simultaneously. Last, RR-Net outputs the probabilities over the edges which link a pair of heterogeneous instances to estimate the mapping results. Extensive experiments on three example tasks, i.e., image classification, social recommendation and sound recognition, clearly demonstrate the superiority and universality of our proposed model.

cs.CV↗

arXiv2021

Regional and Sectoral Structures and Their Dynamics of Chinese Economy: A Network Perspective from Multi-Regional Input-Output Tables

A multi-regional input-output table (MRIOT) containing the transactions among the region-sectors in an economy defines a weighted and directed network. Using network analysis tools, we analyze the regional and sectoral structure of the Chinese economy and their temporal dynamics from 2007 to 2012 via the MRIOTs of China. Global analyses are done with network topology measures. Growth-driving province-sector clusters are identified with community detection methods. Influential province-sectors are ranked by weighted PageRank scores. The results revealed a few interesting and telling insights. The level of inter-province-sector activities increased with the rapid growth of the national economy, but not as fast as that of intra-province economic activities. Regional community structures were deeply associated with geographical factors. The community heterogeneity across the regions was high and the regional fragmentation increased during the study period. Quantified metrics assessing the relative importance of the province-sectors in the national economy echo the national and regional economic development policies to a certain extent.

physics.soc-ph↗

arXiv2021

Huge permittivity and premature metallicity in Bi$_2$O$_2$Se single crystals

Bi$_2$O$_2$Se is a promising material for next-generation semiconducting electronics. It exhibits premature metallicity on the introduction of a tiny amount of electrons, the physics behind which remains elusive. Here we report on transport and dielectric measurements in Bi$_2$O$_2$Se single crystals at various carrier densities. The temperature-dependent resistivity ($ρ$) indicates a smooth evolution from the semiconducting to the metallic state. The critical concentration for the metal-insulator transition (MIT) to occur is extraordinarily low ($n_\textrm{c}\sim10^{16}$ cm$^{-3}$). The relative permittivity of the insulating sample is huge ($ε_\textrm{r}\approx155(10)$) and varies slowly with temperature. Combined with the light effective mass, a long effective Bohr radius ($a_\textrm{B}^*\approx36(2)$ $\textrm{nm}$) is derived, which provides a reasonable interpretation of the metallic prematurity according to Mott's criterion for MITs. The high electron mobility ($μ$) at low temperatures may result from the screening of ionized scattering centers due to the huge $ε_\textrm{r}$. Our findings shed light on the electron dynamics in two dimensional (2D) Bi$_2$O$_2$Se devices.

cond-mat.mtrl-sci↗

arXiv2021

Attention Models for Point Clouds in Deep Learning: A Survey

Recently, the advancement of 3D point clouds in deep learning has attracted intensive research in different application domains such as computer vision and robotic tasks. However, creating feature representation of robust, discriminative from unordered and irregular point clouds is challenging. In this paper, our ultimate goal is to provide a comprehensive overview of the point clouds feature representation which uses attention models. More than 75+ key contributions in the recent three years are summarized in this survey, including the 3D objective detection, 3D semantic segmentation, 3D pose estimation, point clouds completion etc. We provide a detailed characterization (1) the role of attention mechanisms, (2) the usability of attention models into different tasks, (3) the development trend of key technology.

cs.CV↗

arXiv2021

DAN-Net: Dual-Domain Adaptive-Scaling Non-local Network for CT Metal Artifact Reduction

Metal implants can heavily attenuate X-rays in computed tomography (CT) scans, leading to severe artifacts in reconstructed images, which significantly jeopardize image quality and negatively impact subsequent diagnoses and treatment planning. With the rapid development of deep learning in the field of medical imaging, several network models have been proposed for metal artifact reduction (MAR) in CT. Despite the encouraging results achieved by these methods, there is still much room to further improve performance. In this paper, a novel Dual-domain Adaptive-scaling Non-local network (DAN-Net) for MAR. We correct the corrupted sinogram using adaptive scaling first to preserve more tissue and bone details as a more informative input. Then, an end-to-end dual-domain network is adopted to successively process the sinogram and its corresponding reconstructed image generated by the analytical reconstruction layer. In addition, to better suppress the existing artifacts and restrain the potential secondary artifacts caused by inaccurate results of the sinogram-domain network, a novel residual sinogram learning strategy and nonlocal module are leveraged in the proposed network model. In the experiments, the proposed DAN-Net demonstrates performance competitive with several state-of-the-art MAR methods in both qualitative and quantitative aspects.

physics.med-ph↗

arXiv2021

Accelerated galaxy growth and environmental quenching in a protocluster at z=3.24

We present a multiwavelength study of galaxies around D4UD01, a spectroscopically confirmed protocluster at z = 3.24 to investigate environmental trends. 450 galaxies are selected based on Ks band detection with photometric redshifts (photo-z) at 3.0 < z < 3.4, among which ~ 12% are classified as quiescent galaxies. The quiescent galaxies are among the most massive and reddest ones in the entire sample. We identify a large photo-z galaxy overdensity in the field, which lies close to the previously spectroscopically confirmed sources of the protocluster. We find that the quiescent galaxies are largely concentrated in the overdense protocluster region with a higher quiescent fraction, showing a sign of environmental quenching. Galaxies in the protocluster are forming faster than the field counterparts as seen in the stellar mass function, suggesting early and accelerated mass assembly in the overdense regions. Although weak evidence of suppressed star-formation is found in the protocluster, the statistics are not significant enough to draw a definite conclusion. Our work shed light on how the formation of massive galaxies is affected in the dense region of a protocluster when the Universe was only 2 Gyr old.

astro-ph.GA↗

arXiv2021

Rapid Robust Principal Component Analysis: CUR Accelerated Inexact Low Rank Estimation

Robust principal component analysis (RPCA) is a widely used tool for dimension reduction. In this work, we propose a novel non-convex algorithm, coined Iterated Robust CUR (IRCUR), for solving RPCA problems, which dramatically improves the computational efficiency in comparison with the existing algorithms. IRCUR achieves this acceleration by employing CUR decomposition when updating the low rank component, which allows us to obtain an accurate low rank approximation via only three small submatrices. Consequently, IRCUR is able to process only the small submatrices and avoid expensive computing on the full matrix through the entire algorithm. Numerical experiments establish the computational advantage of IRCUR over the state-of-art algorithms on both synthetic and real-world datasets.

stat.ML↗

arXiv2021

You Recommend, I Buy: How and Why People Engage in Instant Messaging Based Social Commerce

As an emerging business phenomenon especially in China, instant messaging (IM) based social commerce is growing increasingly popular, attracting hundreds of millions of users and is becoming one important way where people make everyday purchases. Such platforms embed shopping experiences within IM apps, e.g., WeChat, WhatsApp, where real-world friends post and recommend products from the platforms in IM group chats and quite often form lasting recommending/buying relationships. How and why do users engage in IM based social commerce? Do such platforms create novel experiences that are distinct from prior commerce? And do these platforms bring changes to user social lives and relationships? To shed light on these questions, we launched a qualitative study where we carried out semi-structured interviews on 12 instant messaging based social commerce users in China. We showed that IM based social commerce: 1) enables more reachable, cost-reducing, and immersive user shopping experience, 2) shapes user decision-making process in shopping through pre-existing social relationship, mutual trust, shared identity, and community norm, and 3) creates novel social interactions, which can contribute to new tie formation while maintaining existing social relationships. We demonstrate that all these unique aspects link closely to the characteristics of IM platforms, as well as the coupling of user social and economic lives under such business model. Our study provides important research and design implications for social commerce, and decentralized, trusted socio-technical systems in general.

cs.CY↗

arXiv2021

Mobility-Aware Seamless Handover with MPTCP in Software-Defined HetNets

In this paper, the problem of vertical handover in software-defined network (SDN) based heterogeneous networks (HetNets) is studied. In the studied model, HetNets are required to offer diverse services for mobile users. Using an SDN controller, HetNets have the capability of managing users' access and mobility issues but still have the problems of ping-pong effect and service interruption during vertical handover. To solve these problems, a mobility-aware seamless handover method based on multipath transmission control protocol (MPTCP) is proposed. The proposed handover method is executed in the controller of the software-defined HetNets (SDHetNets) and consists of three steps: location prediction, network selection, and handover execution. In particular, the method first predicts the user's location in the next moment with an echo state network (ESN). Given the predicted location, the SDHetNet controller can determine the candidate network set for the handover to pre-allocate network wireless resources. Second, the target network is selected through fuzzy analytic hierarchical process (FAHP) algorithm, jointly considering user preferences, service requirements, network attributes, and user mobility patterns. Then, seamless handover is realized through the proposed MPTCP-based handover mechanism. Simulations using real-world user trajectory data from Korea Advanced Institute of Science & Technology show that the proposed method can reduce the handover times by 10.85% to 29.12% compared with traditional methods. The proposed method also maintains at least one MPTCP subflow connected during the handover process and achieves a seamless handover.

cs.NI↗

arXiv2020

Large-Scale Discrete Fourier Transform on TPUs

In this work, we present two parallel algorithms for the large-scale discrete Fourier transform (DFT) on Tensor Processing Unit (TPU) clusters. The two parallel algorithms are associated with two formulations of DFT: one is based on the Kronecker product, to be specific, dense matrix multiplications between the input data and the Vandermonde matrix, denoted as KDFT in this work; the other is based on the famous Cooley-Tukey algorithm and phase adjustment, denoted as FFT in this work. Both KDFT and FFT formulations take full advantage of TPU's strength in matrix multiplications. The KDFT formulation allows direct use of nonuniform inputs without additional step. In the two parallel algorithms, the same strategy of data decomposition is applied to the input data. Through the data decomposition, the dense matrix multiplications in KDFT and FFT are kept local within TPU cores, which can be performed completely in parallel. The communication among TPU cores is achieved through the one-shuffle scheme in both parallel algorithms, with which sending and receiving data takes place simultaneously between two neighboring cores and along the same direction on the interconnect network. The one-shuffle scheme is designed for the interconnect topology of TPU clusters, minimizing the time required by the communication among TPU cores. Both KDFT and FFT are implemented in TensorFlow. The three-dimensional complex DFT is performed on an example of dimension $8192 \times 8192 \times 8192$ with a full TPU Pod: the run time of KDFT is 12.66 seconds and that of FFT is 8.3 seconds. Scaling analysis is provided to demonstrate the high parallel efficiency of the two DFT implementations on TPUs.

cs.MS↗

arXiv2020

An Investigation of Potential Function Designs for Neural CRF

The neural linear-chain CRF model is one of the most widely-used approach to sequence labeling. In this paper, we investigate a series of increasingly expressive potential functions for neural CRF models, which not only integrate the emission and transition functions, but also explicitly take the representations of the contextual words as input. Our extensive experiments show that the decomposed quadrilinear potential function based on the vector representations of two neighboring labels and two neighboring words consistently achieves the best performance.

cs.CL↗

arXiv2020

The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation

Most existing object instance detection and segmentation models only work well on fairly balanced benchmarks where per-category training sample numbers are comparable, such as COCO. They tend to suffer performance drop on realistic datasets that are usually long-tailed. This work aims to study and address such open challenges. Specifically, we systematically investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset, and unveil that a major cause is the inaccurate classification of object proposals. Based on such an observation, we first consider various techniques for improving long-tail classification performance which indeed enhance instance segmentation results. We then propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach. Without bells and whistles, it significantly boosts the performance of instance segmentation for tail classes on the recent LVIS dataset and our sampled COCO-LT dataset. Our analysis provides useful insights for solving long-tail instance detection and segmentation problems, and the straightforward \emph{SimCal} method can serve as a simple but strong baseline. With the method we have won the 2019 LVIS challenge. Codes and models are available at https://github.com/twangnh/SimCal.

cs.CV↗

arXiv2020

EDCNN: Edge enhancement-based Densely Connected Network with Compound Loss for Low-Dose CT Denoising

In the past few decades, to reduce the risk of X-ray in computed tomography (CT), low-dose CT image denoising has attracted extensive attention from researchers, which has become an important research issue in the field of medical images. In recent years, with the rapid development of deep learning technology, many algorithms have emerged to apply convolutional neural networks to this task, achieving promising results. However, there are still some problems such as low denoising efficiency, over-smoothed result, etc. In this paper, we propose the Edge enhancement based Densely connected Convolutional Neural Network (EDCNN). In our network, we design an edge enhancement module using the proposed novel trainable Sobel convolution. Based on this module, we construct a model with dense connections to fuse the extracted edge information and realize end-to-end image denoising. Besides, when training the model, we introduce a compound loss that combines MSE loss and multi-scales perceptual loss to solve the over-smoothed problem and attain a marked improvement in image quality after denoising. Compared with the existing low-dose CT image denoising algorithms, our proposed model has a better performance in preserving details and suppressing noise.

eess.IV↗

arXiv2020

Star edge-coloring of some special graphs

The star chromatic index of a multigraph $G$, denoted by $χ_{\mathrm{star}}'(G)$, is the minimum number of colors needed to properly color the edges of $G$ such that no path or cycle of length $4$ is bicolored. In this paper, we study the star edge-coloring of Halin graphs, $k$-power graphs and the generalized Petersen graphs $P(3n, n)$.

math.CO↗

arXiv2020

Gauntlet: Finding Bugs in Compilers for Programmable Packet Processing

Programmable packet-processing devices such as programmable switches and network interface cards are becoming mainstream. These devices are configured in a domain-specific language such as P4, using a compiler to translate packet-processing programs into instructions for different targets. As networks with programmable devices become widespread, it is critical that these compilers be dependable. This paper considers the problem of finding bugs in compilers for packet processing in the context of P4-16. We introduce domain-specific techniques to induce both abnormal termination of the compiler (crash bugs) and miscompilation (semantic bugs). We apply these techniques to (1) the open-source P4 compiler (P4C) infrastructure, which serves as a common base for different P4 back ends; (2) the P4 back end for the P4 reference software switch; and (3) the P4 back end for the Barefoot Tofino switch. Across the 3 platforms, over 8 months of bug finding, our tool Gauntlet detected 96 new and distinct bugs (62 crash and 34 semantic), which we confirmed with the respective compiler developers. 54 have been fixed (31 crash and 23 semantic); the remaining have been assigned to a developer. Our bug-finding efforts also led to 6 P4 specification changes. We have open sourced Gauntlet at p4gauntlet.github.io and it now runs within P4C's continuous integration pipeline.

cs.NI↗

arXiv2020

Ferromagnetic Gyroscopes for Tests of Fundamental Physics

A ferromagnetic gyroscope (FG) is a ferromagnet whose angular momentum is dominated by electron spin polarization and that will precess under the action of an external torque, such as that due to a magnetic field. Here we model and analyze FG dynamics and sensitivity, focusing on practical schemes for experimental realization. In the case of a freely floating FG, we model the transition from dynamics dominated by libration in relatively high externally applied magnetic fields, to those dominated by precession at relatively low applied fields. Measurement of the libration frequency enables in situ measurement of the magnetic field and a technique to reduce the field below the threshold for which precession dominates the FG dynamics. We note that evidence of gyroscopic behavior is present even at magnetic fields much larger than the threshold field below which precession dominates. We also model the dynamics of an FG levitated above a type-I superconductor via the Meissner effect, and find that for FGs with dimensions larger than about 100 nm the observed precession frequency is reduced compared to that of a freely floating FG. This is akin to negative feedback that arises from the distortion of the field from the FG by the superconductor. Finally we assess the sensitivity of an FG levitated above a type-I superconductor to exotic spin-dependent interactions under practical experimental conditions, demonstrating the potential of FGs for tests of fundamental physics.

quant-ph↗

arXiv2020

Toward Accurate Person-level Action Recognition in Videos of Crowded Scenes

Detecting and recognizing human action in videos with crowded scenes is a challenging problem due to the complex environment and diversity events. Prior works always fail to deal with this problem in two aspects: (1) lacking utilizing information of the scenes; (2) lacking training data in the crowd and complex scenes. In this paper, we focus on improving spatio-temporal action recognition by fully-utilizing the information of scenes and collecting new data. A top-down strategy is used to overcome the limitations. Specifically, we adopt a strong human detector to detect the spatial location of each frame. We then apply action recognition models to learn the spatio-temporal information from video frames on both the HIE dataset and new data with diverse scenes from the internet, which can improve the generalization ability of our model. Besides, the scenes information is extracted by the semantic segmentation model to assistant the process. As a result, our method achieved an average 26.05 wf\_mAP (ranking 1st place in the ACM MM grand challenge 2020: Human in Events).

cs.CV↗