arXiv subjects

Tao Zhou

Publications and source records attributed to Tao Zhou.

At least 253 records · Page 14Linked to original sources

arXiv2022

Multi-class Label Noise Learning via Loss Decomposition and Centroid Estimation

In real-world scenarios, many large-scale datasets often contain inaccurate labels, i.e., noisy labels, which may confuse model training and lead to performance degradation. To overcome this issue, Label Noise Learning (LNL) has recently attracted much attention, and various methods have been proposed to design an unbiased risk estimator to the noise-free dataset to combat such label noise. Among them, a trend of works based on Loss Decomposition and Centroid Estimation (LDCE) has shown very promising performance. However, existing LNL methods based on LDCE are only designed for binary classification, and they are not directly extendable to multi-class situations. In this paper, we propose a novel multi-class robust learning method for LDCE, which is termed "MC-LDCE". Specifically, we decompose the commonly adopted loss (e.g., mean squared loss) function into a label-dependent part and a label-independent part, in which only the former is influenced by label noise. Further, by defining a new form of data centroid, we transform the recovery problem of a label-dependent part to a centroid estimation problem. Finally, by critically examining the mathematical expectation of clean data centroid given the observed noisy set, the centroid can be estimated which helps to build an unbiased risk estimator for multi-class learning. The proposed MC-LDCE method is general and applicable to different types (i.e., linear and nonlinear) of classification models. The experimental results on five public datasets demonstrate the superiority of the proposed MC-LDCE against other representative LNL methods in tackling multi-class label noise problem.

cs.LG↗

arXiv2022

On Vision Features in Multimodal Machine Translation

Previous work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is on the quality of vision models. In this work, we investigate the impact of vision models on MMT. Given the fact that Transformer is becoming popular in computer vision, we experiment with various strong models (such as Vision Transformer) and enhanced features (such as object-detection and image captioning). We develop a selective attention model to study the patch-level contribution of an image in MMT. On detailed probing tasks, we find that stronger vision models are helpful for learning translation from the visual modality. Our results also suggest the need of carefully examining MMT models, especially when current benchmarks are small-scale and biased. Our code could be found at \url{https://github.com/libeineu/fairseq_mmt}.

cs.CL↗

arXiv2022

ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation

Residual networks are an Euler discretization of solutions to Ordinary Differential Equations (ODE). This paper explores a deeper relationship between Transformer and numerical ODE methods. We first show that a residual block of layers in Transformer can be described as a higher-order solution to ODE. Inspired by this, we design a new architecture, {\it ODE Transformer}, which is analogous to the Runge-Kutta method that is well motivated in ODE. As a natural extension to Transformer, ODE Transformer is easy to implement and efficient to use. Experimental results on the large-scale machine translation, abstractive summarization, and grammar error correction tasks demonstrate the high genericity of ODE Transformer. It can gain large improvements in model performance over strong baselines (e.g., 30.77 and 44.11 BLEU scores on the WMT'14 English-German and English-French benchmarks) at a slight cost in inference efficiency.

cs.CL↗

arXiv2022

Monte Carlo PINNs: deep learning approach for forward and inverse problems involving high dimensional fractional partial differential equations

We introduce a sampling based machine learning approach, Monte Carlo physics informed neural networks (MC-PINNs), for solving forward and inverse fractional partial differential equations (FPDEs). As a generalization of physics informed neural networks (PINNs), our method relies on deep neural network surrogates in addition to a stochastic approximation strategy for computing the fractional derivatives of the DNN outputs. A key ingredient in our MC-PINNs is to construct an unbiased estimation of the physical soft constraints in the loss function. Our directly sampling approach can yield less overall computational cost compared to fPINNs proposed in \cite{pang2019fpinns} and thus provide an opportunity for solving high dimensional fractional PDEs. We validate the performance of MC-PINNs method via several examples that include high dimensional integral fractional Laplacian equations, parametric identification of time-space fractional PDEs, and fractional diffusion equation with random inputs. The results show that MC-PINNs is flexible and promising to tackle high-dimensional FPDEs.

cs.LG↗

arXiv2022

A well-conditioned direct PinT algorithm for first-and second-order evolutionary equations

In this paper, we propose a direct parallel-in-time (PinT) algorithm for time-dependent problems with first- or second-order derivative. We use a second-order boundary value method as the time integrator that leads to a tridiagonal time discretization matrix. Instead of solving the corresponding all-at-once system iteratively, we diagonalize the time discretization matrix, which yields a direct parallel implementation across all time levels. A crucial issue on this methodology is how the condition number of the eigenvector matrix $V$ grows as $n$ is increased, where $n$ is the number of time levels. A large condition number leads to large roundoff error in the diagonalization procedure, which could seriously pollute the numerical accuracy. Based on a novel connection between the characteristic equation and the Chebyshev polynomials, we present explicit formulas for computing $V$ and $V^{-1}$, by which we prove that $\mathrm{Cond}_2(V)=\mathcal{O}(n^{2})$. This implies that the diagonalization process is well-conditioned and the roundoff error only increases moderately as $n$ grows and thus, compared to other direct PinT algorithms, a much larger $n$ can be used to yield satisfactory parallelism. Numerical results on parallel machine are given to support our findings, where over 60 times speedup is achieved with 256 cores.

math.NA↗

arXiv2022

Detecting network communities via greedy expanding based on local superiority index

Community detection is a significant and challenging task in network science. Nowadays, plenty of attention has been paid on local methods for community detection. Greedy expanding is a popular and efficient class of local algorithms, which typically starts from some selected central nodes and expands those nodes to obtain provisional communities by optimizing a certain quality function. In this paper, we propose a novel index, called local superiority index (LSI), to identify central nodes. In the process of expansion, we apply the fitness function to estimate the quality of provisional communities and ensure that all provisional communities must be weak communities. Evaluation based on the normalized mutual information suggests: (1) LSI is superior to the global maximal degree index and the local maximal degree index on most considered networks; (2) The greedy algorithm based on LSI is better than the classical fast algorithm on most considered networks.

physics.soc-ph↗

arXiv2022

Consistency and Diversity induced Human Motion Segmentation

Subspace clustering is a classical technique that has been widely used for human motion segmentation and other related tasks. However, existing segmentation methods often cluster data without guidance from prior knowledge, resulting in unsatisfactory segmentation results. To this end, we propose a novel Consistency and Diversity induced human Motion Segmentation (CDMS) algorithm. Specifically, our model factorizes the source and target data into distinct multi-layer feature spaces, in which transfer subspace learning is conducted on different layers to capture multi-level information. A multi-mutual consistency learning strategy is carried out to reduce the domain gap between the source and target data. In this way, the domain-specific knowledge and domain-invariant properties can be explored simultaneously. Besides, a novel constraint based on the Hilbert Schmidt Independence Criterion (HSIC) is introduced to ensure the diversity of multi-level subspace representations, which enables the complementarity of multi-level representations to be explored to boost the transfer learning performance. Moreover, to preserve the temporal correlations, an enhanced graph regularizer is imposed on the learned representation coefficients and the multi-level representations of the source data. The proposed model can be efficiently solved using the Alternating Direction Method of Multipliers (ADMM) algorithm. Extensive experimental results on public human motion datasets demonstrate the effectiveness of our method against several state-of-the-art approaches.

cs.CV↗

arXiv2022

Fabry-Pérot interference in 2D low-density Rashba gas

In mesoscopic electronic systems, the Fabry-Pérot (FP) oscillation is observed in various 1D devices. As for higher dimensions, numerous transverse channels usually lead to dephasing that quenches the overall oscillation of the conductance. Up to now, the FP oscillation in 2D electronic systems is only reported in graphene-based devices, and very recently, the \emph{pn} junctions of inverted InAs/GaSb double quantum well [Phys. Rev. X 10, 031007 (2020)]. In the latter, the band shape of a sombrero hat plays an essential role, which introduces a novel mechanism of electron-hole hybridization for the 2D FP oscillation. In this work, we propose that such a scenario can be generalized to the 2D planar junction composed of low-density Rashba gas, where the band bottom possesses a sombrero hat shape as well. We show that the backscattering between the outer and inner Fermi circles dominates the FP interference and significantly suppresses the dephasing effect between different transverse channels, which leads to a visible oscillation of the tunneling conductance. Specially, the visibility of the oscillating pattern can be enhanced by applying interface barriers, in contrast to that in the InAs/GaSb double quantum well. Our results provide a promising way for the implementation of the FP oscillation in the 2D electron gas.

cond-mat.mes-hall↗

arXiv2022

Specificity-preserving RGB-D Saliency Detection

Salient object detection (SOD) on RGB and depth images has attracted more and more research interests, due to its effectiveness and the fact that depth cues can now be conveniently captured. Existing RGB-D SOD models usually adopt different fusion strategies to learn a shared representation from the two modalities (\ie, RGB and depth), while few methods explicitly consider how to preserve modality-specific characteristics. In this study, we propose a novel framework, termed SPNet} (Specificity-preserving network), which benefits SOD performance by exploring both the shared information and modality-specific properties (\eg, specificity). Specifically, we propose to adopt two modality-specific networks and a shared learning network to generate individual and shared saliency prediction maps, respectively. To effectively fuse cross-modal features in the shared learning network, we propose a cross-enhanced integration module (CIM) and then propagate the fused feature to the next layer for integrating cross-level information. Moreover, to capture rich complementary multi-modal information for boosting the SOD performance, we propose a multi-modal feature aggregation (MFA) module to integrate the modality-specific features from each individual decoder into the shared decoder. By using a skip connection, the hierarchical features between the encoder and decoder layers can be fully combined. Extensive experiments demonstrate that our~\ours~outperforms cutting-edge approaches on six popular RGB-D SOD and three camouflaged object detection benchmarks. The project is publicly available at: https://github.com/taozh2017/SPNet.

cs.CV↗

arXiv2022

Jumping to male-dominated occupations: A novel way to reduce gender wage gap for Chinese women

Occupational segregation is widely considered as one major reason leading to the gender discrimination in labor market. Using large-scale Chinese resume data of online job seekers, we uncover an interesting phenomenon that occupations with higher proportion of men have smaller gender wage gap measured by the female-male ratio on wage. We further show that the severity of occupational segregation in China is low both overall and regionally, and the inter-occupational discrimination is much smaller than the intra-occupational discrimination. That is to say, Chinese women do not face large barriers when changing their occupations. Accordingly, we suggest Chineses women a new way to narrow the gender wage gap: to join male-dominated occupations. Meanwhile, it is worth noticing that although the gender wage gap is smaller in male-dominated occupations, it does not mean that the gender discrimination is smaller there.

physics.soc-ph↗

arXiv2021

Substantial but heterogeneous impacts of high-speed rail on talent flow in China

The great expansion of high-speed rail (HSR) in China facilitates communications and interactions among people across cities. Despite extensive literature documenting the effects of HSR on a variety of variables such as local economic development, research collaboration, tourism, and capital mobility, not much is known about how HSR affects the flow of well-educated workers, says talents. Here we estimate talent flow among Chinese cities based on large-scale resume data of online job seekers and explore how it is affected by HSR. Specifically, we employ both a multiple linear regression model that controls for several socioeconomic factors and a two-stage least square regression model that instruments the introduction of HSR to a city to address endogeneity concerns. We find that the introduction of HSR has an overall positive effect on the talent net inflow of a city although both inflow and outflow are increased. Moreover, the effects of HSR on talent flow are rather heterogeneous for cities with different levels of economic development and for talents working in different industries. Specifically, developed cities benefit from HSR, whereas less-developed cities are relatively impaired. Cities connected by HSR show significant advantage in attracting talents from secondary and tertiary industries. These substantial but heterogeneous effects of HSR suggest a critical need for more comprehensive thinking about the long-term benefits of entering the HSR network, especially for less-developed cities and those with comparative advantage in manufacturing and service industries.

physics.soc-ph↗

arXiv2021

Quantifying human mixing patterns in Chinese provinces outside Hubei after the 2020 lockdown was lifted

Contact patterns play a key role in the spread of respiratory infectious diseases in human populations. During the COVID-19 pandemic the regular contact patterns of the population has been disrupted due to social distancing both imposed by the authorities and individual choices. Here we present the results of a contact survey conducted in Chinese provinces outside Hubei in March 2020, right after lockdowns were lifted. We then leveraged the estimated mixing patterns to calibrate a model of SARS-CoV-2 transmission, which was used to estimate different metrics of COVID-19 burden by age. Study participants reported 2.3 contacts per day (IQR: 1.0-3.0) and the mean per-contact duration was 7.0 hours (IQR: 1.0-10.0). No significant differences were observed between provinces, the number of recorded contacts did not show a clear-cut trend by age, and most of the recorded contacts occurred with family members (about 78%). Our findings suggest that, despite the lockdown was no longer in place at the time of the survey, people were still heavily limiting their contacts as compared to the pre-pandemic situation. Moreover, the obtained modeling results highlight the importance of considering age-contact patterns to estimate COVID-19 burden.

cs.SI↗

arXiv2021

threaTrace: Detecting and Tracing Host-based Threats in Node Level Through Provenance Graph Learning

Host-based threats such as Program Attack, Malware Implantation, and Advanced Persistent Threats (APT), are commonly adopted by modern attackers. Recent studies propose leveraging the rich contextual information in data provenance to detect threats in a host. Data provenance is a directed acyclic graph constructed from system audit data. Nodes in a provenance graph represent system entities (e.g., $processes$ and $files$) and edges represent system calls in the direction of information flow. However, previous studies, which extract features of the whole provenance graph, are not sensitive to the small number of threat-related entities and thus result in low performance when hunting stealthy threats. We present threaTrace, an anomaly-based detector that detects host-based threats at system entity level without prior knowledge of attack patterns. We tailor GraphSAGE, an inductive graph neural network, to learn every benign entity's role in a provenance graph. threaTrace is a real-time system, which is scalable of monitoring a long-term running host and capable of detecting host-based intrusion in their early phase. We evaluate threaTrace on three public datasets. The results show that threaTrace outperforms three state-of-the-art host intrusion detection systems.

cs.CR↗

arXiv2021

Fragile topology in nodal-line semimetal superconductors

We study the band topology of the superconducting nodal-line semimetal (SC-NLSM) protected by the inversion symmetry with and without the spin-orbital coupling. Without the spin-orbital coupling, both the $s$-wave SC-NLSM and the chiral $p$-wave SC-NLSM are topologically nontrivial and can be described by the nonzero winding number. Based on the Wilson loop method, we verify that they are both the fragile topological superconductors, namely, their nontrivial band topologies can be moved off by coupling to additional topologically trivial bands. The fragile topological phase persists in spinful system with the time-reversal symmetry when a spin-orbital coupling term is added. For the spinful system, both the $p$-wave SC-NLSM and the $s$-wave SC-NLSM are second-order fragile topological superconductors. We propose that the fragile topology in the SC-NLSM system depends strongly on the degeneracy of the Majorana zero modes and the parity of the superconducting gap function. Interestingly, in presence of a vortex line, the spinful $s$-wave SC-NLSM system hosts two pairs of stable Majorana zero modes in the vortex core.

cond-mat.supr-con↗

arXiv2021

Classification and purification for the independent quantum channel through quantum error-correction

The essence of quantum error-correction is to use redundant Hilbert space to identify and correct errors, and the channel fidelity of the quantum channel does not affect which errors can be identified and corrected. Based on this, it is found that quantum error-correction can be used to classify the independent quantum channel into 5 types, and 4 of the 5 types can be purified. It is found in quantum error-correction, the decoherence of quantum state may be related to the degree of identification for the state under quantum noise, and the results of this work confirmed that the degree of purity of quantum channel determines its ability to retain the quantum property of the quantum state, not the fidelity. In this work, the identification of the independent Pauli channels by quantum error-correction is demonstrated.

quant-ph↗

arXiv2021

Kerr black hole surrounded by a cloud of strings and its weak gravitational lensing in Rastall gravity

In this paper, an exact solution of Kerr black hole surrounded by a cloud of strings in Rastall gravity is obtained by Newman-Janis algorithm without complexification, and the influence of the string parameter $a_0$ on the black hole thermodynamics is studied. Furthermore, according to this black hole solution, we investigate the effect of a cloud of strings on gravitational deflection of massive particles. For a receiver and a source at infinite distance from the lens object, we use an osculating Riemannian manifold method. While the distance is finite, we apply the generalized Jacobi metric method. For the case between the two situations, the Jacobi-Maupertuis Randers-Finsler metric and Gauss-Bonnet theorem are employed. We find that the string parameter $a_0$ has obviously modification on the gravitational deflection of massive particles. Our result is reduced to the deflection angle of light by Kerr black hole surrounded by a cloud of strings in general relativity.

gr-qc↗

arXiv2021

The NiuTrans Machine Translation Systems for WMT21

This paper describes NiuTrans neural machine translation systems of the WMT 2021 news translation tasks. We made submissions to 9 language directions, including English$\leftrightarrow$$\{$Chinese, Japanese, Russian, Icelandic$\}$ and English$\rightarrow$Hausa tasks. Our primary systems are built on several effective variants of Transformer, e.g., Transformer-DLCL, ODE-Transformer. We also utilize back-translation, knowledge distillation, post-ensemble, and iterative fine-tuning techniques to enhance the model performance further.

cs.CL↗

arXiv2021

Information Cocoons in Online Navigation

Social media and online navigation bring us enjoyable experience in accessing information, and simultaneously create information cocoons (ICs) in which we are unconsciously trapped with limited and biased information. We provide a formal definition of IC in the scenario of online navigation. Subsequently, by analyzing real recommendation networks extracted from Science, PNAS and Amazon websites, and testing mainstream algorithms in disparate recommender systems, we demonstrate that similarity-based recommendation techniques result in ICs, which suppress the system navigability by hundreds of times. We further propose a flexible recommendation strategy that solves the IC-induced problem and improves retrieval accuracy in navigation, demonstrated by simulations on real data and online experiments on the largest video website in China.

physics.soc-ph↗