Searcharxiv⌕ Search

arXiv subjects

Zhiwei Yang

Publications and source records attributed to Zhiwei Yang.

62 records · Page 4Linked to original sources
arXiv2022

DecBERT: Enhancing the Language Understanding of BERT with Causal Attention Masks

Since 2017, the Transformer-based models play critical roles in various downstream Natural Language Processing tasks. However, a common limitation of the attention mechanism utilized in Transformer Encoder is that it cannot automatically capture the information of word order, so explicit position embeddings are generally required to be fed into the target model. In contrast, Transformer Decoder with the causal attention masks is naturally sensitive to the word order. In this work, we focus on improving the position encoding ability of BERT with the causal attention masks. Furthermore, we propose a new pre-trained language model DecBERT and evaluate it on the GLUE benchmark. Experimental results show that (1) the causal attention mask is effective for BERT on the language understanding tasks; (2) our DecBERT model without position embeddings achieve comparable performance on the GLUE benchmark; and (3) our modification accelerates the pre-training process and DecBERT w/ PE achieves better overall performance than the baseline systems when pre-training with the same amount of computational resources.

cs.CL↗
arXiv2021

Rumor Detection on Twitter with Claim-Guided Hierarchical Graph Attention Networks

Rumors are rampant in the era of social media. Conversation structures provide valuable clues to differentiate between real and fake claims. However, existing rumor detection methods are either limited to the strict relation of user responses or oversimplify the conversation structure. In this study, to substantially reinforces the interaction of user opinions while alleviating the negative impact imposed by irrelevant posts, we first represent the conversation thread as an undirected interaction graph. We then present a Claim-guided Hierarchical Graph Attention Network for rumor classification, which enhances the representation learning for responsive posts considering the entire social contexts and attends over the posts that can semantically infer the target claim. Extensive experiments on three Twitter datasets demonstrate that our rumor detection method achieves much better performance than state-of-the-art methods and exhibits a superior capacity for detecting rumors at early stages.

cs.CL↗
arXiv2020

Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision

Violence detection has been studied in computer vision for years. However, previous work are either superficial, e.g., classification of short-clips, and the single scenario, or undersupplied, e.g., the single modality, and hand-crafted features based multimodality. To address this problem, in this work we first release a large-scale and multi-scene dataset named XD-Violence with a total duration of 217 hours, containing 4754 untrimmed videos with audio signals and weak labels. Then we propose a neural network containing three parallel branches to capture different relations among video snippets and integrate features, where holistic branch captures long-range dependencies using similarity prior, localized branch captures local positional relation using proximity prior, and score branch dynamically captures the closeness of predicted score. Besides, our method also includes an approximator to meet the needs of online detection. Our method outperforms other state-of-the-art methods on our released dataset and other existing benchmark. Moreover, extensive experimental results also show the positive effect of multimodal (audio-visual) input and modeling relationships. The code and dataset will be released in https://roc-ng.github.io/XD-Violence/.

cs.CV↗
arXiv2019

Robust OFDM integrated radar and communications waveform design based on information theory

An integrated radar and communications system (IRCS) where a monostatic radar transceiver is employed for target classification while simultaneously used as a communications transmitter is considered. The radar combined propagation-target response (joint response of the radar propagation channel and target) and communications channel response are generally frequency selective but the corresponding frequency response functions are not exactly known. In particular, these frequency response functions are only known to lie in an uncertainty class. To ensure the IRCS simultaneously provides acceptable target classification performance and communications rate, a robust orthogonal frequency division multiplexing (OFDM) integrated radar and communications waveform (IRCW) design method is proposed. The approach finds a waveform that simultaneously provides a sufficiently large weighted sum of the communications data information rate (DIR) and the conditional mutual information (MI) between the observed signal and the radar target over the entire uncertainty class. First, the conditional MI and DIR based on the integrated OFDM radar and communications waveform are derived. Then, a robust OFDM IRCW optimization problem based on the minimax design philosophy is developed such that closed-form solution is derived. Finally, several numerical results are presented to demonstrate the effectiveness of the proposed method.

eess.SP↗
arXiv2016

Spectroscopy of cesium Rydberg atoms in strong radio-frequency fields

We study Rydberg atoms modulated by strong radio-frequency (RF) fields with a frequency of 70 MHz. The Rydberg atoms are prepared in a room temperature cesium cell, and their level structure is probed using electromagnetically induced transparency (EIT). As the RF field increases from the weak- into the strong-field regime, the range of observed RF-induced phenomena progresses from AC level shifts through increasingly pronounced and numerous RF-modulation sidebands to complex state-mixing and level-crossings with high-l hydrogen-like states. Weak anharmonic admixtures in the RF field generate clearly visible modifications in the Rydberg-EIT spectra. A Floquet analysis is employed to model the Rydberg spectra, and good agreement with the experimental observations is found. Our results show that all-optical spectroscopy of Rydberg atoms in vapor cells can serve as an antenna-free, atom-based and calibration-free technique to measure and map RF electric fields and to analyze their higher-harmonic contents.

physics.atom-ph↗
arXiv2014

A Carbon Corrosion Model to Evaluate the Effect of Steady State and Transient Operation of a Polymer Electrolyte Membrane Fuel Cell

A carbon corrosion model is developed based on the formation of surface oxides on carbon and platinum of the polymer electrolyte membrane fuel cell electrode. The model predicts the rate of carbon corrosion under potential hold and potential cycling conditions. The model includes the interaction of carbon surface oxides with transient species like OH radicals to explain observed carbon corrosion trends under normal PEM fuel cell operating conditions. The model prediction agrees qualitatively with the experimental data supporting the hypothesis that the interplay of surface oxide formation on carbon and platinum is the primary driver of carbon corrosion.

physics.chem-ph↗
arXiv2011

Bed-inventory Overturn Mechanism for Pant-leg Circulating Fluidized Bed Boilers

A numerical model was established to investigate the lateral mass transfer as well as the mechanism of bed-inventory overturn inside a pant-leg circulating fluidized bed (CFB), which are of great importance to maintain safe and efficient operation of the CFB. Results show that the special flow structure in which the solid particle volume fraction along the central line of the pant-leg CFB is relative high enlarges the lateral mass transfer rate and make it more possible for bed inventory overturn. Although the lateral pressure difference generated from lateral mass transfer inhibits continuing lateral mass transfer, providing the pant-leg CFB with self-balancing ability to some extent, the primary flow rate change due to the outlet pressure change often disable the self-balancing ability by continually enhancing the flow rate difference. As the flow rate of the primary air fan is more sensitive to its outlet pressure, it is easier to lead to bed inventory overturn. While when the solid particle is easier to change its flow patter to follow the surrounding air flow,the self-balancing ability is more active.

physics.flu-dyn↗