arXiv subjects

Yu Cheng

Publications and source records attributed to Yu Cheng.

At least 415 records · Page 23Linked to original sources

arXiv2021

Light dark matter from dark sector decay

We study the possibility that light dark matter can be produced with right relic density by the decay of other dark sector particles. We study this possibility in a model with right-handed neutrinos, a dark sector singlet scalar and a dark sector singlet fermion. The decay of the heavier dark sector singlet scalar gives rise to the lighter dark sector singlet fermion which serves as the dark matter candidate. We show that the right dark matter relic density can be produced by the decay of the dark sector singlet scalar. We find that the mass of the dark sector singlet fermion can be GeV scale or MeV scale and the interaction of the dark sector singlet fermion is very weak. The dark sector singlet scalar can have a mass of GeV-TeV scale and can have weak scale interactions with the Standard Model(SM) particles. So, in this scenario, dark matter has an interaction with the SM much weaker than the weak interaction, but the dark sector still has weak scale interaction with the SM.

hep-ph↗

arXiv2021

InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective

Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies, however, show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We aim to address this problem from an information-theoretic perspective, and propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models. InfoBERT contains two mutual-information-based regularizers for model training: (i) an Information Bottleneck regularizer, which suppresses noisy mutual information between the input and the feature representation; and (ii) a Robust Feature regularizer, which increases the mutual information between local robust features and global features. We provide a principled way to theoretically analyze and improve the robustness of representation learning for language models in both standard and adversarial training. Extensive experiments demonstrate that InfoBERT achieves state-of-the-art robust accuracy over several adversarial datasets on Natural Language Inference (NLI) and Question Answering (QA) tasks. Our code is available at https://github.com/AI-secure/InfoBERT.

cs.CL↗

arXiv2021

Context-aware Biaffine Localizing Network for Temporal Sentence Grounding

This paper addresses the problem of temporal sentence grounding (TSG), which aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. Previous works either compare pre-defined candidate segments with the query and select the best one by ranking, or directly regress the boundary timestamps of the target segment. In this paper, we propose a novel localization framework that scores all pairs of start and end indices within the video simultaneously with a biaffine mechanism. In particular, we present a Context-aware Biaffine Localizing Network (CBLN) which incorporates both local and global contexts into features of each start/end position for biaffine-based localization. The local contexts from the adjacent frames help distinguish the visually similar appearance, and the global contexts from the entire video contribute to reasoning the temporal relation. Besides, we also develop a multi-modal self-attention module to provide fine-grained query-guided video representation for this biaffine strategy. Extensive experiments show that our CBLN significantly outperforms state-of-the-arts on three public datasets (ActivityNet Captions, TACoS, and Charades-STA), demonstrating the effectiveness of the proposed localization framework.

cs.CV↗

arXiv2021

Adversarial Feature Augmentation and Normalization for Visual Recognition

Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings, instead of relying on computationally-expensive pixel-level perturbations. We propose Adversarial Feature Augmentation and Normalization (A-FAN), which (i) first augments visual recognition models with adversarial features that integrate flexible scales of perturbation strengths, (ii) then extracts adversarial feature statistics from batch normalization, and re-injects them into clean features through feature normalization. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks, including ResNets and EfficientNets for classification, Faster-RCNN for detection, and Deeplab V3+ for segmentation. Extensive experiments show that A-FAN yields consistent generalization improvement over strong baselines across various datasets for classification, detection and segmentation tasks, such as CIFAR-10, CIFAR-100, ImageNet, Pascal VOC2007, Pascal VOC2012, COCO2017, and Cityspaces. Comprehensive ablation studies and detailed analyses also demonstrate that adding perturbations to specific modules and layers of classification/detection/segmentation backbones yields optimal performance. Codes and pre-trained models will be made available at: https://github.com/VITA-Group/CV_A-FAN.

cs.LG↗

arXiv2021

Structure Of Flavor Changing Goldstone Boson Interactions

General flavor changing interactions for a Goldstone boson (GB) to fermions due to a spontaneous global $U(1)_G$ symmetry breaking are discussed. This GB may be the Axion, solving the strong QCD CP problem, if there is a QCD anomaly for the $U(1)_G$ charge assignments for quarks. Or it may be the Majoron in models from lepton number violation in producing seesaw Majorana neutrino masses if the symmetry breaking scale is much higher than the electroweak scale. It may also, in principle, play the roles of Axion and Majoron simultaneously as far as providing solution for the strong CP problem and generating a small Majorana neutrino masses are concerned. Great attentions have been focused on flavor conserving GB interactions. Recently flavor changing Axion and Majoron models have been studied in the hope to find new physics from rare decays in the intensity frontier. In this work, we will provide a systematic model building aspect study for a GB having flavor changing neutral current (FCNC) interactions in both the quark and lepton sectors, or separately in the quark, charged lepton and neutrino sectors with sources of FCNC interactions identified in detail. We provide a general proof of the equivalence of using physical GB components and GB broken generators for calculating GB couplings to two gluons and two photons, and some issues related to models having spontaneous CP violation are discussed. We will also provide some details for obtaining FCNC GB interactions in several popular models, such as the Type-I, -II, -III seesaw and Left-Right symmetric models, and point out some special features in these models.

hep-ph↗

arXiv2021

Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning

Few-shot text classification is a fundamental NLP task in which a model aims to classify text into a large number of categories, given only a few training examples per category. This paper explores data augmentation -- a technique particularly suitable for training with limited data -- for this few-shot, highly-multiclass text classification setting. On four diverse text classification tasks, we find that common data augmentation techniques can improve the performance of triplet networks by up to 3.0% on average. To further boost performance, we present a simple training strategy called curriculum data augmentation, which leverages curriculum learning by first training on only original examples and then introducing augmented data as training progresses. We explore a two-stage and a gradual schedule, and find that, compared with standard single-stage training, curriculum data augmentation trains faster, improves performance, and remains robust to high amounts of noising from augmentation.

cs.CL↗

arXiv2021

Fair for All: Best-effort Fairness Guarantees for Classification

Standard approaches to group-based notions of fairness, such as \emph{parity} and \emph{equalized odds}, try to equalize absolute measures of performance across known groups (based on race, gender, etc.). Consequently, a group that is inherently harder to classify may hold back the performance on other groups; and no guarantees can be provided for unforeseen groups. Instead, we propose a fairness notion whose guarantee, on each group $g$ in a class $\mathcal{G}$, is relative to the performance of the best classifier on $g$. We apply this notion to broad classes of groups, in particular, where (a) $\mathcal{G}$ consists of all possible groups (subsets) in the data, and (b) $\mathcal{G}$ is more streamlined. For the first setting, which is akin to groups being completely unknown, we devise the {\sc PF} (Proportional Fairness) classifier, which guarantees, on any possible group $g$, an accuracy that is proportional to that of the optimal classifier for $g$, scaled by the relative size of $g$ in the data set. Due to including all possible groups, some of which could be too complex to be relevant, the worst-case theoretical guarantees here have to be proportionally weaker for smaller subsets. For the second setting, we devise the {\sc BeFair} (Best-effort Fair) framework which seeks an accuracy, on every $g \in \mathcal{G}$, which approximates that of the optimal classifier on $g$, independent of the size of $g$. Aiming for such a guarantee results in a non-convex problem, and we design novel techniques to get around this difficulty when $\mathcal{G}$ is the set of linear hypotheses. We test our algorithms on real-world data sets, and present interesting comparative insights on their performance.

cs.LG↗

arXiv2021

Deep Co-Attention Network for Multi-View Subspace Learning

Many real-world applications involve data from multiple modalities and thus exhibit the view heterogeneity. For example, user modeling on social media might leverage both the topology of the underlying social network and the content of the users' posts; in the medical domain, multiple views could be X-ray images taken at different poses. To date, various techniques have been proposed to achieve promising results, such as canonical correlation analysis based methods, etc. In the meanwhile, it is critical for decision-makers to be able to understand the prediction results from these methods. For example, given the diagnostic result that a model provided based on the X-ray images of a patient at different poses, the doctor needs to know why the model made such a prediction. However, state-of-the-art techniques usually suffer from the inability to utilize the complementary information of each view and to explain the predictions in an interpretable manner. To address these issues, in this paper, we propose a deep co-attention network for multi-view subspace learning, which aims to extract both the common information and the complementary information in an adversarial setting and provide robust interpretations behind the prediction to the end-users via the co-attention mechanism. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation by incorporating the classifier into our model. This improves the quality of latent representation and accelerates the convergence speed. Finally, we develop an efficient iterative algorithm to find the optimal encoders and discriminator, which are evaluated extensively on synthetic and real-world data sets. We also conduct a case study to demonstrate how the proposed method robustly interprets the predictions on an image data set.

cs.LG↗

arXiv2021

Broadband terahertz half-wave plate with multi-layers metamaterial via quantum engineering

In this paper, we employ the novel design of the metamaterial half-wave plate by using the multiple layers of the metamaterials with some specific rotation angles. The rotation angles are given by composite pulse control which is the well-known quantum control technique. A big advantage of this method can analytically calculate the rotation angles and can be easily extended to the multiple layers. The more layers of the metamaterials can present more bandwidth of half-wave plate. In this paper, we present 1, 3, 5, 7 configurations of layers of a metamaterial half-wave plate and we demonstrate that our device increasingly enhance the bandwidth of performance.

physics.optics↗

arXiv2021

EnlightenGAN: Deep Light Enhancement without Paired Supervision

Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data? As one such example, this paper explores the low-light image enhancement problem, where in practice it is extremely challenging to simultaneously take a low-light and a normal-light photo of the same visual scene. We propose a highly effective unsupervised generative adversarial network, dubbed EnlightenGAN, that can be trained without low/normal-light image pairs, yet proves to generalize very well on various real-world test images. Instead of supervising the learning using ground truth data, we propose to regularize the unpaired training using the information extracted from the input itself, and benchmark a series of innovations for the low-light image enhancement problem, including a global-local discriminator structure, a self-regularized perceptual loss fusion, and attention mechanism. Through extensive experiments, our proposed approach outperforms recent methods under a variety of metrics in terms of visual quality and subjective user study. Thanks to the great flexibility brought by unpaired training, EnlightenGAN is demonstrated to be easily adaptable to enhancing real-world images from various domains. The code is available at \url{https://github.com/yueruchen/EnlightenGAN}

cs.CV↗

arXiv2021

ALMA Observations of Massive Clouds in the Central Molecular Zone: Ubiquitous Protostellar Outflows

We observe 1.3~mm spectral lines at 2000~AU resolution toward four massive molecular clouds in the Central Molecular Zone of the Galaxy to investigate their star formation activities. We focus on several potential shock tracers that are usually abundant in protostellar outflows, including SiO, SO, CH$_3$OH, H$_2$CO, HC$_3$N, and HNCO. We identify 43 protostellar outflows, including 37 highly likely ones and 6 candidates. The outflows are found toward both known high-mass star forming cores and less massive, seemingly quiescent cores, while 791 out of the 834 cores identified based on the continuum do not have detected outflows. The outflow masses range from less than 1~$M_\odot$ to a few tens of $M_\odot$, with typical uncertainties of a factor of 70. We do not find evidence of disagreement between relative molecular abundances in these outflows and in nearby analogs such as the well-studied L1157 and NGC7538S outflows. The results suggest that i) protostellar accretion disks driving outflows ubiquitously exist in the CMZ environment, ii) the large fraction of candidate starless cores is expected if these clouds are at very early evolutionary phases, with a caveat on the potential incompleteness of the outflows, iii) high-mass and low-mass star formation is ongoing simultaneously in these clouds, and iv) current data do not show evidence of difference between the shock chemistry in the outflows that determines the molecular abundances in the CMZ environment and in nearby clouds.

astro-ph.GA↗

arXiv2021

Electronic controllable broadband and robust terahertz surface plasmon-polaritons switch based on hybrid ITO waveguide coupler

The surface plasmon-polaritons (SPPs) switch is the key element of the integrated devices in optical computation and terahertz (THz) communications. In this paper, we propose a novel design of THz SPPs switch based on quantum engineering. Due to the robustness of coherent quantum control technique, our switch is very robust against with perturbations of geometrical parameters and presents a good performance at on-state (and off-state) from 0.5 THz to 0.7 THz. The on-state and off-state of our device can be controlled by the external voltage. We believe this finding will be the great improvement for the integrated optical computing and THz communications.

physics.optics↗

arXiv2021

Deuterium Chemodynamics of Massive Pre-Stellar Cores

High levels of deuterium fractionation of $\rm N_2H^+$ (i.e., $\rm D_{frac}^{N_2H^+} \gtrsim 0.1$) are often observed in pre-stellar cores (PSCs) and detection of $\rm N_2D^+$ is a promising method to identify elusive massive PSCs. However, the physical and chemical conditions required to reach such high levels of deuteration are still uncertain, as is the diagnostic utility of $\rm N_2H^+$ and $\rm N_2D^+$ observations of PSCs. We perform 3D magnetohydrodynamics simulations of a massive, turbulent, magnetised PSC, coupled with a sophisticated deuteration astrochemical network. Although the core has some magnetic/turbulent support, it collapses under gravity in about one freefall time, which marks the end of the simulations. Our fiducial model achieves relatively low $\rm D_{frac}^{N_2H^+} \sim 0.002$ during this time. We then investigate effects of initial ortho-para ratio of $\rm H_2$ ($\rm OPR^{H_2}$), temperature, cosmic ray (CR) ionization rate, CO and N-species depletion factors and prior PSC chemical evolution. We find that high CR ionization rates and high depletion factors allow the simulated $\rm D_{frac}^{N_2H^+}$ and absolute abundances to match observational values within one freefall time. For $\rm OPR^{H_2}$, while a lower initial value helps the growth of $\rm D_{frac}^{N_2H^+}$, the spatial structure of deuteration is too widespread compared to observed systems. For an example model with elevated CR ionization rates and significant heavy element depletion, we then study the kinematic and dynamic properties of the core as traced by its $\rm N_2D^+$ emission. The core, undergoing quite rapid collapse, exhibits disturbed kinematics in its average velocity map. Still, because of magnetic support, the core often appears kinematically sub-virial based on its $\rm N_2D^+$ velocity dispersion.

astro-ph.GA↗

arXiv2021

Topology Aware Deep Learning for Wireless Network Optimization

Data-driven machine learning approaches have recently been proposed to facilitate wireless network optimization by learning latent knowledge from historical optimization instances. However, existing methods do not well handle the topology information that directly impacts the network optimization results. Directly operating on simple representations, e.g., adjacency matrices, results in poor generalization performance as the learned results depend on specific ordering of the network elements in the training data. To address this issue, we propose a two-stage topology-aware machine learning framework (TALF), which trains a graph embedding unit and a deep feed-forward network (DFN) jointly. By propagating and summarizing the underlying graph topological information, TALF encodes the topology in the vector representation of the optimization instance, which is used by the later DFN to infer critical structures of an optimal or near-optimal solution. The proposed approach is evaluated on a canonical wireless network flow problem with diverse network typologies and flow deployments. In-depth study on trade-off between efficiency and effectiveness of the inference results is also conducted, and we show that our approach is better at differentiate links by saving up to 60% computation time at over 90% solution quality.

cs.NI↗

arXiv2020

Meta Module Network for Compositional Visual Reasoning

Neural Module Network (NMN) exhibits strong interpretability and compositionality thanks to its handcrafted neural modules with explicit multi-hop reasoning capability. However, most NMNs suffer from two critical drawbacks: 1) scalability: customized module for specific function renders it impractical when scaling up to a larger set of functions in complex tasks; 2) generalizability: rigid pre-defined module inventory makes it difficult to generalize to unseen functions in new tasks/domains. To design a more powerful NMN architecture for practical use, we propose Meta Module Network (MMN) centered on a novel meta module, which can take in function recipes and morph into diverse instance modules dynamically. The instance modules are then woven into an execution graph for complex visual reasoning, inheriting the strong explainability and compositionality of NMN. With such a flexible instantiation mechanism, the parameters of instance modules are inherited from the central meta module, retaining the same model complexity as the function set grows, which promises better scalability. Meanwhile, as functions are encoded into the embedding space, unseen functions can be readily represented based on its structural similarity with previously observed ones, which ensures better generalizability. Experiments on GQA and CLEVR datasets validate the superiority of MMN over state-of-the-art NMN designs. Synthetic experiments on held-out unseen functions from GQA dataset also demonstrate the strong generalizability of MMN. Our code and model are released in Github https://github.com/wenhuchen/Meta-Module-Network.

cs.CV↗

arXiv2020

Large-Scale Adversarial Training for Vision-and-Language Representation Learning

We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning. VILLA consists of two training stages: (i) task-agnostic adversarial pre-training; followed by (ii) task-specific adversarial finetuning. Instead of adding adversarial perturbations on image pixels and textual tokens, we propose to perform adversarial training in the embedding space of each modality. To enable large-scale training, we adopt the "free" adversarial training strategy, and combine it with KL-divergence-based regularization to promote higher invariance in the embedding space. We apply VILLA to current best-performing V+L models, and achieve new state of the art on a wide range of tasks, including Visual Question Answering, Visual Commonsense Reasoning, Image-Text Retrieval, Referring Expression Comprehension, Visual Entailment, and NLVR2.

cs.CV↗

arXiv2020

Object Tracking Using Spatio-Temporal Future Prediction

Occlusion is a long-standing problem that causes many modern tracking methods to be erroneous. In this paper, we address the occlusion problem by exploiting the current and future possible locations of the target object from its past trajectory. To achieve this, we introduce a learning-based tracking method that takes into account background motion modeling and trajectory prediction. Our trajectory prediction module predicts the target object's locations in the current and future frames based on the object's past trajectory. Since, in the input video, the target object's trajectory is not only affected by the object motion but also the camera motion, our background motion module estimates the camera motion. So that the object's trajectory can be made independent from it. To dynamically switch between the appearance-based tracker and the trajectory prediction, we employ a network that can assess how good a tracking prediction is, and we use the assessment scores to choose between the appearance-based tracker's prediction and the trajectory-based prediction. Comprehensive evaluations show that the proposed method sets a new state-of-the-art performance on commonly used tracking benchmarks.

cs.CV↗

arXiv2020

Cross-Thought for Sentence Encoder Pre-training

In this paper, we propose Cross-Thought, a novel approach to pre-training sequence encoder, which is instrumental in building reusable sequence embeddings for large-scale NLP tasks such as question answering. Instead of using the original signals of full sentences, we train a Transformer-based sequence encoder over a large set of short sequences, which allows the model to automatically select the most useful information for predicting masked words. Experiments on question answering and textual entailment tasks demonstrate that our pre-trained encoder can outperform state-of-the-art encoders trained with continuous sentence signals as well as traditional masked language modeling baselines. Our proposed approach also achieves new state of the art on HotpotQA (full-wiki setting) by improving intermediate information retrieval performance.

cs.CL↗