arXiv subjects

Jiawei Zhou

Publications and source records attributed to Jiawei Zhou.

At least 127 records · Page 7Linked to original sources

arXiv2022

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code

Evaluation in machine learning is usually informed by past choices, for example which datasets or metrics to use. This standardization enables the comparison on equal footing using leaderboards, but the evaluation choices become sub-optimal as better alternatives arise. This problem is especially pertinent in natural language generation which requires ever-improving suites of datasets, metrics, and human evaluation to make definitive claims. To make following best model evaluation practices easier, we introduce GEMv2. The new version of the Generation, Evaluation, and Metrics Benchmark introduces a modular infrastructure for dataset, model, and metric developers to benefit from each others work. GEMv2 supports 40 documented datasets in 51 languages. Models for all datasets can be evaluated online and our interactive data card creation and rendering tools make it easier to add new datasets to the living benchmark.

cs.CL↗

arXiv2022

Inducing and Using Alignments for Transition-based AMR Parsing

Transition-based parsers for Abstract Meaning Representation (AMR) rely on node-to-word alignments. These alignments are learned separately from parser training and require a complex pipeline of rule-based components, pre-processing, and post-processing to satisfy domain-specific constraints. Parsers also train on a point-estimate of the alignment pipeline, neglecting the uncertainty due to the inherent ambiguity of alignment. In this work we explore two avenues for overcoming these limitations. First, we propose a neural aligner for AMR that learns node-to-word alignments without relying on complex pipelines. We subsequently explore a tighter integration of aligner and parser training by considering a distribution over oracle action sequences arising from aligner uncertainty. Empirical results show this approach leads to more accurate alignments and generalization better from the AMR2.0 to AMR3.0 corpora. We attain a new state-of-the art for gold-only trained models, matching silver-trained performance without the need for beam search on AMR3.0.

cs.CL↗

arXiv2022

Significant reduction in semiconductor interface resistance via interfacial atomic mixing

The contact resistance between two dissimilar semiconductors is determined by the carrier transmission through their interface. Despite the ubiquitous presence of interfaces, quantitative simulation of charge transport across such interfaces is difficult, limiting the understanding of interfacial charge transport. This work employs Green's functions to study the charge transport across representative Si/Ge interfaces. For perfect interfaces, it is found that the transmittance is small and the contact resistance is high, not only because the mismatch of carrier pockets makes it hard to meet the momentum conservation requirement, but also because of the incompatible symmetries of the Bloch wave functions of the two sides. In contrast, atomic mixing at the interface increases the carrier transmittance as the interface roughness opens many nonspecular transmission channels, which greatly reduces the contact resistance compared with the perfect interface. Specifically, we show that disordered interfaces with certain symmetries create more nonspecular transmission. The insights from our study will benefit the future design of high-performance heterostructures with low contact resistance.

cond-mat.mes-hall↗

arXiv2022

Global existence and optimal decay rate of the classical solution to 3-D Radiative Hydrodynamics with and without Heat Conductivity

The classical solution of the 3-D radiative hydrodynamics model is studied in $H^k$-norm under two different conditions, with and without heat conductivity. We have proved the following results in both cases. First, when the $H^k$ norm of the initial perturbation around a constant state is sufficiently small and the integer $k\geq2$, a unique classical solution to such Cauchy problem is shown to exist. Second, if we further assume that the $L^1$ norm of the initial perturbation is small too, the i-order($0\leq i\leq k-2$) derivative of the solutions have the decay rate of $(1+t)^{-\frac 34-\frac i2}$ in $H^2$ norm. Third, from the results above we can see that for radiative hydrodynamics, the radiation can do the same job as the heat conduction, which means if the thermal conductivity coefficient turns to $0$, because of the effect of radiation, the solvability of the system and decay rate of the solution stay the same.

math.AP↗

arXiv2022

Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering

To alleviate the data scarcity problem in training question answering systems, recent works propose additional intermediate pre-training for dense passage retrieval (DPR). However, there still remains a large discrepancy between the provided upstream signals and the downstream question-passage relevance, which leads to less improvement. To bridge this gap, we propose the HyperLink-induced Pre-training (HLP), a method to pre-train the dense retriever with the text relevance induced by hyperlink-based topology within Web documents. We demonstrate that the hyperlink-based structures of dual-link and co-mention can provide effective relevance signals for large-scale pre-training that better facilitate downstream passage retrieval. We investigate the effectiveness of our approach across a wide range of open-domain QA datasets under zero-shot, few-shot, multi-hop, and out-of-domain scenarios. The experiments show our HLP outperforms the BM25 by up to 7 points as well as other pre-training methods by more than 10 points in terms of top-20 retrieval accuracy under the zero-shot scenario. Furthermore, HLP significantly outperforms other pre-training methods under the other scenarios.

cs.CL↗

arXiv2022

Photomolecular Effect Leading to Water Evaporation Exceeding Thermal Limit

We report the discovery of photomolecular effect: cleavage of water clusters off surfaces by photons. This effect is demonstrated through surprising absorption of partially wetted hydrogel in the visible spectrum where both water and hydrogel materials' absorption are negligible. Illumination of hydrogel under solar or visible-spectrum light-emitting-diode leads to evaporation rates exceeding the thermal evaporation limit, even in hydrogels without additional absorbers. Measurements of temperature and transmission spectrum of vapor above evaporating surfaces show clear signatures of water clusters. The photomolecular effect happens at liquid-vapor interface due to large electrical field gradients and quadrupole force on molecular clusters. This photomolecular evaporation process might be happening widely in nature, potentially impacting climate and plants growth, and can be exploited for clean water and drying technologies.

physics.chem-ph↗

arXiv2021

Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing

Predicting linearized Abstract Meaning Representation (AMR) graphs using pre-trained sequence-to-sequence Transformer models has recently led to large improvements on AMR parsing benchmarks. These parsers are simple and avoid explicit modeling of structure but lack desirable properties such as graph well-formedness guarantees or built-in graph-sentence alignments. In this work we explore the integration of general pre-trained sequence-to-sequence language models and a structure-aware transition-based approach. We depart from a pointer-based transition system and propose a simplified transition set, designed to better exploit pre-trained language models for structured fine-tuning. We also explore modeling the parser state within the pre-trained encoder-decoder architecture and different vocabulary strategies for the same purpose. We provide a detailed comparison with recent progress in AMR parsing and show that the proposed parser retains the desirable properties of previous transition-based approaches, while being simpler and reaching the new parsing state of the art for AMR 2.0, without the need for graph re-categorization.

cs.CL↗

arXiv2021

Method for making multi-attribute decisions in wargames by combining intuitionistic fuzzy numbers with reinforcement learning

Researchers are increasingly focusing on intelligent games as a hot research area.The article proposes an algorithm that combines the multi-attribute management and reinforcement learning methods, and that combined their effect on wargaming, it solves the problem of the agent's low rate of winning against specific rules and its inability to quickly converge during intelligent wargame training.At the same time, this paper studied a multi-attribute decision making and reinforcement learning algorithm in a wargame simulation environment, and obtained data on red and blue conflict.Calculate the weight of each attribute based on the intuitionistic fuzzy number weight calculations. Then determine the threat posed by each opponent's chess pieces.Using the red side reinforcement learning reward function, the AC framework is trained on the reward function, and an algorithm combining multi-attribute decision-making with reinforcement learning is obtained. A simulation experiment confirms that the algorithm of multi-attribute decision-making combined with reinforcement learning presented in this paper is significantly more intelligent than the pure reinforcement learning algorithm.By resolving the shortcomings of the agent's neural network, coupled with sparse rewards in large-map combat games, this robust algorithm effectively reduces the difficulties of convergence. It is also the first time in this field that an algorithm design for intelligent wargaming combines multi-attribute decision making with reinforcement learning.Attempt interdisciplinary cross-innovation in the academic field, like designing intelligent wargames and improving reinforcement learning algorithms.

cs.AI↗

arXiv2021

AMR Parsing with Action-Pointer Transformer

Abstract Meaning Representation parsing is a sentence-to-graph prediction task where target nodes are not explicitly aligned to sentence tokens. However, since graph nodes are semantically based on one or more sentence tokens, implicit alignments can be derived. Transition-based parsers operate over the sentence from left to right, capturing this inductive bias via alignments at the cost of limited expressiveness. In this work, we propose a transition-based system that combines hard-attention over sentences with a target-side action pointer mechanism to decouple source tokens from node representations and address alignments. We model the transitions as well as the pointer mechanism through straightforward modifications within a single Transformer architecture. Parser state and graph structure information are efficiently encoded using attention heads. We show that our action-pointer approach leads to increased expressiveness and attains large gains (+1.6 points) against the best transition-based AMR parser in very similar conditions. While using no graph re-categorization, our single model yields the second best Smatch score on AMR 2.0 (81.8), which is further improved to 83.4 with silver data and ensemble decoding.

cs.CL↗

arXiv2021

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of automated metrics, datasets, and human evaluation standards. Due to this moving target, new models often still evaluate on divergent anglo-centric corpora with well-established, but flawed, metrics. This disconnect makes it challenging to identify the limitations of current models and opportunities for progress. Addressing this limitation, GEM provides an environment in which models can easily be applied to a wide set of tasks and in which evaluation strategies can be tested. Regular updates to the benchmark will help NLG research become more multilingual and evolve the challenge alongside models. This paper serves as the description of the data for which we are organizing a shared task at our ACL 2021 Workshop and to which we invite the entire NLG community to participate.

cs.CL↗

arXiv2020

On Statistical Efficiency in Learning

A central issue of many statistical learning problems is to select an appropriate model from a set of candidate models. Large models tend to inflate the variance (or overfitting), while small models tend to cause biases (or underfitting) for a given fixed dataset. In this work, we address the critical challenge of model selection to strike a balance between model fitting and model complexity, thus gaining reliable predictive power. We consider the task of approaching the theoretical limit of statistical learning, meaning that the selected model has the predictive performance that is as good as the best possible model given a class of potentially misspecified candidate models. We propose a generalized notion of Takeuchi's information criterion and prove that the proposed method can asymptotically achieve the optimal out-sample prediction loss under reasonable assumptions. It is the first proof of the asymptotic property of Takeuchi's information criterion to our best knowledge. Our proof applies to a wide variety of nonlinear models, loss functions, and high dimensionality (in the sense that the models' complexity can grow with sample size). The proposed method can be used as a computationally efficient surrogate for leave-one-out cross-validation. Moreover, for modeling streaming data, we propose an online algorithm that sequentially expands the model complexity to enhance selection stability and reduce computation cost. Experimental studies show that the proposed method has desirable predictive power and significantly less computational cost than some popular methods.

math.ST↗

arXiv2020

Improving Non-autoregressive Neural Machine Translation with Monolingual Data

Non-autoregressive (NAR) neural machine translation is usually done via knowledge distillation from an autoregressive (AR) model. Under this framework, we leverage large monolingual corpora to improve the NAR model's performance, with the goal of transferring the AR model's generalization ability while preventing overfitting. On top of a strong NAR baseline, our experimental results on the WMT14 En-De and WMT16 En-Ro news translation tasks confirm that monolingual data augmentation consistently improves the performance of the NAR model to approach the teacher AR model's performance, yields comparable or better results than the best non-iterative NAR methods in the literature and helps reduce overfitting in the training process.

cs.CL↗

arXiv2020

Automating Botnet Detection with Graph Neural Networks

Botnets are now a major source for many network attacks, such as DDoS attacks and spam. However, most traditional detection methods heavily rely on heuristically designed multi-stage detection criteria. In this paper, we consider the neural network design challenges of using modern deep learning techniques to learn policies for botnet detection automatically. To generate training data, we synthesize botnet connections with different underlying communication patterns overlaid on large-scale real networks as datasets. To capture the important hierarchical structure of centralized botnets and the fast-mixing structure for decentralized botnets, we tailor graph neural networks (GNN) to detect the properties of these structures. Experimental results show that GNNs are better able to capture botnet structure than previous non-learning methods when trained with appropriate data, and that deeper GNNs are crucial for learning difficult botnet topologies. We believe our data and studies can be useful for both the network security and graph learning communities.

cs.CR↗

arXiv2019

A Hierarchy of Graph Neural Networks Based on Learnable Local Features

Graph neural networks (GNNs) are a powerful tool to learn representations on graphs by iteratively aggregating features from node neighbourhoods. Many variant models have been proposed, but there is limited understanding on both how to compare different architectures and how to construct GNNs systematically. Here, we propose a hierarchy of GNNs based on their aggregation regions. We derive theoretical results about the discriminative power and feature representation capabilities of each class. Then, we show how this framework can be utilized to systematically construct arbitrarily powerful GNNs. As an example, we construct a simple architecture that exceeds the expressiveness of the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theory on both synthetic and real-world benchmarks, and demonstrate our example's theoretical power translates to strong results on node classification, graph classification, and graph regression tasks.

cs.LG↗

arXiv2019

Hydrodynamic limit of the Boltzmann equation to the planar rarefaction wave in three dimensional space

In this paper, we establish the global in time hydrodynamic limit of Boltzmann equation to the planar rarefaction wave of compressible Euler system in three dimensional space $x\in\mathbb{R}^3$ for general collision kernels. Our approch is based on a generalized Hilbert expansion, and a recent $L^2-L^\infty$ framework. In particular, we improve the $L^2$-estimate to be a localized version because the planar rarefaction wave is indeed a one-dimensional wave which makes the source terms to be not integrable in the $L^2$ energy estimate of three dimensional problem. We also point out that the wave strength of rarefaction may be large.

math.AP↗

arXiv2019

Simple Unsupervised Summarization by Contextual Matching

We propose an unsupervised method for sentence summarization using only language modeling. The approach employs two language models, one that is generic (i.e. pretrained), and the other that is specific to the target domain. We show that by using a product-of-experts criteria these are enough for maintaining continuous contextual matching while maintaining output fluency. Experiments on both abstractive and extractive sentence summarization data sets show promising results of our method without being exposed to any paired data.

cs.CL↗

arXiv2018

Simultaneously high electron and hole mobilities in cubic boron-V compounds: BP, BAs and BSb

Through first-principles calculations, the phonon-limited transport properties of cubic boron-V compounds (BP, BAs and BSb) are studied. We find that the high optical phonon frequency in these compounds leads to the substantial suppression of polar scattering and the reduction of inter-valley transition mediated by large-wavevector optical phonons, both of which significantly facilitate charge transport. We also discover that BAs simultaneously has a high hole mobility (2110 cm2/V-s) and electron mobility (1400 cm2/V-s) at room temperature, which is rare in semiconductors. Our findings present a new insight in searching high mobility polar semiconductors, and point to BAs as a promising material for electronic and photovoltaic devices in addition to its predicted high thermal conductivity.

cond-mat.mtrl-sci↗

arXiv2018

Efficiency Limits of Solar Energy Harvesting via Internal Photoemission in Carbon Materials

We describe strategies to estimate the upper limits of the efficiency of photon energy harvesting via hot electron extraction from gapless absorbers. Gapless materials such as noble metals can be used for harvesting the whole solar spectrum, including visible and near-infrared light. The energy of photo-generated non-equilibrium or hot charge carriers can be harvested before they thermalize with the crystal lattice via the process of their internal photo-emission (IPE) through the rectifying Schottky junction with a semiconductor. However, the low efficiency and the high cost of noble metals necessitates the search for cheaper abundant alternative materials, and we show here that carbon can serve as a promising IPE material candidate. We compare the upper limits of performance of IPE photon energy-harvesting platforms, which incorporate either gold or carbon as the photoactive material where hot electrons are generated. Through a combination of density functional theory, joint electron density of states calculations, and Schottky diode efficiency modeling, we show that the material electron band structure imposes a strict upper limit on the achievable efficiency of the IPE devices. Our calculations reveal that graphite is a good material candidate for the IPE absorber for harvesting visible and near-infrared photons. Graphite electron density of states yields a sizeable population of hot electrons with energies high enough to be collected across the potential barrier. We also discuss the mechanisms that prevent the IPE device efficiency from reaching the upper limits imposed by their material electron band structures. The proposed approach is general and allows for efficient pre-screening of materials for their potential use in IPE energy converters and photodetectors within application-specific spectral windows.

cond-mat.mtrl-sci↗