arXiv subjects

Peng Li

Publications and source records attributed to Peng Li.

At least 379 records · Page 21Linked to original sources

arXiv2021

Demonstration of Panda: A Weakly Supervised Entity Matching System

Entity matching (EM) refers to the problem of identifying tuple pairs in one or more relations that refer to the same real world entities. Supervised machine learning (ML) approaches, and deep learning based approaches in particular, typically achieve state-of-the-art matching results. However, these approaches require many labeled examples, in the form of matching and non-matching pairs, which are expensive and time-consuming to label. In this paper, we introduce Panda, a weakly supervised system specifically designed for EM. Panda uses the same labeling function abstraction as Snorkel, where labeling functions (LF) are user-provided programs that can generate large amounts of (somewhat noisy) labels quickly and cheaply, which can then be combined via a labeling model to generate accurate final predictions. To support users developing LFs for EM, Panda provides an integrated development environment (IDE) that lives in a modern browser architecture. Panda's IDE facilitates the development, debugging, and life-cycle management of LFs in the context of EM tasks, similar to how IDEs such as Visual Studio or Eclipse excel in general-purpose programming. Panda's IDE includes many novel features purpose-built for EM, such as smart data sampling, a builtin library of EM utility functions, automatically generated LFs, visual debugging of LFs, and finally, an EM-specific labeling model. We show in this demo that Panda IDE can greatly accelerate the development of high-quality EM solutions using weak supervision.

cs.DB↗

arXiv2021

Evaluation of mechanical and energy properties for the phase field modeling of failure

In recent years, various phase field models have been developed in variational methods to simulate the failure of brittle solids. However, there is a lack of objective evaluation of the existing results, and in particular, there are few studies on model nonhomogeneous resolution, stress-strain linear elastic properties, and failure stress estimation. To compensate for the above gaps, the commonly used variational phase field model is systematically analyzed to solve the problem of evaluating the mechanics and energy properties of the model in this paper. The unified expression of the analytical solution and the nonhomogeneous solution under specific boundary conditions is analyzed and verified. Additionally, we theoretically analyze the energy properties of the phase field model and study the influence of the critical strain energy on the damage field, stress and strain of different models. Finally, the effect of different effective material parameters on the material failure stress is analyzed, the stress estimation formula for the failure of mode I of various phase field models is given, and the accuracy of the theory is verified by some examples.

math.NA↗

arXiv2021

Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation

Visual dialogue is a challenging task since it needs to answer a series of coherent questions on the basis of understanding the visual environment. Previous studies focus on the implicit exploration of multimodal co-reference by implicitly attending to spatial image features or object-level image features but neglect the importance of locating the objects explicitly in the visual content, which is associated with entities in the textual content. Therefore, in this paper we propose a {\bf M}ultimodal {\bf I}ncremental {\bf T}ransformer with {\bf V}isual {\bf G}rounding, named MITVG, which consists of two key parts: visual grounding and multimodal incremental transformer. Visual grounding aims to explicitly locate related objects in the image guided by textual entities, which helps the model exclude the visual content that does not need attention. On the basis of visual grounding, the multimodal incremental transformer encodes the multi-turn dialogue history combined with visual scene step by step according to the order of the dialogue and then generates a contextually and visually coherent response. Experimental results on the VisDial v0.9 and v1.0 datasets demonstrate the superiority of the proposed model, which achieves comparable performance.

cs.CL↗

arXiv2021

Facilitating Parallel Fuzzing with mutually-exclusive Task Distribution

Fuzz testing, or fuzzing, has become one of the de facto standard techniques for bug finding in the software industry. In general, fuzzing provides various inputs to the target program to discover unhandled exceptions and crashes. In business sectors where the time budget is limited, software vendors often launch many fuzzing instances in parallel as common means of increasing code coverage. However, most of the popular fuzzing tools in their parallel mode-naively run multiple instances concurrently, without elaborate distribution of workload. This can lead different instances to explore overlapped code regions, eventually reducing the benefits of concurrency. In this paper, we propose a general model to describe parallel fuzzing. This model distributes mutually-exclusive but similarly-weighted tasks to different instances, facilitating concurrency and also fairness across instances. Following this model, we develop a solution, called AFL-EDGE, to improve the parallel mode of AFL, considering a round of mutations to a unique seed as a task and adopting edge coverage to define the uniqueness of a seed. We have implemented AFL-EDGE on top of AFL and evaluated the implementation with AFL on 9 widely used benchmark programs. It shows that AFL-EDGE can benefit the edge coverage of AFL. In a 24-hour test, the increase of edge coverage brought by AFL-EDGE to AFL ranges from 9.49% to 10.20%, depending on the number of instances. As a side benefit, we discovered 14 previously unknown bugs.

cs.SE↗

arXiv2021

Magnetism and Spin Dynamics in Room-Temperature van der Waals Magnet Fe$_5$GeTe$_2$

Two-dimensional (2D) van der Waals (vdWs) materials have gathered a lot of attention recently. However, the majority of these materials have Curie temperatures that are well below room temperature, making it challenging to incorporate them into device applications. In this work, we synthesized a room-temperature vdW magnetic crystal Fe$_5$GeTe$_2$ with a Curie temperature T$_c = 332$ K, and studied its magnetic properties by vibrating sample magnetometry (VSM) and broadband ferromagnetic resonance (FMR) spectroscopy. The experiments were performed with external magnetic fields applied along the c-axis (H$\parallel$c) and the ab-plane (H$\parallel$ab), with temperatures ranging from 300 K to 10 K. We have found a sizable Landé g-factor difference between the H$\parallel$c and H$\parallel$ab cases. In both cases, the Landé g-factor values deviated from g = 2. This indicates contribution of orbital angular momentum to the magnetic moment. The FMR measurements reveal that Fe$_5$GeTe$_2$ has a damping constant comparable to Permalloy. With reducing temperature, the linewidth was broadened. Together with the VSM data, our measurements indicate that Fe$_5$GeTe$_2$ transitions from ferromagnetic to ferrimagnetic at lower temperatures. Our experiments highlight key information regarding the magnetic state and spin scattering processes in Fe$_5$GeTe$_2$, which promote the understanding of magnetism in Fe$_5$GeTe$_2$, leading to implementations of Fe$_5$GeTe$_2$ based room-temperature spintronic devices.

cond-mat.mtrl-sci↗

arXiv2021

WeChat Neural Machine Translation Systems for WMT21

This paper introduces WeChat AI's participation in WMT 2021 shared news translation task on English->Chinese, English->Japanese, Japanese->English and English->German. Our systems are based on the Transformer (Vaswani et al., 2017) with several novel and effective variants. In our experiments, we employ data filtering, large-scale synthetic data generation (i.e., back-translation, knowledge distillation, forward-translation, iterative in-domain knowledge transfer), advanced finetuning approaches, and boosted Self-BLEU based model ensemble. Our constrained systems achieve 36.9, 46.9, 27.8 and 31.3 case-sensitive BLEU scores on English->Chinese, English->Japanese, Japanese->English and English->German, respectively. The BLEU scores of English->Chinese, English->Japanese and Japanese->English are the highest among all submissions, and that of English->German is the highest among all constrained submissions.

cs.CL↗

arXiv2021

CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade

Dynamic early exiting aims to accelerate the inference of pre-trained language models (PLMs) by emitting predictions in internal layers without passing through the entire model. In this paper, we empirically analyze the working mechanism of dynamic early exiting and find that it faces a performance bottleneck under high speed-up ratios. On one hand, the PLMs' representations in shallow layers lack high-level semantic information and thus are not sufficient for accurate predictions. On the other hand, the exiting decisions made by internal classifiers are unreliable, leading to wrongly emitted early predictions. We instead propose a new framework for accelerating the inference of PLMs, CascadeBERT, which dynamically selects proper-sized and complete models in a cascading manner, providing comprehensive representations for predictions. We further devise a difficulty-aware objective, encouraging the model to output the class probability that reflects the real difficulty of each instance for a more reliable cascading mechanism. Experimental results show that CascadeBERT can achieve an overall 15\% improvement under 4$\times$ speed-up compared with existing dynamic early exiting methods on six classification tasks, yielding more calibrated and accurate predictions.

cs.CL↗

arXiv2021

Quantum cluster kink and ring frustration

In this paper, we work on the pure and mixed cluster models with periodic boundary condition. The first purpose is to establish the concept of quantum cluster kink. We clarify that there are two types of cluster kinks since there are two types of ground states depending on the choice of cluster length, of which the first type exhibits symmetry breaking order and the second one string order. Simple pictures are constructed according to the pure cluster model, which facilitates us to introduce the quantity, cluster kink number. As we demonstrate, cluster kinks deriving from different sources can coexist, compete with each other, and lead to quantum phase transition in a mixed cluster model. The second purpose is to elucidate that the effect of ring frustration can be realized in the cluster model with symmetry breaking order, instead of the one with string order. The reason lies in that ring frustration can induce a huge ground-state degeneracy or a special extended-kink phase with gapless excitations in the former case. And, although ring frustration does not change the phase transition point, it can produce unusual ground state in the extended-kink phase, whose special properties are uncovered by the nonlocal scaling factor in the correlation function and the doubled degeneracies of the eigenvalues of the entanglement spectrum.

cond-mat.str-el↗

arXiv2021

Giant Topological Hall Effect in van der Waals Heterostructures of CrTe2/Bi2Te3

Discoveries of interfacial topological Hall effect (THE) provide an ideal platform for exploring physics arising from the interplay between topology and magnetism. The interfacial topological Hall effect is closely related to the Dzyaloshinskii-Moriya interaction (DMI) at interface and topological spin textures. However, it is difficult to achieve a sizable THE in heterostructures due to the stringent constraints on the constituents of THE heterostructures such as strong spin-orbit coupling (SOC). Here we report the observation of a giant THE signal of 1.39 $μΩ\cdot$cm in the van der Waals heterostructures of CrTe2/Bi2Te3 fabricated by molecular beam epitaxy, a prototype of two-dimensional (2D) ferromagnet (FM)/topological insulator (TI). This large magnitude of THE is attributed to an optimized combination of 2D ferromagnetism in CrTe2, strong SOC in Bi2Te3, and an atomically sharp interface. Our work reveals CrTe2/Bi2Te3 as a convenient platform for achieving large interfacial THE in hybrid systems, which could be utilized to develop quantum science and high-density information storage.

cond-mat.mes-hall↗

arXiv2021

EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications

The literature has witnessed the success of leveraging Pre-trained Language Models (PLMs) and Transfer Learning (TL) algorithms to a wide range of Natural Language Processing (NLP) applications, yet it is not easy to build an easy-to-use and scalable TL toolkit for this purpose. To bridge this gap, the EasyTransfer platform is designed to develop deep TL algorithms for NLP applications. EasyTransfer is backended with a high-performance and scalable engine for efficient training and inference, and also integrates comprehensive deep TL algorithms, to make the development of industrial-scale TL applications easier. In EasyTransfer, the built-in data and model parallelism strategies, combined with AI compiler optimization, show to be 4.0x faster than the community version of distributed training. EasyTransfer supports various NLP models in the ModelZoo, including mainstream PLMs and multi-modality models. It also features various in-house developed TL algorithms, together with the AppZoo for NLP applications. The toolkit is convenient for users to quickly start model training, evaluation, and online deployment. EasyTransfer is currently deployed at Alibaba to support a variety of business scenarios, including item recommendation, personalized search, conversational question answering, etc. Extensive experiments on real-world datasets and online applications show that EasyTransfer is suitable for online production with cutting-edge performance for various applications. The source code of EasyTransfer is released at Github (https://github.com/alibaba/EasyTransfer).

cs.CL↗

arXiv2021

H2Learn: High-Efficiency Learning Accelerator for High-Accuracy Spiking Neural Networks

Although spiking neural networks (SNNs) take benefits from the bio-plausible neural modeling, the low accuracy under the common local synaptic plasticity learning rules limits their application in many practical tasks. Recently, an emerging SNN supervised learning algorithm inspired by backpropagation through time (BPTT) from the domain of artificial neural networks (ANNs) has successfully boosted the accuracy of SNNs and helped improve the practicability of SNNs. However, current general-purpose processors suffer from low efficiency when performing BPTT for SNNs due to the ANN-tailored optimization. On the other hand, current neuromorphic chips cannot support BPTT because they mainly adopt local synaptic plasticity rules for simplified implementation. In this work, we propose H2Learn, a novel architecture that can achieve high efficiency for BPTT-based SNN learning which ensures high accuracy of SNNs. At the beginning, we characterized the behaviors of BPTT-based SNN learning. Benefited from the binary spike-based computation in the forward pass and the weight update, we first design lookup table (LUT) based processing elements in Forward Engine and Weight Update Engine to make accumulations implicit and to fuse the computations of multiple input points. Second, benefited from the rich sparsity in the backward pass, we design a dual-sparsity-aware Backward Engine which exploits both input and output sparsity. Finally, we apply a pipeline optimization between different engines to build an end-to-end solution for the BPTT-based SNN learning. Compared with the modern NVIDIA V100 GPU, H2Learn achieves 7.38x area saving, 5.74-10.20x speedup, and 5.25-7.12x energy saving on several benchmark datasets.

cs.NE↗

arXiv2021

Revealing the Heavy Quasiparticles in the Heavy-Fermion Superconductor CeCu2Si2

The superconducting order parameter of the first heavy-fermion superconductor CeCu2Si2 is currently under debate. A key ingredient to understand its superconductivity and physical properties is the quasiparticle dispersion and Fermi surface, which remains elusive experimentally. Here we present measurements from angle-resolved photoemission spectroscopy. Our results emphasize the key role played by the Ce 4f electrons for the low-temperature Fermi surface, highlighting a band-dependent conduction-f electron hybridization. In particular, we find a very heavy quasi-two-dimensional electron band near the bulk X point and moderately heavy three-dimensional hole pockets near the Z point. Comparison with theoretical calculations reveals the strong local correlation in this compound, calling for further theoretical studies. Our results provide the electronic basis to understand the heavy fermion behavior and superconductivity; implications for the enigmatic superconductivity of this compound are also discussed.

cond-mat.str-el↗

arXiv2021

Backpropagated Neighborhood Aggregation for Accurate Training of Spiking Neural Networks

While backpropagation (BP) has been applied to spiking neural networks (SNNs) achieving encouraging results, a key challenge involved is to backpropagate a continuous-valued loss over layers of spiking neurons exhibiting discontinuous all-or-none firing activities. Existing methods deal with this difficulty by introducing compromises that come with their own limitations, leading to potential performance degradation. We propose a novel BP-like method, called neighborhood aggregation (NA), which computes accurate error gradients guiding weight updates that may lead to discontinuous modifications of firing activities. NA achieves this goal by aggregating finite differences of the loss over multiple perturbed membrane potential waveforms in the neighborhood of the present membrane potential of each neuron while utilizing a new membrane potential distance function. Our experiments show that the proposed NA algorithm delivers the state-of-the-art performance for SNN training on several datasets.

cs.NE↗

arXiv2021

Sample Recycling Method -- A New Approach to Efficient Nested Monte Carlo Simulations

Nested stochastic modeling has been on the rise in many fields of the financial industry. Such modeling arises whenever certain components of a stochastic model are stochastically determined by other models. There are at least two main areas of applications, including (1) portfolio risk management in the banking sector and (2) principle-based reserving and capital requirements in the insurance sector. As financial instrument values often change with economic fundamentals, the risk management of a portfolio (outer loop) often requires the assessment of financial positions subject to changes in risk factors in the immediate future. The valuation of financial position (inner loop) is based on projections of cashflows and risk factors into the distant future. The nesting of such stochastic modeling can be computationally challenging. Most of existing techniques to speed up nested simulations are based on curve fitting. The main idea is to establish a functional relationship between inner loop estimator and risk factors by running a limited set of economic scenarios, and, instead of running inner loop simulations, inner loop estimations are made by feeding other scenarios into the fitted curve. This paper presents a non-conventional approach based on the concept of sample recycling. Its essence is to run inner loop estimation for a small set of outer loop scenarios and to find inner loop estimates under other outer loop scenarios by recycling those known inner loop paths. This new approach can be much more efficient when traditional techniques are difficult to implement in practice.

q-fin.CP↗

arXiv2021

Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks

Spiking neural networks (SNNs) are well suited for spatio-temporal learning and implementations on energy-efficient event-driven neuromorphic processors. However, existing SNN error backpropagation (BP) methods lack proper handling of spiking discontinuities and suffer from low performance compared with the BP methods for traditional artificial neural networks. In addition, a large number of time steps are typically required to achieve decent performance, leading to high latency and rendering spike-based computation unscalable to deep architectures. We present a novel Temporal Spike Sequence Learning Backpropagation (TSSL-BP) method for training deep SNNs, which breaks down error backpropagation across two types of inter-neuron and intra-neuron dependencies and leads to improved temporal learning precision. It captures inter-neuron dependencies through presynaptic firing times by considering the all-or-none characteristics of firing activities and captures intra-neuron dependencies by handling the internal evolution of each neuronal state in time. TSSL-BP efficiently trains deep SNNs within a much shortened temporal window of a few steps while improving the accuracy for various image classification datasets including CIFAR10.

cs.NE↗

arXiv2021

Charge-Spin Interconversion in Epitaxial Pt Probed by Spin-Orbit Torques in a Magnetic Insulator

We measure spin-orbit torques (SOTs) in a unique model system of all-epitaxial ferrite/Pt bilayers to gain insights into charge-spin interconversion in Pt. With negligible electronic conduction in the insulating ferrite, the crystalline Pt film acts as the sole source of charge-to-spin conversion. A small field-like SOT independent of Pt thickness suggests a weak Rashba-Edelstein effect at the ferrite/Pt interface. By contrast, we observe a sizable damping-like SOT that depends on the Pt thickness, from which we deduce the dominance of an extrinsic spin-Hall effect (skew scattering) and Dyakonov-Perel spin relaxation in the crystalline Pt film. Furthermore, our results point to a large internal spin-Hall ratio of $\approx$0.8 in epitaxial Pt. Our experimental work takes an essential step towards understanding the mechanisms of charge-spin interconversion and SOTs in Pt-based heterostructures, which are crucial for power-efficient spintronic devices.

cond-mat.mes-hall↗

arXiv2021

CLEVE: Contrastive Pre-training for Event Extraction

Event extraction (EE) has considerably benefited from pre-trained language models (PLMs) by fine-tuning. However, existing pre-training methods have not involved modeling event characteristics, resulting in the developed EE models cannot take full advantage of large-scale unsupervised data. To this end, we propose CLEVE, a contrastive pre-training framework for EE to better learn event knowledge from large unsupervised data and their semantic structures (e.g. AMR) obtained with automatic parsers. CLEVE contains a text encoder to learn event semantics and a graph encoder to learn event structures respectively. Specifically, the text encoder learns event semantic representations by self-supervised contrastive learning to represent the words of the same events closer than those unrelated words; the graph encoder learns event structure representations by graph contrastive pre-training on parsed event-related semantic structures. The two complementary representations then work together to improve both the conventional supervised EE and the unsupervised "liberal" EE, which requires jointly extracting events and discovering event schemata without any annotated data. Experiments on ACE 2005 and MAVEN datasets show that CLEVE achieves significant improvements, especially in the challenging unsupervised setting. The source code and pre-trained checkpoints can be obtained from https://github.com/THU-KEG/CLEVE.

cs.CL↗

arXiv2021

The Dantzig selector: Recovery of Signal via $\ell_1-α\ell_2$ Minimization

In the paper, we proposed the Dantzig selector based on the $\ell_{1}-α\ell_{2}$~$(0< α\leq1)$ minimization for the signal recovery. In the Dantzig selector, the constraint $\|{\bf A}^{\top}({\bf b}-{\bf A}{\bf x})\|_\infty \leq η$ for some small constant $η>0$ means the columns of ${\bf A}$ has very weakly correlated with the error vector ${\bf e}={\bf A}{\bf x}-{\bf b}$. First, recovery guarantees based on the restricted isometry property (RIP) are established for signals. Next, we propose the effective algorithm to solve the proposed Dantzig selector. Last, we illustrate the proposed model and algorithm by extensive numerical experiments for the recovery of signals in the cases of Gaussian, impulsive and uniform noise. And the performance of the proposed Dantzig selector is better than that of the existing methods.

cs.IT↗