arXiv subjects

Wei Peng

Publications and source records attributed to Wei Peng.

At least 163 records · Page 9Linked to original sources

arXiv2021

HyKnow: End-to-End Task-Oriented Dialog Modeling with Hybrid Knowledge Management

Task-oriented dialog (TOD) systems typically manage structured knowledge (e.g. ontologies and databases) to guide the goal-oriented conversations. However, they fall short of handling dialog turns grounded on unstructured knowledge (e.g. reviews and documents). In this paper, we formulate a task of modeling TOD grounded on both structured and unstructured knowledge. To address this task, we propose a TOD system with hybrid knowledge management, HyKnow. It extends the belief state to manage both structured and unstructured knowledge, and is the first end-to-end model that jointly optimizes dialog modeling grounded on these two kinds of knowledge. We conduct experiments on the modified version of MultiWOZ 2.1 dataset, where dialogs are grounded on hybrid knowledge. Experimental results show that HyKnow has strong end-to-end performance compared to existing TOD systems. It also outperforms the pipeline knowledge management schemes, with higher unstructured knowledge retrieval accuracy.

cs.CL↗

arXiv2021

A hybrid quantum-classical neural network with deep residual learning

Inspired by the success of classical neural networks, there has been tremendous effort to develop classical effective neural networks into quantum concept. In this paper, a novel hybrid quantum-classical neural network with deep residual learning (Res-HQCNN) is proposed. We firstly analysis how to connect residual block structure with a quantum neural network, and give the corresponding training algorithm. At the same time, the advantages and disadvantages of transforming deep residual learning into quantum concept are provided. As a result, the model can be trained in an end-to-end fashion, analogue to the backpropagation in classical neural networks. To explore the effectiveness of Res-HQCNN , we perform extensive experiments for quantum data with or without noisy on classical computer. The experimental results show the Res-HQCNN performs better to learn an unknown unitary transformation and has stronger robustness for noisy data, when compared to state of the arts. Moreover, the possible methods of combining residual learning with quantum neural networks are also discussed.

cs.LG↗

arXiv2021

MCR-Net: A Multi-Step Co-Interactive Relation Network for Unanswerable Questions on Machine Reading Comprehension

Question answering systems usually use keyword searches to retrieve potential passages related to a question, and then extract the answer from passages with the machine reading comprehension methods. However, many questions tend to be unanswerable in the real world. In this case, it is significant and challenging how the model determines when no answer is supported by the passage and abstains from answering. Most of the existing systems design a simple classifier to determine answerability implicitly without explicitly modeling mutual interaction and relation between the question and passage, leading to the poor performance for determining the unanswerable questions. To tackle this problem, we propose a Multi-Step Co-Interactive Relation Network (MCR-Net) to explicitly model the mutual interaction and locate key clues from coarse to fine by introducing a co-interactive relation module. The co-interactive relation module contains a stack of interaction and fusion blocks to continuously integrate and fuse history-guided and current-query-guided clues in an explicit way. Experiments on the SQuAD 2.0 and DuReader datasets show that our model achieves a remarkable improvement, outperforming the BERT-style baselines in literature. Visualization analysis also verifies the importance of the mutual interaction between the question and passage.

cs.CL↗

arXiv2021

Enhancement of the superconductivity and quantum metallic state in the thin film of superconducting Kagome metal KV$_3$Sb$_5$

Recently V-based Kagome metal attracted intense attention due to the emergence of superconductivity in the low temperature. Here we report the fabrication and physical investigations of the high quality single-crystalline thin films of the Kagome metal KV$_3$Sb$_5$. For the sample with the thickness of about 15 nm, the temperature dependent resistance reveals a Berezinskii-Kosterlitz-Thouless (BKT) type behavior, indicating the presence of two-dimensional superconductivity. Compared with the bulk sample, the onset transition temperature $T^{onset}_{c}$ and the out-of-plane upper critical field $H_{c2}$ are enhanced by 15\% and more than 10 times respectively. Moreover, the zero-resistance state is destroyed by a magnetic field as low as 50 Oe. Meanwhile, the temperature-independent resistance is observed in a wide field region, which is the hallmark of quantum metallic state. Our results provide evidences for the existence of unconventional superconductivity in this material.

cond-mat.supr-con↗

arXiv2021

Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations

There have been several efforts to Physics-informed neural networks (PINNs) in the solution of the incompressible Navier-Stokes fluid. The loss function in PINNs is a weighted sum of multiple terms, including the mismatch in the observed velocity and pressure data, the boundary and initial constraints, as well as the residuals of the Navier-Stokes equations. In this paper, we observe that the weighted combination of competitive multiple loss functions plays a significant role in training PINNs effectively. We establish Gaussian probabilistic models to define the loss terms, where the noise collection describes the weight parameter for each loss term. We propose a self-adaptive loss function method, which automatically assigns the weights of losses by updating the noise parameters in each epoch based on the maximum likelihood estimation. Subsequently, we employ the self-adaptive loss balanced Physics-informed neural networks (lbPINNs) to solve the incompressible Navier-Stokes equations,\hspace{-1pt} including\hspace{-1pt} two-dimensional\hspace{-1pt} steady Kovasznay flow, two-dimensional unsteady cylinder wake, and three-dimensional unsteady Beltrami flow. Our results suggest that the accuracy of PINNs for effectively simulating complex incompressible flows is improved by adaptively appropriate weights in the loss terms. The outstanding adaptability of lbPINNs is not irrelevant to the initialization choice of noise parameters, which illustrates the robustness. The proposed method can also be employed in other problems where PINNs apply besides fluid problems.

physics.flu-dyn↗

arXiv2021

Free-Running Long-Distance Reference-Frame-Independent Quantum Key Distribution

Rapidly and randomly drifted reference frames will shorten the link distance and decrease the secure key rate of realistic quantum key distribution (QKD) systems. However, an actively or inappropriately implemented calibration scheme will increase complexity of the systems and may open security loopholes. In this article, we present a free-running reference-frame-independent (RFI) QKD scheme, where measurement events are classified into multiple slices with the same misalignment variation of reference frames and each slice performs the post-processing procedure individually. We perform the free-running RFI QKD experiment with a fiber link of 100km and the misalignment of the reference frame between Alice and Bob is varied more than 29 periods in a 50.7-hour experiment test. The average secure key rate is about 734 bps with a total loss of 31.5 dB, which achieves the state-of-art performance of the long-distance RFI QKD implementations. Our free-running RFI scheme can be efficiently adapted into the satellite-to-ground and drone based mobile communication scenarios, as it can be performed with rapidly varying reference frame and a loss more than 40 dB, where no secure key can be obtained by the original RFI scheme.

quant-ph↗

arXiv2021

IIE-NLP-Eyas at SemEval-2021 Task 4: Enhancing PLM for ReCAM with Special Tokens, Re-Ranking, Siamese Encoders and Back Translation

This paper introduces our systems for all three subtasks of SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning. To help our model better represent and understand abstract concepts in natural language, we well-design many simple and effective approaches adapted to the backbone model (RoBERTa). Specifically, we formalize the subtasks into the multiple-choice question answering format and add special tokens to abstract concepts, then, the final prediction of question answering is considered as the result of subtasks. Additionally, we employ many finetuning tricks to improve the performance. Experimental results show that our approaches achieve significant performance compared with the baseline systems. Our approaches achieve eighth rank on subtask-1 and tenth rank on subtask-2.

cs.CL↗

arXiv2021

Thermal Transport Dynamics in Active Heat Transfer Fluids (AHTF)

We present results of molecular dynamics (MD) calculations of the effective thermal conductivity of nanofluids containing self-propelled nanoparticles. The translational and rotational dynamics observed in the simulations follow the behavior expected from the standard theoretical analysis of Brownian and self-propelled nanoparticles. The superposition of self-propulsion and rotational Brownian motion causes the behavior of the self-propelled nanoparticles to resemble Brownian diffusion with an effective coefficient that is larger than the standard Brownian value by a factor of several thousand. As a result of the enhanced diffusion (and the convective mixing resulting from the motion), we observe a discriminable increase of the effective thermal conductivity of the solution. While the increases we observe are in the range of several percent, they are significant considering that, without propulsion, the nanofluid thermal conductivity is essentially not affected by the Brownian motion and can be understood within the effective medium theory of thermal conduction. Our results constitute a proof of concept that self-propelled particles have the potential to enhance thermal conductivity of the liquid in which they are immersed, an idea that could ultimately be implemented in a broad variety of cooling applications.

cond-mat.soft↗

arXiv2021

Hyperbolic Deep Neural Networks: A Survey

Recently, there has been a rising surge of momentum for deep representation learning in hyperbolic spaces due to theirhigh capacity of modeling data like knowledge graphs or synonym hierarchies, possessing hierarchical structure. We refer to the model as hyperbolic deep neural network in this paper. Such a hyperbolic neural architecture potentially leads to drastically compact model withmuch more physical interpretability than its counterpart in Euclidean space. To stimulate future research, this paper presents acoherent and comprehensive review of the literature around the neural components in the construction of hyperbolic deep neuralnetworks, as well as the generalization of the leading deep approaches to the Hyperbolic space. It also presents current applicationsaround various machine learning tasks on several publicly available datasets, together with insightful observations and identifying openquestions and promising future directions.

cs.LG↗

arXiv2021

Crystal structure and properties of iron-based spin-chain compound Ba9Fe3Se15

We report the synthesis of a new quasi one-dimensional (1D) iron selenide. Ba9Fe3Se15 was synthesized at high temperature and high pressure of 5.5 GPa and systematically studied via structural, magnetic and transport measurements at ambient and at high-pressures. Ba9Fe3Se15 crystallizes in a monoclinic structure and consists of face-sharing FeSe6 octahedral chains along the c axis. At ambient pressure it exhibits an insulating behavior with a band gap ~460 meV and undergoes a ferrimagnet-like phase transition at 14 K. Under high pressure, a complete metallization occurs at ~29 GPa, which is accompanied by a spin state crossover from high spin (HS) state to low spin (LS) state. The LS appears for pressures P >36 GPa.

cond-mat.supr-con↗

arXiv2021

Growth and atomically resolved polarization mapping of ferroelectric $Bi_2WO_6$ thin film

Aurivillius ferroelectric $Bi_2WO_6$ (BWO) encompasses a broad range of functionalities, including robust fatigue-free ferroelectricity, high photocatalytic activity, and ionic conductivity. Despite these promising characteristics, an in-depth study on the growth of BWO thin films and ferroelectric characterization, especially at the atomic scale, is still lacking. Here, we report pulsed laser deposition (PLD) of BWO thin films on (001) $SrTiO_3$ substrates and characterization of ferroelectricity using the scanning transmission electron microscopy (STEM) and piezoresponse force microscopy (PFM) techniques. We show that the background oxygen gas pressure used during PLD growth mainly determines the phase stability of BWO films, whereas the influence of growth temperature is comparatively minor. Atomically resolved STEM study of a fully strained BWO film revealed collective in-plane polar off-centering displacement of W atoms. We estimated the spontaneous polarization value based on polar displacement mapping to be about 54 $\pm$ 4 $μC cm^{-2}$, which is in good agreement with the bulk polarization value. Furthermore, we found that pristine film is composed of type-I and type-II domains, with mutually orthogonal polar axes. Complementary PFM measurements further elucidated that the coexisting type-I and type-II domains formed a multidomain state that consisted of 90$°$ domain walls (DWs) alongside multiple head-to-head and tail-to-tail 180$°$ DWs. Application of an electrical bias led to in-plane 180$°$ polarization switching and 90$°$ polarization rotation, highlighting a unique aspect of domain switching, which is immune to substrate-induced strain.

cond-mat.mtrl-sci↗

arXiv2020

Development of Multi-Layer Fabrication Process for SFQ Large Scale Integrated Digital Circuits

We have developed a fabrication technology for the development of large-scale superconducting integrated circuits with Nb-based Josephson junctions. The standard fabrication process with 10 mask levels uses four metal layers including 3 Nb superconducting layers and a Mo resistor layer. The influence of deposition parameters on film stress, electrical properties, and surface roughness were studied systematically. High quality Nb, Al, Mo, and SiO2 films were successfully deposited for the subsequent fabrication of circuits. The circuit fabrication started with the fabrication of Mo resistors with a target sheet resistance Rsh of 2 Ome, followed by the deposition of Josephson-junction. The target critical current density Jc was set at 6 kA per cm2. The thicknesses and etch depths of the films were monitored during fabrication with on-wafer process-control-monitor (PCM) patterns for all the wafers.

cond-mat.supr-con↗

arXiv2020

Bi-directional Cognitive Thinking Network for Machine Reading Comprehension

We propose a novel Bi-directional Cognitive Knowledge Framework (BCKF) for reading comprehension from the perspective of complementary learning systems theory. It aims to simulate two ways of thinking in the brain to answer questions, including reverse thinking and inertial thinking. To validate the effectiveness of our framework, we design a corresponding Bi-directional Cognitive Thinking Network (BCTN) to encode the passage and generate a question (answer) given an answer (question) and decouple the bi-directional knowledge. The model has the ability to reverse reasoning questions which can assist inertial thinking to generate more accurate answers. Competitive improvement is observed in DuReader dataset, confirming our hypothesis that bi-directional knowledge helps the QA task. The novel framework shows an interesting perspective on machine reading comprehension and cognitive science.

cs.CL↗

arXiv2020

2nd Place Scheme on Action Recognition Track of ECCV 2020 VIPriors Challenges: An Efficient Optical Flow Stream Guided Framework

To address the problem of training on small datasets for action recognition tasks, most prior works are either based on a large number of training samples or require pre-trained models transferred from other large datasets to tackle overfitting problems. However, it limits the research within organizations that have strong computational abilities. In this work, we try to propose a data-efficient framework that can train the model from scratch on small datasets while achieving promising results. Specifically, by introducing a 3D central difference convolution operation, we proposed a novel C3D neural network-based two-stream (Rank Pooling RGB and Optical Flow) framework for the task. The method is validated on the action recognition track of the ECCV 2020 VIPriors challenges and got the 2nd place (88.31%). It is proved that our method can achieve a promising result even without a pre-trained model on large scale datasets. The code will be released soon.

cs.CV↗

arXiv2020

Mix Dimension in Poincaré Geometry for 3D Skeleton-based Action Recognition

Graph Convolutional Networks (GCNs) have already demonstrated their powerful ability to model the irregular data, e.g., skeletal data in human action recognition, providing an exciting new way to fuse rich structural information for nodes residing in different parts of a graph. In human action recognition, current works introduce a dynamic graph generation mechanism to better capture the underlying semantic skeleton connections and thus improves the performance. In this paper, we provide an orthogonal way to explore the underlying connections. Instead of introducing an expensive dynamic graph generation paradigm, we build a more efficient GCN on a Riemann manifold, which we think is a more suitable space to model the graph data, to make the extracted representations fit the embedding matrix. Specifically, we present a novel spatial-temporal GCN (ST-GCN) architecture which is defined via the Poincaré geometry such that it is able to better model the latent anatomy of the structure data. To further explore the optimal projection dimension in the Riemann space, we mix different dimensions on the manifold and provide an efficient way to explore the dimension for each ST-GCN layer. With the final resulted architecture, we evaluate our method on two current largest scale 3D datasets, i.e., NTU RGB+D and NTU RGB+D 120. The comparison results show that the model could achieve a superior performance under any given evaluation metrics with only 40\% model size when compared with the previous best GCN method, which proves the effectiveness of our model.

cs.CV↗

arXiv2020

Miniaturization of the Superconducting Memory Cell via a Three-Dimensional Nb Nano-Superconducting Quantum Interference Device

Scalable memories that can match the speeds of superconducting logic circuits have long been desired to enable a superconducting computer. A superconducting loop that includes a Josephson junction can store a flux quantum state in picoseconds. However, the requirement for the loop inductance to create a bi-state hysteresis sets a limit on the minimal area occupied by a single memory cell. Here, we present a miniaturized superconducting memory cell based on a Three-Dimensional (3D) Nb nano-Superconducting QUantum Interference Device (nano-SQUID). The major cell area here fits within an 8*9 μm^2 rectangle with a cross-selected function for memory implementation. The cell shows periodic tunable hysteresis between two neighbouring flux quantum states produced by bias current sweeping because of the large modulation depth of the 3D nano-SQUID (~66%). Furthermore, the measured Current-Phase Relations (CPRs) of nano-SQUIDs are shown to be skewed from a sine function, as predicted by theoretical modelling. The skewness and the critical current of 3D nano-SQUIDs are linearly correlated. It is also found that the hysteresis loop size is in a linear scaling relationship with the CPR skewness using the statistics from characterisation of 26 devices. We show that the CPR skewness range of π/4~3π/4 is equivalent to a large loop inductance in creating a stable bi-state hysteresis for memory implementation. Therefore, the skewed CPR of 3D nano-SQUID enables further superconducting memory cell miniaturization by overcoming the inductance limitation of the loop area.

cond-mat.mes-hall↗

arXiv2020

IIE-NLP-NUT at SemEval-2020 Task 4: Guiding PLM with Prompt Template Reconstruction Strategy for ComVE

This paper introduces our systems for the first two subtasks of SemEval Task4: Commonsense Validation and Explanation. To clarify the intention for judgment and inject contrastive information for selection, we propose the input reconstruction strategy with prompt templates. Specifically, we formalize the subtasks into the multiple-choice question answering format and construct the input with the prompt templates, then, the final prediction of question answering is considered as the result of subtasks. Experimental results show that our approaches achieve significant performance compared with the baseline systems. Our approaches secure the third rank on both official test sets of the first two subtasks with an accuracy of 96.4 and an accuracy of 94.3 respectively.

cs.CL↗

arXiv2020

Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition

Composite-database micro-expression recognition is attracting increasing attention as it is more practical to real-world applications. Though the composite database provides more sample diversity for learning good representation models, the important subtle dynamics are prone to disappearing in the domain shift such that the models greatly degrade their performance, especially for deep models. In this paper, we analyze the influence of learning complexity, including the input complexity and model complexity, and discover that the lower-resolution input data and shallower-architecture model are helpful to ease the degradation of deep models in composite-database task. Based on this, we propose a recurrent convolutional network (RCN) to explore the shallower-architecture and lower-resolution input data, shrinking model and input complexities simultaneously. Furthermore, we develop three parameter-free modules (i.e., wide expansion, shortcut connection and attention unit) to integrate with RCN without increasing any learnable parameters. These three modules can enhance the representation ability in various perspectives while preserving not-very-deep architecture for lower-resolution data. Besides, three modules can further be combined by an automatic strategy (a neural architecture search strategy) and the searched architecture becomes more robust. Extensive experiments on MEGC2019 dataset (composited of existing SMIC, CASME II and SAMM datasets) have verified the influence of learning complexity and shown that RCNs with three modules and the searched combination outperform the state-of-the-art approaches.

cs.CV↗