arXiv subjects

Liu Yang

Publications and source records attributed to Liu Yang.

At least 217 records · Page 12Linked to original sources

arXiv2019

aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model

As an alternative to question answering methods based on feature engineering, deep learning approaches such as convolutional neural networks (CNNs) and Long Short-Term Memory Models (LSTMs) have recently been proposed for semantic matching of questions and answers. To achieve good results, however, these models have been combined with additional features such as word overlap or BM25 scores. Without this combination, these models perform significantly worse than methods based on linguistic feature engineering. In this paper, we propose an attention based neural matching model for ranking short answer text. We adopt value-shared weighting scheme instead of position-shared weighting scheme for combining different matching signals and incorporate question term importance learning using question attention network. Using the popular benchmark TREC QA data, we show that the relatively simple aNMM model can significantly outperform other neural network models that have been used for the question answering task, and is competitive with models that are combined with additional features. When aNMM is combined with additional features, it outperforms all baselines.

cs.IR↗

arXiv2019

A Faster Local Algorithm for Detecting Bounded-Size Cuts with Applications to Higher-Connectivity Problems

Consider the following "local" cut-detection problem in a directed graph: We are given a starting vertex $s$ and need to detect whether there is a cut with at most $k$ edges crossing the cut such that the side of the cut containing $s$ has at most $Δ$ interior edges. If we are given query access to the input graph, then this problem can in principle be solved in sublinear time without reading the whole graph and with query complexity depending on $k$ and $Δ$. We design an elegant randomized procedure that solves a slack variant of this problem with $O(k^2 Δ)$ queries, improving in particular a previous bound of $O((2(k+1))^{k+2} Δ)$ by Chechik et al. [SODA 2017]. In this slack variant, the procedure must successfully detect a component containing $s$ with at most $k$ outgoing edges and $Δ$ interior edges if such a component exists, but the component it actually detects may have up to $O(k Δ)$ interior edges. Besides being of interest on its own, such cut-detection procedures have been used in many algorithmic applications for higher-connectivity problems. Our new cut-detection procedure therefore almost readily implies (1) a faster vertex connectivity algorithm which in particular has nearly linear running time for polylogarithmic value of the vertex connectivity, (2) a faster algorithm for computing the maximal $k$-edge connected subgraphs, and (3) faster property testing algorithms for higher edge and vertex connectivity, which resolves two open problems, one by Goldreich and Ron [STOC '97] and one by Orenstein and Ron [TCS 2011].

cs.DS↗

arXiv2019

Contextual Aware Joint Probability Model Towards Question Answering System

In this paper, we address the question answering challenge with the SQuAD 2.0 dataset. We design a model architecture which leverages BERT's capability of context-aware word embeddings and BiDAF's context interactive exploration mechanism. By integrating these two state-of-the-art architectures, our system tries to extract the contextual word representation at word and character levels, for better comprehension of both question and context and their correlations. We also propose our original joint posterior probability predictor module and its associated loss functions. Our best model so far obtains F1 score of 75.842% and EM score of 72.24% on the test PCE leaderboad.

cs.CL↗

arXiv2019

Answer Interaction in Non-factoid Question Answering Systems

Information retrieval systems are evolving from document retrieval to answer retrieval. Web search logs provide large amounts of data about how people interact with ranked lists of documents, but very little is known about interaction with answer texts. In this paper, we use Amazon Mechanical Turk to investigate three answer presentation and interaction approaches in a non-factoid question answering setting. We find that people perceive and react to good and bad answers very differently, and can identify good answers relatively quickly. Our results provide the basis for further investigation of effective answer interaction and feedback methods.

cs.IR↗

arXiv2019

User Intent Prediction in Information-seeking Conversations

Conversational assistants are being progressively adopted by the general population. However, they are not capable of handling complicated information-seeking tasks that involve multiple turns of information exchange. Due to the limited communication bandwidth in conversational search, it is important for conversational assistants to accurately detect and predict user intent in information-seeking conversations. In this paper, we investigate two aspects of user intent prediction in an information-seeking setting. First, we extract features based on the content, structural, and sentiment characteristics of a given utterance, and use classic machine learning methods to perform user intent prediction. We then conduct an in-depth feature importance analysis to identify key features in this prediction task. We find that structural features contribute most to the prediction performance. Given this finding, we construct neural classifiers to incorporate context information and achieve better performance without feature engineering. Our findings can provide insights into the important factors and effective methods of user intent prediction in information-seeking conversations.

cs.IR↗

arXiv2018

Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching

Deep text matching approaches have been widely studied for many applications including question answering and information retrieval systems. To deal with a domain that has insufficient labeled data, these approaches can be used in a Transfer Learning (TL) setting to leverage labeled data from a resource-rich source domain. To achieve better performance, source domain data selection is essential in this process to prevent the "negative transfer" problem. However, the emerging deep transfer models do not fit well with most existing data selection methods, because the data selection policy and the transfer learning model are not jointly trained, leading to sub-optimal training efficiency. In this paper, we propose a novel reinforced data selector to select high-quality source domain data to help the TL model. Specifically, the data selector "acts" on the source domain data to find a subset for optimization of the TL model, and the performance of the TL model can provide "rewards" in turn to update the selector. We build the reinforced data selector based on the actor-critic framework and integrate it to a DNN based transfer learning model, resulting in a Reinforced Transfer Learning (RTL) method. We perform a thorough experimental evaluation on two major tasks for text matching, namely, paraphrase identification and natural language inference. Experimental results show the proposed RTL can significantly improve the performance of the TL model. We further investigate different settings of states, rewards, and policy optimization methods to examine the robustness of our method. Last, we conduct a case study on the selected data and find our method is able to select source domain data whose Wasserstein distance is close to the target domain data. This is reasonable and intuitive as such source domain data can provide more transferability power to the model.

cs.IR↗

arXiv2018

Deep learning based track reconstruction on CEPC luminometer

We study the track reconstruction algorithms of the CEPC luminometer. Depend on the current geometry design, the conventional track reconstruction method is applied, but it suffers the energy leakage problem when tracks falling into the tile gaps regions. To solve this problem, a novel reconstruction method based on deep neural networks has been investigated, and the reconstruction efficiency has been improved significantly, as well as the energy and direction resolutions. This new reconstruction method is proposed to replace the conventional one for the CEPC luminometer.

physics.ins-det↗

arXiv2018

$p$-adic singular integral and their commutator in generalized Morrey space

For a prime number $p,$ let $\mathbb{Q}_p$ be the field of $p$-adic numbers. In this paper, we established the boundedness of a class of $p$-adic singular integral operators on the $p$-adic generalized Morrey spaces. The corresponding boundedness for the commutators generalized by the $p$-adic singular integral operators and $p$-adic Lipschitz functions or $p$-adic generalized Campanato functions is also considered.

math.FA↗

arXiv2018

Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve in a unified manner forward, inverse and mixed stochastic problems based on a limited number of scattered measurements. Unlike standard GANs relying only on data for training, here we encoded into the architecture of GANs the governing physical laws in the form of stochastic differential equations (SDEs) using automatic differentiation. In particular, we applied Wasserstein GANs with gradient penalty (WGAN-GP) for its enhanced stability compared to vanilla GANs. We first tested WGAN-GP in approximating Gaussian processes of different correlation lengths based on data realizations collected from simultaneous reads at sparsely placed sensors. We obtained good approximation of the generated stochastic processes to the target ones even for a mismatch between the input noise dimensionality and the effective dimensionality of the target stochastic processes. We also studied the overfitting issue for both the discriminator and generator, and we found that overfitting occurs also in the generator in addition to the discriminator as previously reported. Subsequently, we considered the solution of elliptic SDEs requiring approximations of three stochastic processes, namely the solution, the forcing, and the diffusion coefficient. We used three generators for the PI-GANs, two of them were feed forward deep neural networks (DNNs) while the other one was the neural network induced by the SDE. Depending on the data, we employed one or multiple feed forward DNNs as the discriminators in PI-GANs. Here, we have demonstrated the accuracy and effectiveness of PI-GANs in solving SDEs for up to 30 dimensions, but in principle, PI-GANs could tackle very high dimensional problems given more sensor data with low-polynomial growth in computational cost.

stat.ML↗

arXiv2018

Injection locking of multiple auto-oscillation modes in a tapered nanowire spin Hall oscillator

Spin Hall oscillators (SHO) are promising candidates for the generation, detection and amplification of high frequency signals, that are tunable through a wide range of operating frequencies. They offer to be read out electrically, magnetically and optically in combination with a simple bilayer design. Here, we experimentally study the spatial dependence and spectral properties of auto-oscillations in SHO devices based on Pt(7 nm)/\ Ni$_{\mathrm{80}}$Fe$_{\mathrm{20}}$(5 nm) tapered nanowires. Using Brillouin light scattering microscopy, we observe two individual self-localized spin-wave bullets that oscillate at two distinct frequencies (5.2 GHz and 5.45 GHz) and are localized at different positions separated by about 750 nm within the SHO. This state of a tapered SHO has been predicted by a Ginzburg-Landau auto-oscillator model, but not yet been directly confirmed experimentally. We demonstrate that the observed bullets can be individually synchronized to external microwave signals, leading to a frequency entrainment, linewidth reduction and increase in oscillation amplitude for the bullet that is selected by the microwave frequency. At the same time, the amplitude of other parasitic modes decreases, which promotes the single-mode operation of the SHO. Finally, the synchronization of the spin-wave bullets is studied as a function of the microwave power. We believe that our findings promote the realization of extended spin Hall oscillators accomodating several distinct spin-wave bullets, that jointly cover an extended range of tunability.

cond-mat.mes-hall↗

arXiv2018

On Picard Type Theorems and Entire Solutions of Differential Equations

We give a connection between the Picard type theorem of Polya-Saxer-Milliox and characterization of entire solutions of a differential equation and then their higher dimensional extensions, which leads further results on both (ordinary and partial) differential equations and Picard type theorems.

math.CV↗

arXiv2018

Neural-net-induced Gaussian process regression for function approximation and PDE solution

Neural-net-induced Gaussian process (NNGP) regression inherits both the high expressivity of deep neural networks (deep NNs) as well as the uncertainty quantification property of Gaussian processes (GPs). We generalize the current NNGP to first include a larger number of hyperparameters and subsequently train the model by maximum likelihood estimation. Unlike previous works on NNGP that targeted classification, here we apply the generalized NNGP to function approximation and to solving partial differential equations (PDEs). Specifically, we develop an analytical iteration formula to compute the covariance function of GP induced by deep NN with an error-function nonlinearity. We compare the performance of the generalized NNGP for function approximations and PDE solutions with those of GPs and fully-connected NNs. We observe that for smooth functions the generalized NNGP can yield the same order of accuracy with GP, while both NNGP and GP outperform deep NN. For non-smooth functions, the generalized NNGP is superior to GP and comparable or superior to deep NN.

cs.LG↗

arXiv2018

Two and three electrons on a sphere: A generalized Thomson problem

Generalizing the classical Thomson problem to the quantum regime provides an ideal model to explore the underlying physics regarding electron correlations. In this work, we systematically investigate the combined effects of the geometry of the substrate and the symmetry of the wave function on correlations of geometrically confined electrons. By the numerical configuration interaction method in combination with analytical theory, we construct symmetrized ground-state wave functions; analyze the energetics, correlations, and collective vibration modes of the electrons; and illustrate the routine for the strongly correlated, highly localized electron states with the expansion of the sphere. This work furthers our understanding about electron correlations on confined geometries and shows the promising potential of exploiting confinement geometry to control electron states.

cond-mat.str-el↗

arXiv2018

Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce

Building multi-turn information-seeking conversation systems is an important and challenging research topic. Although several advanced neural text matching models have been proposed for this task, they are generally not efficient for industrial applications. Furthermore, they rely on a large amount of labeled data, which may not be available in real-world applications. To alleviate these problems, we study transfer learning for multi-turn information seeking conversations in this paper. We first propose an efficient and effective multi-turn conversation model based on convolutional neural networks. After that, we extend our model to adapt the knowledge learned from a resource-rich domain to enhance the performance. Finally, we deployed our model in an industrial chatbot called AliMe Assist (https://consumerservice.taobao.com/online-help) and observed a significant improvement over the existing online model.

cs.CL↗

arXiv2018

Statistical Learning under Nonstationary Mixing Processes

We study a special case of the problem of statistical learning without the i.i.d. assumption. Specifically, we suppose a learning method is presented with a sequence of data points, and required to make a prediction (e.g., a classification) for each one, and can then observe the loss incurred by this prediction. We go beyond traditional analyses, which have focused on stationary mixing processes or nonstationary product processes, by combining these two relaxations to allow nonstationary mixing processes. We are particularly interested in the case of $β$-mixing processes, with the sum of changes in marginal distributions growing sublinearly in the number of samples. Under these conditions, we propose a learning method, and establish that for bounded VC subgraph classes, the cumulative excess risk grows sublinearly in the number of predictions, at a quantified rate.

cs.LG↗

arXiv2018

Testing Piecewise Functions

This work explores the query complexity of property testing for general piecewise functions on the real line, in the active and passive property testing settings. The results are proven under an abstract zero-measure crossings condition, which has as special cases piecewise constant functions and piecewise polynomial functions. We find that, in the active testing setting, the query complexity of testing general piecewise functions is independent of the number of pieces. We also identify the optimal dependence on the number of pieces in the query complexity of passive testing in the special case of piecewise constant functions.

cs.DS↗

arXiv2018

WikiPassageQA: A Benchmark Collection for Research on Non-factoid Answer Passage Retrieval

With the rise in mobile and voice search, answer passage retrieval acts as a critical component of an effective information retrieval system for open domain question answering. Currently, there are no comparable collections that address non-factoid question answering within larger documents while simultaneously providing enough examples sufficient to train a deep neural network. In this paper, we introduce a new Wikipedia based collection specific for non-factoid answer passage retrieval containing thousands of questions with annotated answers and show benchmark results on a variety of state of the art neural architectures and retrieval models. The experimental results demonstrate the unique challenges presented by answer passage retrieval within topically relevant documents for future research.

cs.IR↗

arXiv2018

Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems

Intelligent personal assistant systems with either text-based or voice-based conversational interfaces are becoming increasingly popular around the world. Retrieval-based conversation models have the advantages of returning fluent and informative responses. Most existing studies in this area are on open domain "chit-chat" conversations or task / transaction oriented conversations. More research is needed for information-seeking conversations. There is also a lack of modeling external knowledge beyond the dialog utterances among current conversational models. In this paper, we propose a learning framework on the top of deep neural matching networks that leverages external knowledge for response ranking in information-seeking conversation systems. We incorporate external knowledge into deep neural models with pseudo-relevance feedback and QA correspondence knowledge distillation. Extensive experiments with three information-seeking conversation data sets including both open benchmarks and commercial data show that, our methods outperform various baseline methods including several deep text matching models and the state-of-the-art method on response selection in multi-turn conversations. We also perform analysis over different response types, model variations and ranking examples. Our models and research findings provide new insights on how to utilize external knowledge with deep neural models for response selection and have implications for the design of the next generation of information-seeking conversation systems.

cs.IR↗