arXiv subjects

Ziyang Liu

Publications and source records attributed to Ziyang Liu.

59 records · Page 4Linked to original sources

arXiv2021

Heterogeneous Network Embedding for Deep Semantic Relevance Match in E-commerce Search

Result relevance prediction is an essential task of e-commerce search engines to boost the utility of search engines and ensure smooth user experience. The last few years eyewitnessed a flurry of research on the use of Transformer-style models and deep text-match models to improve relevance. However, these two types of models ignored the inherent bipartite network structures that are ubiquitous in e-commerce search logs, making these models ineffective. We propose in this paper a novel Second-order Relevance, which is fundamentally different from the previous First-order Relevance, to improve result relevance prediction. We design, for the first time, an end-to-end First-and-Second-order Relevance prediction model for e-commerce item relevance. The model is augmented by the neighborhood structures of bipartite networks that are built using the information of user behavioral feedback, including clicks and purchases. To ensure that edges accurately encode relevance information, we introduce external knowledge generated from BERT to refine the network of user behaviors. This allows the new model to integrate information from neighboring items and queries, which are highly relevant to the focus query-item pair under consideration. Results of offline experiments showed that the new model significantly improved the prediction accuracy in terms of human relevance judgment. An ablation study showed that the First-and-Second-order model gained a 4.3% average gain over the First-order model. Results of an online A/B test revealed that the new model derived more commercial benefits compared to the base model.

cs.IR↗

arXiv2020

BERT2DNN: BERT Distillation with Massive Unlabeled Data for Online E-Commerce Search

Relevance has significant impact on user experience and business profit for e-commerce search platform. In this work, we propose a data-driven framework for search relevance prediction, by distilling knowledge from BERT and related multi-layer Transformer teacher models into simple feed-forward networks with large amount of unlabeled data. The distillation process produces a student model that recovers more than 97\% test accuracy of teacher models on new queries, at a serving cost that's several magnitude lower (latency 150x lower than BERT-Base and 15x lower than the most efficient BERT variant, TinyBERT). The applications of temperature rescaling and teacher model stacking further boost model accuracy, without increasing the student model complexity. We present experimental results on both in-house e-commerce search relevance data as well as a public data set on sentiment analysis from the GLUE benchmark. The latter takes advantage of another related public data set of much larger scale, while disregarding its potentially noisy labels. Embedding analysis and case study on the in-house data further highlight the strength of the resulting model. By making the data processing and model training source code public, we hope the techniques presented here can help reduce energy consumption of the state of the art Transformer models and also level the playing field for small organizations lacking access to cutting edge machine learning hardwares.

cs.LG↗

arXiv2020

CORD-19: The COVID-19 Open Research Dataset

The COVID-19 Open Research Dataset (CORD-19) is a growing resource of scientific papers on COVID-19 and related historical coronavirus research. CORD-19 is designed to facilitate the development of text mining and information retrieval systems over its rich collection of metadata and structured full text papers. Since its release, CORD-19 has been downloaded over 200K times and has served as the basis of many COVID-19 text mining and discovery systems. In this article, we describe the mechanics of dataset construction, highlighting challenges and key design decisions, provide an overview of how CORD-19 has been used, and describe several shared tasks built around the dataset. We hope this resource will continue to bring together the computing community, biomedical experts, and policy makers in the search for effective treatments and management policies for COVID-19.

cs.DL↗

arXiv2020

Hybrid deep neural network based prediction method for unsteady flows with moving boundaries

A novel hybrid deep neural network architecture is designed to capture the spatial-temporal features of unsteady flows around moving boundaries directly from high-dimensional unsteady flow fields data. The hybrid deep neural network is constituted by the convolutional neural network (CNN), improved convolutional Long-Short Term Memory neural network (ConvLSTM) and deconvolutional neural network (DeCNN). Flow fields at future time step can be predicted through flow fields by previous time steps and boundary positions at those steps by the novel hybrid deep neural network. Unsteady wake flows around a forced oscillation cylinder with various amplitudes are calculated to establish the datasets as training samples for training the hybrid deep neural networks. The trained hybrid deep neural networks are then tested by predicting the unsteady flow fields around a forced oscillation cylinder with new amplitude. The effect of neural network structure parameters on prediction accuracy was analyzed. The hybrid deep neural network, constituted by the best parameter combination, is used to predict the flow fields in the future time. The predicted flow fields are in good agreement with those calculated directly by computational fluid dynamic solver, which means that this kind of deep neural network can capture accurate spatial-temporal information from the spatial-temporal series of unsteady flows around moving boundaries. The result shows the potential capability of this kind novel hybrid deep neural network in flow control for vibrating cylinder, where the fast calculation of high-dimensional nonlinear unsteady flow around moving boundaries is needed.

physics.comp-ph↗

arXiv2011

Query Expansion Based on Clustered Results

Query expansion is a functionality of search engines that suggests a set of related queries for a user-issued keyword query. Typical corpus-driven keyword query expansion approaches return popular words in the results as expanded queries. Using these approaches, the expanded queries may correspond to a subset of possible query semantics, and thus miss relevant results. To handle ambiguous queries and exploratory queries, whose result relevance is difficult to judge, we propose a new framework for keyword query expansion: we start with clustering the results according to user specified granularity, and then generate expanded queries, such that one expanded query is generated for each cluster whose result set should ideally be the corresponding cluster. We formalize this problem and show its APX-hardness. Then we propose two efficient algorithms named iterative single-keyword refinement and partial elimination based convergence, respectively, which effectively generate a set of expanded queries from clustered results that provide a classification of the original query results. We believe our study of generating an optimal query based on the ground truth of the query results not only has applications in query expansion, but has significance for studying keyword search quality in general.

cs.IR↗