arXiv subjects

Yunjiang Jiang

Publications and source records attributed to Yunjiang Jiang.

30 records · Page 2Linked to original sources

arXiv2020

BERT2DNN: BERT Distillation with Massive Unlabeled Data for Online E-Commerce Search

Relevance has significant impact on user experience and business profit for e-commerce search platform. In this work, we propose a data-driven framework for search relevance prediction, by distilling knowledge from BERT and related multi-layer Transformer teacher models into simple feed-forward networks with large amount of unlabeled data. The distillation process produces a student model that recovers more than 97\% test accuracy of teacher models on new queries, at a serving cost that's several magnitude lower (latency 150x lower than BERT-Base and 15x lower than the most efficient BERT variant, TinyBERT). The applications of temperature rescaling and teacher model stacking further boost model accuracy, without increasing the student model complexity. We present experimental results on both in-house e-commerce search relevance data as well as a public data set on sentiment analysis from the GLUE benchmark. The latter takes advantage of another related public data set of much larger scale, while disregarding its potentially noisy labels. Embedding analysis and case study on the in-house data further highlight the strength of the resulting model. By making the data processing and model training source code public, we hope the techniques presented here can help reduce energy consumption of the state of the art Transformer models and also level the playing field for small organizations lacking access to cutting edge machine learning hardwares.

cs.LG↗

arXiv2020

Fine-tune BERT for E-commerce Non-Default Search Ranking

The quality of non-default ranking on e-commerce platforms, such as based on ascending item price or descending historical sales volume, often suffers from acute relevance problems, since the irrelevant items are much easier to be exposed at the top of the ranking results. In this work, we propose a two-stage ranking scheme, which first recalls wide range of candidate items through refined query/title keyword matching, and then classifies the recalled items using BERT-Large fine-tuned on human label data. We also implemented parallel prediction on multiple GPU hosts and a C++ tokenization custom op of Tensorflow. In this data challenge, our model won the 1st place in the supervised phase (based on overall F1 score) and 2nd place in the final phase (based on average per query F1 score).

cs.IR↗

arXiv2020

Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning

Nowadays e-commerce search has become an integral part of many people's shopping routines. Two critical challenges stay in today's e-commerce search: how to retrieve items that are semantically relevant but not exact matching to query terms, and how to retrieve items that are more personalized to different users for the same search query. In this paper, we present a novel approach called DPSR, which stands for Deep Personalized and Semantic Retrieval, to tackle this problem. Explicitly, we share our design decisions on how to architect a retrieval system so as to serve industry-scale traffic efficiently and how to train a model so as to learn query and item semantics accurately. Based on offline evaluations and online A/B test with live traffics, we show that DPSR model outperforms existing models, and DPSR system can retrieve more personalized and semantically relevant items to significantly improve users' search experience by +1.29% conversion rate, especially for long tail queries by +10.03%. As a result, our DPSR system has been successfully deployed into JD.com's search production since 2019.

cs.IR↗

arXiv2018

Maximally Consistent Sampling and the Jaccard Index of Probability Distributions

We introduce simple, efficient algorithms for computing a MinHash of a probability distribution, suitable for both sparse and dense data, with equivalent running times to the state of the art for both cases. The collision probability of these algorithms is a new measure of the similarity of positive vectors which we investigate in detail. We describe the sense in which this collision probability is optimal for any Locality Sensitive Hash based on sampling. We argue that this similarity measure is more useful for probability distributions than the similarity pursued by other algorithms for weighted MinHash, and is the natural generalization of the Jaccard index.

cs.DS↗

arXiv2015

Cut-off phenomenon in the uniform plane Kac walk

We consider an analogue of the Kac random walk on the special orthogonal group $SO(N)$, in which at each step a random rotation is performed in a randomly chosen 2-plane of $\bR^N$. We obtain sharp asymptotics for the rate of convergence in total variance distance, establishing a cut-off phenomenon in the large $N$ limit. In the special case where the angle of rotation is deterministic this confirms a conjecture of Rosenthal \cite{Rosenthal}. Under mild conditions we also establish a cut-off for convergence of the walk to stationarity under the $L^2$ norm. Depending on the distribution of the randomly chosen angle of rotation, several surprising features emerge. For instance, it is sometimes the case that the mixing times differ in the total variation and $L^2$ norms. Our estimates use an integral representation of the characters of the special orthogonal group together with saddle point analysis.

math.PR↗

arXiv2015

Mixing time of Metropolis chain based on random transposition walk converging to multivariate Ewens distribution

We prove sharp rates of convergence to the Ewens equilibrium distribution for a family of Metropolis algorithms based on the random transposition shuffle on the symmetric group, with starting point at the identity. The proofs rely heavily on the theory of symmetric Jack polynomials, developed initially by Jack [Proc. Roy. Soc. Edinburgh Sect. A 69 (1970/1971) 1-18], Macdonald [Symmetric Functions and Hall Polynomials (1995) New York] and Stanley [Adv. Math. 77 (1989) 76-115]. This completes the analysis started by Diaconis and Hanlon in [Contemp. Math. 138 (1992) 99-117]. In the end we also explore other integrable Markov chains that can be obtained from symmetric function theory.

math.PR↗

arXiv2012

Total variation bound for Kac's random walk

We show that the classical Kac's random walk on $(n-1)$-sphere $S^{n-1}$ starting from the point mass at $e_1$ mixes in $\mathcal{O}(n^5(\log n)^3)$ steps in total variation distance. The main argument uses a truncation of the running density after a burn-in period, followed by $\mathcal{L}^2$ convergence using the spectral gap information derived by other authors. This improves upon a previous bound by Diaconis and Saloff-Coste of order $\mathcal {O}(n^{2n})$.

math.PR↗

arXiv2012

Smallest Gaps Between Eigenvalues of Random Matrices With Complex Ginibre, Wishart and Universal Unitary Ensembles

In this paper we study the limiting distribution of the $k$ smallest gaps between eigenvalues of three kinds of random matrices -- the Ginibre ensemble, the Wishart ensemble and the universal unitary ensemble. All of them follow a Poissonian ansatz. More precisely, for the Ginibre ensemble we have a global result in which the $k$-th smallest gap has typical length $n^{-3/4}$ with density $x^{4k-1}e^{-x^4}$ after normalization. For the Wishart and the universal unitary ensemble, it has typical length $n^{-4/3}$ and has density $x^{3k-1}e^{-x^3}$ after normalization.

math.PR↗

arXiv2011

Mixing time upper bound for the uniformized Rosenthal walk on the special orthogonal groups

We prove that a uniformized variant of both the Rosenthal walk \cite{Rosenthal} and the Kac random walk \cite{Kac} on SO(n) mixes in $\cO(n^3)$ steps in total variation distance. The proof also extends easily to Rosenthal walk with fixed angle $θ\neq π$. To the best of our knowledge, this is the first polynomial time bound for both walks. The techniques employed are mainly from representation theory of SO(n). But a crucial new ingredient is the interpretation of the Fourier coefficients of the character ratio as counting the number of particle cascade paths arising from the classical branching rules.

math.PR↗

arXiv2011

Asymptotic correlations of metrics on the symmetric groups

We consider the asymptotic joint distributions among several families of well-known metrics on $S_n$, the symmetric group. These include the bi-invariant metrics such as the Cayley and Hamming distance, and the left-invariant metrics such as Spearman's footrule, Kendall's tau, and the Ulam distance. We also introduce a natural limit of the Spearman family, $ρ_\infty$, and study its asymptotic distribution and relation with other metrics. This is a continuation of earlier work on the asymptotic independence of bi-invariant metrics on both $S_n$ and general linear groups over a finite field. The technique is based on some simple observation about the record map and Hammersley's device. In several cases, we give near-optimal estimate of the error term for asymptotic independence. This simplifies significantly the proof of a central limit theorem by Bai, Chao, and Liang regarding the oscillation of a permutation.

math.ST↗

arXiv2011

Asymptotic spectral independence of Wigner ensembles

We consider the joint distribution of eigenvalue clusters of the Wigner ensemble separated by macroscopic distances (i.e., on the same scale as the difference between the edges of the semicircle law). We prove that under an averaging condition, the correlation function governing any finite collection of clusters converges to that of independence point processes. The proof relies heavily on the machinery developed by Erdos, Ramirez, Schlein, and Yau.

math.PR↗

arXiv2011

On Number of Turns in Reduced Random Lattice Paths

We consider the tree-reduced path of symmetric random walk on $\ZZ^{d}$. It is interesting to ask about the number of turns $T_n$ in the reduced path after $n$ steps. This question arises from inverting signature for lattice paths. We show that, when $n$ is large, the mean and variance of $T_n$ have the same order as $n$, while the second order terms are O(1). We then use these estimates to obtain limit theorems for $T_n$. Similar results hold for any other finite patterns as well.

math.PR↗