arXiv subjects

Yingrui Yang

Publications and source records attributed to Yingrui Yang.

13 recordsLinked to original sources

arXiv2025

LSTM-based Selective Dense Text Retrieval Guided by Sparse Lexical Retrieval

This paper studies fast fusion of dense retrieval and sparse lexical retrieval, and proposes a cluster-based selective dense retrieval method called CluSD guided by sparse lexical retrieval. CluSD takes a lightweight cluster-based approach and exploits the overlap of sparse retrieval results and embedding clusters in a two-stage selection process with an LSTM model to quickly identify relevant clusters while incurring limited extra memory space overhead. CluSD triggers partial dense retrieval and performs cluster-based block disk I/O if needed. This paper evaluates CluSD and compares it with several baselines for searching in-memory and on-disk MS MARCO and BEIR datasets.

cs.IR↗

arXiv2024

Weighted KL-Divergence for Document Ranking Model Refinement

Transformer-based retrieval and reranking models for text document search are often refined through knowledge distillation together with contrastive learning. A tight distribution matching between the teacher and student models can be hard as over-calibration may degrade training effectiveness when a teacher does not perform well. This paper contrastively reweights KL divergence terms to prioritize the alignment between a student and a teacher model for proper separation of positive and negative documents. This paper analyzes and evaluates the proposed loss function on the MS MARCO and BEIR datasets to demonstrate its effectiveness in improving the relevance of tested student models.

cs.IR↗

arXiv2024

Approximate Cluster-Based Sparse Document Retrieval with Segmented Maximum Term Weights

This paper revisits cluster-based retrieval that partitions the inverted index into multiple groups and skips the index partially at cluster and document levels during online inference using a learned sparse representation. It proposes an approximate search scheme with two parameters to control the rank-safeness competitiveness of pruning with segmented maximum term weights within each cluster. Cluster-level maximum weight segmentation allows an improvement in the rank score bound estimation and threshold-based pruning to be approximately adaptive to bound estimation tightness, resulting in better relevance and efficiency. The experiments with MS MARCO passage ranking and BEIR datasets demonstrate the usefulness of the proposed scheme with a comparison to the baselines. This paper presents the design of this approximate retrieval scheme with rank-safeness analysis, compares clustering and segmentation options, and reports evaluation results.

cs.IR↗

arXiv2023

Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval

Learned sparse document representations using a transformer-based neural model has been found to be attractive in both relevance effectiveness and time efficiency. This paper describes a representation sparsification scheme based on hard and soft thresholding with an inverted index approximation for faster SPLADE-based document retrieval. It provides analytical and experimental results on the impact of this learnable hybrid thresholding scheme.

cs.IR↗

arXiv2023

Optimizing Guided Traversal for Fast Learned Sparse Retrieval

Recent studies show that BM25-driven dynamic index skipping can greatly accelerate MaxScore-based document retrieval based on the learned sparse representation derived by DeepImpact. This paper investigates the effectiveness of such a traversal guidance strategy during top k retrieval when using other models such as SPLADE and uniCOIL, and finds that unconstrained BM25-driven skipping could have a visible relevance degradation when the BM25 model is not well aligned with a learned weight model or when retrieval depth k is small. This paper generalizes the previous work and optimizes the BM25 guided index traversal with a two-level pruning control scheme and model alignment for fast retrieval using a sparse representation. Although there can be a cost of increased latency, the proposed scheme is much faster than the original MaxScore method without BM25 guidance while retaining the relevance effectiveness. This paper analyzes the competitiveness of this two-level pruning scheme, and evaluates its tradeoff in ranking relevance and time efficiency when searching several test datasets.

cs.IR↗

arXiv2022

Dual Skipping Guidance for Document Retrieval with Learned Sparse Representations

This paper proposes a dual skipping guidance scheme with hybrid scoring to accelerate document retrieval that uses learned sparse representations while still delivering a good relevance. This scheme uses both lexical BM25 and learned neural term weights to bound and compose the rank score of a candidate document separately for skipping and final ranking, and maintains two top-k thresholds during inverted index traversal. This paper evaluates time efficiency and ranking relevance of the proposed scheme in searching MS MARCO TREC datasets.

cs.IR↗

arXiv2022

Compact Token Representations with Contextual Quantization for Efficient Document Re-ranking

Transformer based re-ranking models can achieve high search relevance through context-aware soft matching of query tokens with document tokens. To alleviate runtime complexity of such inference, previous work has adopted a late interaction architecture with pre-computed contextual token representations at the cost of a large online storage. This paper proposes contextual quantization of token embeddings by decoupling document-specific and document-independent ranking contributions during codebook-based compression. This allows effective online decompression and embedding composition for better search relevance. This paper presents an evaluation of the above compact token representation model in terms of relevance and space efficiency.

cs.IR↗

arXiv2022

Composite Re-Ranking for Efficient Document Search with BERT

Although considerable efforts have been devoted to transformer-based ranking models for document search, the relevance-efficiency tradeoff remains a critical problem for ad-hoc ranking. To overcome this challenge, this paper presents BECR (BERT-based Composite Re-Ranking), a composite re-ranking scheme that combines deep contextual token interactions and traditional lexical term-matching features. In particular, BECR exploits a token encoding mechanism to decompose the query representations into pre-computable uni-grams and skip-n-grams. By applying token encoding on top of a dual-encoder architecture, BECR separates the attentions between a query and a document while capturing the contextual semantics of a query. In contrast to previous approaches, this framework does not perform expensive BERT computations during online inference. Thus, it is significantly faster, yet still able to achieve high competitiveness in ad-hoc ranking relevance. Finally, an in-depth comparison between BECR and other start-of-the-art neural ranking baselines is described using the TREC datasets, thereby further demonstrating the enhanced relevance and efficiency of BECR.

cs.IR↗

arXiv2020

Logical Foundations of Local Gauge Symmetry and Symmetry Breaking

Quantum field theory has successfully generated a number of general conclusions. It seems meaningful to disclose the logical forms of these conclusions. The present paper reports two results. The first result shows the logic of local gauge symmetry and indefinability of mass. The second result shows the logic of Higgs mechanism and definability of mass. The results are obtained by integrating four components, namely, gauge symmetry and Higgs mechanism in quantum field theory, and incompleteness theorem and indefinability theorem in mathematical logic. Godel numbering is the key for arithmetic modeling applied in this paper.

math.LO↗

arXiv2019

Semiparametric Methods for Exposure Misclassification in Propensity Score-Based Time-to-Event Data Analysis

In epidemiology, identifying the effect of exposure variables in relation to a time-to-event outcome is a classical research area of practical importance. Incorporating propensity score in the Cox regression model, as a measure to control for confounding, has certain advantages when outcome is rare. However, in situations involving exposure measured with moderate to substantial error, identifying the exposure effect using propensity score in Cox models remains a challenging yet unresolved problem. In this paper, we propose an estimating equation method to correct for the exposure misclassification-caused bias in the estimation of exposure-outcome associations. We also discuss the asymptotic properties and derive the asymptotic variances of the proposed estimators. We conduct a simulation study to evaluate the performance of the proposed estimators in various settings. As an illustration, we apply our method to correct for the misclassification-caused bias in estimating the association of PM2.5 level with lung cancer mortality using a nationwide prospective cohort, the Nurses' Health Study (NHS). The proposed methodology can be applied using our user-friendly R function published online.

stat.ME↗

arXiv2019

Efficient vaccination strategies for epidemic control using network information

Network-based interventions against epidemic spread are most powerful when the full network structure is known. However, in practice, resource constraints require decisions to be made based on partial network information. We investigated how the accuracy of network data available at individual and village levels affected network-based vaccination effectiveness. We simulated a Susceptible-Infected-Recovered process on empirical social networks from 75 villages. First, we used regression to predict the percentage of individuals ever infected based on village-level network. Second, we simulated vaccinating 10 percent of each of the 75 empirical village networks at baseline, selecting vaccinees through one of five network-based approaches: random individuals; random contacts of random individuals; random high-degree individuals; highest degree individuals; or most central individuals. The first three approaches require only sample data; the latter two require full network data. We also simulated imposing a limit on how many contacts an individual can nominate (Fixed Choice Design, FCD), which reduces the data collection burden but generates only partially observed networks. We found mean and standard deviation of the degree distribution to strongly predict cumulative incidence. In simulations, the Nomination method reduced cumulative incidence by one-sixth compared to Random vaccination; full network methods reduced infection by two-thirds. The High Degree approach had intermediate effectiveness. Surprisingly, FCD truncating individuals' degrees at three was as effective as using complete networks. Using even partial network information to prioritize vaccines at either the village or individual level substantially improved epidemic outcomes. Such approaches may be feasible and effective in outbreak settings, and full ascertainment of network structure may not be required.

stat.AP↗

arXiv2019

Ranking in Genealogy: Search Results Fusion at Ancestry

Genealogy research is the study of family history using available resources such as historical records. Ancestry provides its customers with one of the world's largest online genealogical index with billions of records from a wide range of sources, including vital records such as birth and death certificates, census records, court and probate records among many others. Search at Ancestry aims to return relevant records from various record types, allowing our subscribers to build their family trees, research their family history, and make meaningful discoveries about their ancestors from diverse perspectives. In a modern search engine designed for genealogical study, the appropriate ranking of search results to provide highly relevant information represents a daunting challenge. In particular, the disparity in historical records makes it inherently difficult to score records in an equitable fashion. Herein, we provide an overview of our solutions to overcome such record disparity problems in the Ancestry search engine. Specifically, we introduce customized coordinate ascent (customized CA) to speed up ranking within a specific record type. We then propose stochastic search (SS) that linearly combines ranked results federated across contents from various record types. Furthermore, we propose a novel information retrieval metric, normalized cumulative entropy (NCE), to measure the diversity of results. We demonstrate the effectiveness of these two algorithms in terms of relevance (by NDCG) and diversity (by NCE) if applicable in the offline experiments using real customer data at Ancestry.

cs.IR↗

arXiv2018

Efficient vaccination strategies for epidemic control using network information

Background: Network-based interventions are most powerful against epidemics when the full network structure is known. However, resource constraints often require decisions based on partial network data. We investigated how the effectiveness of network-based vaccination schemes varied based on the accuracy of network data available. Methods: We simulated propagating an SIR process on static empirical social networks from 75 rural Indian villages. First, we used regression analysis to predict the percentage of individuals ever infected (cumulative incidence) based on village-level networks. Second, we simulated vaccinating 10% of each village at baseline, selecting vaccinees using 1 of 5 network-based approaches: at random (Random); random contacts of random individuals (Nomination); random high-degree individuals (High Degree); highest degree (Highest Degree); or most central (Central). The first three approaches require only sample data; the latter two require full network data. We also simulated imposing a limit on how many contacts an individual can nominate (Fixed Choice Design, FCD), which reduces data collection burden but generates only partially observed networks. Results: We found the mean and standard deviation of the degree distribution to strongly predict cumulative incidence. In simulations, Nomination reduced cumulative incidence by one-sixth compared to Random vaccination; full network methods reduced infection by two-thirds. High Degree had intermediate effectiveness. Somewhat surprisingly, FCD truncating individuals' degrees at three was as effective as using complete networks. Conclusions: Using even partial network information to prioritize vaccines at either the village or individual level substantially improved epidemic outcomes. Such approaches may be feasible and effective in outbreak settings, and full ascertainment of network structure may not be required.

q-bio.PE↗