arXiv subjects

Randy Sawaya

Publications and source records attributed to Randy Sawaya.

3 recordsLinked to original sources

arXiv2025

How important is Recall for Measuring Retrieval Quality?

In realistic retrieval settings with large and evolving knowledge bases, the total number of documents relevant to a query is typically unknown, and recall cannot be computed. In this paper, we evaluate several established strategies for handling this limitation by measuring the correlation between retrieval quality metrics and LLM-based judgments of response quality, where responses are generated from the retrieved documents. We conduct experiments across multiple datasets with a relatively low number of relevant documents (2-15). We also introduce a simple retrieval quality measure that performs well without requiring knowledge of the total number of relevant documents.

cs.CL↗

arXiv2024

Preserving Multilingual Quality While Tuning Query Encoder on English Only

A query encoder of a dual passage retrieval system can be tuned for specific types of queries or domains, while the precomputed and stored documents representations are kept intact. Switching from one query encoder to another when needed is easily feasible, unlike overhauling the embeddings of a whole knowledge base. In this work we raise a question: Can the generic, original qualities of the encoder be preserved or at least left not too degraded when it is tuned on a narrow domain? We conducted experiments on a high quality multilingual embedding model: Tuning it on a single English-only dataset, we observe that the tuning not only preserves the multilingual qualities, but even improves them. The embedding qualities on distinctly different data are also improved or at least preserved. Drawing on our observations, we suggest a more general hypothesis: Tuning with intentionally low learning rate can preserve or improve a system's properties acquired in training, but not specifically targeted by tuning. We call this adiabatic tuning and provide tentative explanations.

cs.CL↗

arXiv2020

Ground-state properties of the hydrogen chain: insulator-to-metal transition, dimerization, and magnetic phases

Accurate and predictive computations of the quantum-mechanical behavior of many interacting electrons in realistic atomic environments are critical for the theoretical design of materials with desired properties, and require solving the grand-challenge problem of the many-electron Schrodinger equation. An infinite chain of equispaced hydrogen atoms is perhaps the simplest realistic model for a bulk material, embodying several central themes of modern condensed matter physics and chemistry, while retaining a connection to the paradigmatic Hubbard model. Here we report a combined application of cutting-edge computational methods to determine the properties of the hydrogen chain in its quantum-mechanical ground state. Varying the separation between the nuclei leads to a rich phase diagram, including a Mott phase with quasi long-range antiferromagnetic order, electron density dimerization with power-law correlations, an insulator-to-metal transition and an intricate set of intertwined magnetic orders.

cond-mat.str-el↗