Weakly Supervised Adaptation of Scientific Embeddings for Exploratory Literature Mapping

Abstract

Scientific document embeddings such as SPECTER2 are optimized for retrieval over large, heterogeneous corpora. On a small specialized corpus their similarity distributions can become highly compressed, which limits their usefulness for exploratory literature mapping. We study whether weakly supervised in-domain adaptation, using only metadata already available in open citation databases, changes this geometry. On ArtCon, a curated corpus of 704 publications on artificial and machine consciousness, we adapt the SPECTER2 proximity model with a lightweight LoRA adapter trained on positive pairs derived from bibliographic coupling in OpenAlex, using no manually annotated similarity labels. We compare the adapted embeddings with lexical (TF-IDF), citation-based (bibliographic coupling) and generic semantic (SPECTER2) representations under thresholded, fixed-density and symmetric k-nearest-neighbour graph constructions. Adaptation substantially decompresses the similarity distribution (mean pairwise cosine similarity 0.9104 to 0.8663, standard deviation 0.0259 to 0.0594) and yields graphs with markedly higher Louvain modularity and better connectivity than the base encoder under matched edge and neighbourhood budgets. The effect is stable across all three graph constructions and is therefore not an artefact of a single sparsification choice. We report this as a case study on one corpus, and we are explicit about what it does not establish: the clustering indices used here are computed against community labels derived from each representation's own similarity graph, so they measure internal geometric self-consistency rather than agreement with an external topical ground truth, and adaptation and evaluation use the same documents. Evaluation against independent labels, on held-out documents, and against geometric controls is ongoing work.