arXiv subjects

Vicente Amado Olivo

Publications and source records attributed to Vicente Amado Olivo.

3 recordsLinked to original sources

arXiv2026

Traditional statistical representations outperform generative AI in identifying expert peer reviewers

The exponential growth of scientific submissions has strained the peer review system. Despite the rapidly expanding global pool of researchers, this unprecedented scale has rendered the previous approach of manual expert identification unfeasible. Therefore, institutions have naturally turned to Large Language Models (LLMs) to automate intricate processes like expert reviewer identification. However, the reliability of these new models in accurately identifying domain experts lacks rigorous evaluation. We conduct a comprehensive empirical evaluation of statistical and AI-driven expertise identification methodologies to benchmark their reliability and limitations. Framing expert identification as an information retrieval problem, we utilize the distributed peer review system of a major international astronomical observatory, where proposal authorship serves as our proxy ground truth for domain expertise. Evaluating six retrieval methodologies utilized across observatories and computer science conferences, we demonstrate that traditional statistical representations outperform generative AI. Specifically, Term Frequency-Inverse Document Frequency successfully identified a labeled expert within the top 25 recommendations 79.5% of the time, compared to 51.5% for GPT-4o mini. Our results highlight that distinguishing subfield expertise requires fine-grained vocabulary, which is obscured by the semantic smoothing in generative methods. By establishing a rigorous evaluation framework for automated peer review, we demonstrate that transparent and reproducible statistical representations still outperform computationally expensive LLMs in specialized scientific tasks.

cs.IR↗

arXiv2025

Practical Author Name Disambiguation under Metadata Constraints: A Contrastive Learning Approach for Astronomy Literature

The ability to distinctly and properly collate an individual researcher's publications is crucial for ensuring appropriate recognition, guiding the allocation of research funding and informing hiring decisions. However, accurately grouping and linking a researcher's entire body of work with their individual identity is challenging because of widespread name ambiguity across the growing literature. Algorithmic author name disambiguation provides a scalable approach to disambiguating author identities, yet existing methods have limitations. Many modern author name disambiguation methods rely on comprehensive metadata features such as venue or affiliation. Despite advancements in digitally indexing publications, metadata is often unavailable or inconsistent in large digital libraries(e.g. NASA/ADS). We introduce the Neural Author Name Disambiguator, a method that disambiguates author identities in large digital libraries despite limited metadata availability. We formulate the disambiguation task as a similarity learning problem by employing a Siamese neural network to disambiguate author names across publications relying solely on widely available publication metadata-author names, titles and abstracts. We construct the Large-Scale Physics ORCiD Linked dataset to evaluate the Neural Author Name Disambiguator by cross-matching NASA/ADS publications ORCiD. By leveraging foundation models to embed metadata into features, our model achieves up to 94% accuracy in pairwise disambiguation and over 95% F1 in clustering publications into their researcher identities. We release the testing dataset as a benchmark for physics and astronomy, providing realistic evaluation conditions for future disambiguation methods. The Neural Author Name Disambiguator algorithm demonstrates effective disambiguation with minimal metadata, offering a scalable solution for name ambiguity in large digital libraries.

astro-ph.IM↗

arXiv2024

Identifying Telescope Usage in Astrophysics Publications: A Machine Learning Framework for Institutional Research Management at Observatories

Large scientific institutions, such as the Space Telescope Science Institute, track the usage of their facilities to understand the needs of the research community. Astrophysicists incorporate facility usage data into their scientific publications, embedding this information in plain-text. Traditional automatic search queries prove unreliable for accurate tracking due to the misidentification of facility names in plain-text. As automatic search queries fail, researchers are required to manually classify publications for facility usage, which consumes valuable research time. In this work, we introduce a machine learning classification framework for the automatic identification of facility usage of observation sections in astrophysics publications. Our framework identifies sentences containing telescope mission keywords (e.g., Kepler and TESS) in each publication. Subsequently, the identified sentences are transformed using Term Frequency-Inverse Document Frequency and classified with a Support Vector Machine. The classification framework leverages the context surrounding the identified telescope mission keywords to provide relevant information to the classifier. The framework successfully classifies usage of MAST hosted missions with a 92.9% accuracy. Furthermore, our framework demonstrates robustness when compared to other approaches, considering common metrics and computational complexity. The framework's interpretability makes it adaptable for use across observatories and other scientific facilities worldwide.

astro-ph.IM↗