arXiv subjects

Pedro Medeiros

Publications and source records attributed to Pedro Medeiros.

4 recordsLinked to original sources

arXiv2026

BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation

As large language models become increasingly capable, concerns about their potential to assist with biological misuse continue to grow. Prioritization of safety differs across the model ecosystem, with some models freely providing high-risk information that could be misused, and others refusing benign scientific content, potentially hindering legitimate research. Both failures stem from a lack of targeted mitigation to distinguish the most dangerous information from broader scientific content. To address this, we introduce BioTIER (Biological Targeted Information for Exclusion and Refusal), a benchmark designed to enable more targeted biological risk mitigation. BioTIER organizes biological content into three risk sets: Catastrophe Avoidance (CA), Biomedical DURC (BD) and Related Biology (RB). These sets represent a spectrum from extremely narrow high-risk topics to a broad range of benign and beneficial biological knowledge. The benchmark consists of 542 expert-curated prompts with rich associated metadata to support differentiated access policies. We release BioTIER to aid in isolating and gating the tiny fraction of information that could engender catastrophic risk from misuse, while ensuring access to the vast wealth of knowledge that is essential for advancing biological science.

cs.CY↗

arXiv2026

LLM Novice Uplift on Dual-Use, In Silico Biology Tasks

Large language models (LLMs) perform increasingly well on biology benchmarks, but it remains unclear whether they uplift novice users -- i.e., enable humans to perform better than with internet-only resources. This uncertainty is central to understanding both scientific acceleration and dual-use risk. We conducted a multi-model, multi-benchmark human uplift study comparing novices with LLM access versus internet-only access across eight biosecurity-relevant task sets. Participants worked on complex problems with ample time (up to 13 hours for the most involved tasks). We found that LLM access provided substantial uplift: novices with LLMs were 4.16 times more accurate than controls (95% CI [2.63, 6.87]). On four benchmarks with available expert baselines (internet-only), novices with LLMs outperformed experts on three of them. Perhaps surprisingly, standalone LLMs often exceeded LLM-assisted novices, indicating that users were not eliciting the strongest available contributions from the LLMs. Most participants (89.6%) reported little difficulty obtaining dual-use-relevant information despite safeguards. Overall, LLMs substantially uplift novices on biological tasks previously reserved for trained practitioners, underscoring the need for sustained, interactive uplift evaluations alongside traditional benchmarks.

cs.AI↗

arXiv2025

Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark

We present the Virology Capabilities Test (VCT), a large language model (LLM) benchmark that measures the capability to troubleshoot complex virology laboratory protocols. Constructed from the inputs of dozens of PhD-level expert virologists, VCT consists of $322$ multimodal questions covering fundamental, tacit, and visual knowledge that is essential for practical work in virology laboratories. VCT is difficult: expert virologists with access to the internet score an average of $22.1\%$ on questions specifically in their sub-areas of expertise. However, the most performant LLM, OpenAI's o3, reaches $43.8\%$ accuracy, outperforming $94\%$ of expert virologists even within their sub-areas of specialization. The ability to provide expert-level virology troubleshooting is inherently dual-use: it is useful for beneficial research, but it can also be misused. Therefore, the fact that publicly available models outperform virologists on VCT raises pressing governance considerations. We propose that the capability of LLMs to provide expert-level troubleshooting of dual-use virology work should be integrated into existing frameworks for handling dual-use technologies in the life sciences.

cs.CY↗

arXiv2023

Ten Quick Tips for Harnessing the Power of ChatGPT/GPT-4 in Computational Biology

The rise of advanced chatbots, such as ChatGPT, has sparked curiosity in the scientific community. ChatGPT is a general-purpose chatbot powered by large language models (LLMs) GPT-3.5 and GPT-4, with the potential to impact numerous fields, including computational biology. In this article, we offer ten tips based on our experience with ChatGPT to assist computational biologists in optimizing their workflows. We have collected relevant prompts and reviewed the nascent literature in the field, compiling tips we project to remain pertinent for future ChatGPT and LLM iterations, ranging from code refactoring to scientific writing to prompt engineering. We hope our work will help bioinformaticians to complement their workflows while staying aware of the various implications of using this technology. Additionally, to track new and creative applications for bioinformatics tools such as ChatGPT, we have established a GitHub repository at https://github.com/csbl-br/awesome-compbio-chatgpt. Our belief is that ethical adherence to ChatGPT and other LLMs will increase the efficiency of computational biologists, ultimately advancing the pace of scientific discovery in the life sciences.

q-bio.OT↗