Searcharxiv⌕ Search

arXiv subjects

Travis J. Wheeler

Publications and source records attributed to Travis J. Wheeler.

6 recordsLinked to original sources
arXiv2026

Expert-Aware Refusal Steering

Safety alignment in instruction-tuned large language models (LLMs) depends on a model's ability to reliably refuse to respond to harmful or disallowed requests. Recent work has shown that a steering vector can be applied to a dense LLM during inference to effectively suppress refusal behavior, inducing response to harmful requests. We extend this refusal steering method to three open-source Mixture-of-Experts (MoE) LLMs and find that steering performance is uninhibited by the complex routing patterns inherent to the MoE architecture. We then propose two expert-aware refusal steering methods that leverage refusal-specific expert routing patterns and expert-specific steering directions to suppress normal refusal behavior. We find that refusal behavior can be effectively steered based on the output of a single expert. Our results show that refusal signals captured by steering methods differ from expert routing behavior, suggesting a substantial role for attention in MoE refusal behavior.

cs.CL↗
arXiv2025

Contributions of the Petabyte Scale Sequence Search Codeathon toward efforts to scale sequence-based searches on SRA

The volume of biological data being generated by the scientific community is growing exponentially, reflecting technological advances and research activities. The National Institutes of Health's (NIH) Sequence Read Archive (SRA), which is maintained by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), is a rapidly growing public database that researchers use to drive scientific discovery across all domains of life. This increase in available data has great promise for pushing scientific discovery but also introduces new challenges that scientific communities need to address. As genomic datasets have grown in scale and diversity, a parade of new methods and associated software have been developed to address the challenges posed by this growth. These methodological advances are vital for maximally leveraging the power of next-generation sequencing (NGS) technologies. With the goal of laying a foundation for evaluation of methods for petabyte-scale sequence search, the Department of Energy (DOE) Office of Biological and Environmental Research (BER), the NIH Office of Data Science Strategy (ODSS), and NCBI held a virtual codeathon 'Petabyte Scale Sequence Search: Metagenomics Benchmarking Codeathon' on September 27 - Oct 1 2021, to evaluate emerging solutions in petabyte scale sequence search. The codeathon attracted experts from national laboratories, research institutions, and universities across the world to (a) develop benchmarking approaches to address challenges in conducting large-scale analyses of metagenomic data (which comprises approximately 20% of SRA), (b) identify potential applications that benefit from SRA-wide searches and the tools required to execute the search, and (c) produce community resources i.e. a public facing repository with information to rebuild and reproduce the problems addressed by each team challenge.

q-bio.OT↗
arXiv2025

Describing the Persistence Landscape for Introducing Microbes into Complex Communities

The introduction of non-native organisms into complex microbiome communities holds enormous potential to benefit society. However, microbiome engineering faces several challenges including successful establishment of the organism into the community, its persistence in the microbiome to serve a specified purpose, and constraint of the organism and its activity to the intended environment. A theoretical framework is needed to represent the complex interactions that drive these dynamics. Building on the concept of the community functional landscape, we define the persistence landscape as the metabolic, genetic, and broader functional composition and ecological context of the target microbiome that can be used to predict the environmental fitness of an introduced organism. Here, we discuss critical aspects of persistence landscapes that impact interactions between an introduced organism and the target microbiome, including the communitys genetic and metabolic complementation potential, cellular defense strategies, spatial and temporal dynamics, and the introduced organisms ability to compete for resources to survive. Finally, we highlight important knowledge gaps in the fields of microbial ecology and microbiome engineering that limit characterization and engineering of persistence landscapes. As a model for understanding microbiome structure and interaction in the context of microbiome engineering, the persistence landscape model should enable development of novel containment approaches while improving controlled colonization of a complex microbiome community to address pressing challenges in human health, agronomy, and biomanufacturing.

q-bio.PE↗
arXiv2024

Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance

It is generally thought that transformer-based large language models benefit from pre-training by learning generic linguistic knowledge that can be focused on a specific task during fine-tuning. However, we propose that much of the benefit from pre-training may be captured by geometric characteristics of the latent space representations, divorced from any specific linguistic knowledge. In this work we explore the relationship between GLUE benchmarking task performance and a variety of measures applied to the latent space resulting from BERT-type contextual language models. We find that there is a strong linear relationship between a measure of quantized cell density and average GLUE performance and that these measures may be predictive of otherwise surprising GLUE performance for several non-standard BERT-type models from the literature. These results may be suggestive of a strategy for decreasing pre-training requirements, wherein model initialization can be informed by the geometric characteristics of the model's latent space.

cs.CL↗
arXiv2023

Reliable Measures of Spread in High Dimensional Latent Spaces

Understanding geometric properties of natural language processing models' latent spaces allows the manipulation of these properties for improved performance on downstream tasks. One such property is the amount of data spread in a model's latent space, or how fully the available latent space is being used. In this work, we define data spread and demonstrate that the commonly used measures of data spread, Average Cosine Similarity and a partition function min/max ratio I(V), do not provide reliable metrics to compare the use of latent space across models. We propose and examine eight alternative measures of data spread, all but one of which improve over these current metrics when applied to seven synthetic data distributions. Of our proposed measures, we recommend one principal component-based measure and one entropy-based measure that provide reliable, relative measures of spread and can be used to compare models of different sizes and dimensionalities.

cs.LG↗