Anna L. Beukenhorst | Searcharxiv

arXiv2026

Mechanistic Interpretability of Antibody Language Models Using SAEs

Sparse autoencoders (SAEs) are a mechanistic interpretability technique that have been used to provide insight into learned concepts within large protein language models. Here, we employ TopK and Ordered SAEs to investigate autoregressive antibody language models, and steer their generation. We show that TopK SAEs can reveal biologically meaningful latent features, but high feature-concept correlation does not guarantee causal control over generation. In contrast, Ordered SAEs impose a hierarchical structure that reliably identifies steerable features, but at the expense of more complex and less interpretable activation patterns. These findings advance the mechanistic interpretability of domain-specific protein language models and suggest that, while TopK SAEs suffice for mapping latent features to concepts, Ordered SAEs are preferable when precise generative steering is required.

cs.LG↗

arXiv2024

How Being Inside or Outside of Buildings Affects the Causal Relationship Between Weather and Pain Among People Living with Chronic Pain

Although many people believe their pain fluctuates with weather conditions, both weather and pain may be associated with time spent outside. For example, pleasant weather may mean that people spend more time outside doing physical activity and exposed to the weather, leading to more (or less) pain, and poor weather or severe pain may keep people inside, sedentary, and not exposed to the weather. We conducted a smartphone study where participants with chronic pain reported daily pain severity, as well as time spent outside. We address the relationship between four weather variables (temperature, dewpoint temperature, pressure, and wind speed) and pain by proposing a three-step approach to untangle their effects: (i) propose a set of plausible directed acyclic graphs (also known as DAGs) that account for potential roles of time spent outside (e.g., collider, effect modifier, mediator), (ii) analyze the compatibility of the observed data with the assumed model, and (iii) identify the most plausible model by combining evidence from the observed data and domain-specific knowledge. We found that the data do not support time spent outside as a collider or mediator of the relationship between weather variables and pain. On the other hand, time spent outside modifies the effect between temperature and pain, as well as wind speed and pain, with the effect being absent on days that participants spent inside and present if they spent some or all of the day outside. Our results show the utility of using directed acyclic graphs for studying causal inference.

q-bio.OT↗