arXiv subjects

Matt Thomson

Publications and source records attributed to Matt Thomson.

At least 19 recordsLinked to original sources

arXiv2026

What Should a Large Language Model See? Physical Invariants as a Data Representation for PDE Discovery

Understanding how molecular interactions govern macroscopic behaviour is a central challenge in molecular sciences. However, conventional theory building cannot keep pace with the vast datasets modern experimentation routinely produces. Large language models offer a promising route to automating theory construction, but a spatiotemporal field cannot be directly placed in a prompt. Existing models generally learn about the data only through a score measuring how well each proposal fits it. Here we introduce data interpretation, a stage that measures the field into the quantities a theorist would consult and supplies them to the model as a direct input. On a benchmark of simulated fields, interpretation nearly triples the accuracy of recovered equations relative to showing the raw data, at negligible computational cost and without any training. By allowing a language model to read field data as a theorist does, data interpretation offers a practical route to automated field theory construction that can coevolve with experimentation.

cs.LG↗

arXiv2025

A Sensing Whole Brain Zebrafish Foundation Model for Neuron Dynamics and Behavior

Neural dynamics underlie behaviors from memory to sleep, yet identifying mechanisms for higher-order phenomena (e.g., social interaction) is experimentally challenging. Existing whole-brain models often fail to scale to single-neuron resolution, omit behavioral readouts, or rely on PCA/conv pipelines that miss long-range, non-linear interactions. We introduce a sparse-attention whole-brain foundation model (SBM) for larval zebrafish that forecasts neuron spike probabilities conditioned on sensory stimuli and links brain state to behavior. SBM factorizes attention across neurons and along time, enabling whole-brain scale and interpretability. On a held-out subject, it achieves mean absolute error <0.02 with calibrated predictions and stable autoregressive rollouts. Coupled to a permutation-invariant behavior head, SBM enables gradient-based synthesis of neural patterns that elicit target behaviors. This framework supports rapid, behavior-grounded exploration of complex neural phenomena.

q-bio.NC↗

arXiv2025

Boundaries Program Deformation in Isolated Active Networks

Cellular structures must organize themselves within strict physical constraints, operating with finite resources and well-defined boundaries. Classical systems demonstrate only passive responses to boundaries, from surface energy minimization in soap films to strain distributions in elastic networks. Active matter fundamentally alters this paradigm - internally generated stresses create a bidirectional coupling between boundary geometry and mass conservation that enables dynamic control over network organization. Here we demonstrate boundary geometry actively directs network deformation in reconstituted microtubule-kinesin systems, revealing a programmable regime of shape transformation through controlled boundary manipulation. A coarse-grained theoretical framework reveals how boundary geometry couples to internal stress fields via mass conservation, producing distinct dynamical modes that enable engineered deformations. The emergence of shape-preserving and shape-changing regimes, predicted by theory and confirmed through experiments, establishes boundary geometry as a fundamental control parameter for active materials. The control principle based on boundaries advances both the understanding of biological organization and enables design of synthetic active matter devices with programmable deformation.

cond-mat.soft↗

arXiv2025

Fail Fast, or Ask: Mitigating the Deficiencies of Reasoning LLMs with Human-in-the-Loop Systems Engineering

State-of-the-art reasoning LLMs are powerful problem solvers, but they still occasionally make mistakes. However, adopting AI models in risk-sensitive domains often requires error rates near 0%. To address this gap, we propose collaboration between a reasoning model and a human expert who resolves queries the model cannot confidently answer. We find that quantifying the uncertainty of a reasoning model through the length of its reasoning trace yields an effective basis for deferral to a human, e.g., cutting the error rate of Qwen3 235B-A22B on difficult MATH problems from 3% to less than 1% when deferring 7.5% of queries. However, the high latency of reasoning models still makes them challenging to deploy on use cases with high query volume. To address this challenge, we explore fronting a reasoning model with a large non-reasoning model. We call this modified human-in-the-loop system "Fail Fast, or Ask", since the non-reasoning model may defer difficult queries to the human expert directly ("failing fast"), without incurring the reasoning model's higher latency. We show that this approach yields around 40% latency reduction and about 50% cost savings for DeepSeek R1 while maintaining 90+% area under the accuracy-rejection curve. However, we observe that latency savings are lower than expected because of "latency drag", the phenomenon that processing easier queries with a non-reasoning model pushes the reasoning model's latency distribution towards longer latencies. Broadly, our results suggest that the deficiencies of state-of-the-art reasoning models -- nontrivial error rates and high latency -- can be substantially mitigated through black-box systems engineering, without requiring access to LLM internals.

cs.AI↗

arXiv2025

Economic Evaluation of LLMs

Practitioners often navigate LLM performance trade-offs by plotting Pareto frontiers of optimal accuracy-cost trade-offs. However, this approach offers no way to compare between LLMs with distinct strengths and weaknesses: for example, a cheap, error-prone model vs a pricey but accurate one. To address this gap, we propose economic evaluation of LLMs. Our framework quantifies the performance trade-off of an LLM as a single number based on the economic constraints of a concrete use case, all expressed in dollars: the cost of making a mistake, the cost of incremental latency, and the cost of abstaining from a query. We apply our economic evaluation framework to compare the performance of reasoning and non-reasoning models on difficult questions from the MATH benchmark, discovering that reasoning models offer better accuracy-cost tradeoffs as soon as the economic cost of a mistake exceeds \$0.01. In addition, we find that single large LLMs often outperform cascades when the cost of making a mistake is as low as \$0.1. Overall, our findings suggest that when automating meaningful human tasks with AI models, practitioners should typically use the most powerful available model, rather than attempt to minimize AI deployment costs, since deployment costs are likely dwarfed by the economic impact of AI errors.

cs.AI↗

arXiv2025

Cost-Saving LLM Cascades with Early Abstention

LLM cascades deploy small LLMs to answer most queries, limiting the use of large and expensive LLMs to difficult queries. This approach can significantly reduce costs without impacting performance. However, risk-sensitive domains such as finance or medicine place an additional premium on avoiding model errors. Since even the most expensive models are susceptible to making mistakes, applications in these domains benefit from allowing LLM systems to completely abstain from answering difficult queries. Introducing abstention poses a design question for LLM cascades: should abstention only be allowed at the final model or also at earlier models? Since the error patterns of small and large models are correlated, allowing earlier models to abstain may reduce inference costs and latency by anticipating abstention decisions by expensive and slow models, thus avoiding the need to run these models. We investigate the benefits of such "early abstention" in LLM cascades and find that it reduces overall test loss by 2.2% on average across six benchmarks (GSM8K, MedMCQA, MMLU, TriviaQA, TruthfulQA, and XSum). These gains result from a more effective use of abstention, trading a 4.1% average increase in the overall abstention rate for a 13.0% reduction in cost and a 5.0% reduction in error rate. Our findings demonstrate the possibility of leveraging correlations between the error patterns of different language models to drive performance improvements for LLM systems with abstention.

cs.AI↗

arXiv2025

Rational Tuning of LLM Cascades via Probabilistic Modeling

Understanding the reliability of large language models (LLMs) has recently garnered significant attention. Given LLMs' propensity to hallucinate, as well as their high sensitivity to prompt design, it is already challenging to predict the performance of an individual LLM. However, the problem becomes more complex for compound LLM systems such as cascades, where in addition to each model's standalone performance, we must understand how the error rates of different models interact. In this paper, we present a probabilistic model for the joint performance distribution of a sequence of LLMs, which enables a framework for rationally tuning the confidence thresholds of a LLM cascade using continuous optimization. Compared to selecting confidence thresholds using Bayesian optimization, our parametric Markov-copula model yields more favorable error-cost trade-offs, improving the area under the error-cost curve by 4.3% on average for cascades with $k\geq 3$ models. In the low-sample regime with $n \leq 30$ training examples, the performance improvement widens to 10.2%, suggesting that our framework's inductive assumptions about the interactions between the error rates of different LLMs enhance sample efficiency. Overall, our Markov-copula model provides a rational basis for tuning LLM cascade performance and points to the potential of probabilistic methods in analyzing systems of LLMs.

cs.LG↗

arXiv2024

Efficiently Deploying LLMs with Controlled Risk

Deploying large language models in production requires simultaneous attention to efficiency and risk control. Prior work has shown the possibility to cut costs while maintaining similar accuracy, but has neglected to focus on risk control. By contrast, here we present hierarchical chains with multi-level abstention (HCMA), which use model-intrinsic uncertainty to delegate queries along the LLM intelligence hierarchy, enabling training-free model switching based solely on black-box API calls. Our framework presents novel trade-offs between efficiency and risk. For example, deploying HCMA on MMLU cuts the error rate of Llama3 405B by 30% when the model is allowed to abstain on 20% of the queries. To calibrate HCMA for optimal performance, our approach uses data-efficient logistic regressions (based on a simple nonlinear feature transformation), which require only 50 or 100 labeled examples to achieve excellent calibration error (ECE), cutting ECE by 50% compared to naive Platt scaling. On free-form generation tasks, we find that chain-of-thought is ineffectual for selective prediction, whereas zero-shot prompting drives error to 0% on TruthfulQA at high abstention rates. As LLMs are increasingly deployed across computing environments with different capabilities (such as mobile, laptop, and cloud), our framework paves the way towards maintaining deployment efficiency while putting in place sharp risk controls.

cs.LG↗

arXiv2024

Prompt Baking

Two primary ways to change LLM behavior are prompting and weight updates (e.g., fine-tuning). Prompting LLMs is simple and effective, specifying the desired changes explicitly in natural language, whereas weight updates provide more expressive and permanent behavior changes, specified implicitly via training on large datasets. We present a technique for "baking" prompts into the weights of an LLM. Prompt Baking converts a prompt $u$ and initial weights $\theta$ to a new set of weights $\theta_u$ such that new "baked" LLM behaves like the original prompted LLM. Mathematically, we minimize the KL divergence between $P_\theta(\cdot | u)$ and $P_{\theta_u}(\cdot)$, where $P$ is the LLM's probability distribution over token sequences. Across all our experiments, we find prompts can be readily baked into weight updates. Baking chain-of-thought prompts improves zero-shot performance on GSM8K, ASDiv, MBPP, ARC-Easy, ARC-Challenge, and CommonsenseQA benchmarks. Baking news headlines directly updates an LLM's knowledge. And baking instructions & personas alleviates "prompt forgetting" over long sequences. Furthermore, stopping baking early creates "half-baked" models, continuously scaling prompt strength. Baked models retain their sensitivity to further prompting and baking, including re-prompting with the baked-in prompt. Surprisingly, the re-prompted models yield further performance gains in instruction following, as well as math reasoning and coding benchmarks. Taking re-prompting and re-baking to the limit yields a form of iterative self-improvement we call Prompt Pursuit, and preliminary results on instruction following exhibit dramatic performance gains. Finally, we discuss implications for AI safety, continuous model updating, enhancing real-time learning capabilities in LLM-based agents, and generating more stable AI personas.

cs.CL↗

arXiv2024

Motor-driven microtubule diffusion in a photobleached dynamical coordinate system

Motor-driven cytoskeletal remodeling in cellular systems can often be accompanied by a diffusive-like effect at local scales, but distinguishing the contributions of the ordering process, such as active contraction of a network, from this active diffusion is difficult to achieve. Using light-dimerizable kinesin motors to spatially control the formation and contraction of a microtubule network, we deliberately photobleach a grid pattern onto the filament network serving as a transient and dynamic coordinate system to observe the deformation and translation of the remaining fluorescent squares of microtubules. We find that the network contracts at a rate set by motor speed but is accompanied by a diffusive-like spread throughout the bulk of the contracting network with effective diffusion constant two orders of magnitude lower than that for a freely-diffusing microtubule. We further find that on micron scales, the diffusive timescale is only a factor of approximately 3 slower than that of advection regardless of conditions, showing that the global contraction and long-time relaxation from this diffusive behavior are both motor-driven but exhibit local competition within the network bulk.

physics.bio-ph↗

arXiv2024

Geometry-Dependent Defect Merging Induces Bifurcated Dynamics in Active Networks

Cytoskeletal networks can repair defects to maintain structural integrity. However, the mechanisms and dynamics of defect merging remain poorly understood. Here we report a geometry-tunable merging mechanism in microtubule-motor networks initiated by active crosslinking. We directly generate defects using a light-controlled microtubule-motor system in O-shaped and V-shaped networks, and observe that the defects can self-close. Combining theory and experiment, we find that the V-shaped networks must overcome internal elastic resistance in order to zip up cracks, giving rise to a bifurcation of dynamics dependent on the initial opening angle of the crack: the crack merges below a critical angle and opens up at larger angles. Simulation of a continuum model reproduces the bifurcation dynamics, revealing the importance of overlapping boundary layers where free motors and microtubules can actively crosslink and thereby merge the defects. We also formulate a simple elastic-rod model that can qualitatively predict the critical angle, which is tunable by the network geometry.

cond-mat.soft↗

arXiv2024

Leveraging Open-Source Large Language Models for encoding Social Determinants of Health using an Intelligent Router

Social Determinants of Health (SDOH), also known as Health-Related Social Needs (HSRN), play a significant role in patient health outcomes. The Centers for Disease Control and Prevention (CDC) introduced a subset of ICD-10 codes called Z-codes to recognize and measure SDOH. However, Z-codes are infrequently coded in a patient's Electronic Health Record (EHR), and instead, in many cases, need to be inferred from clinical notes. Previous research has shown that large language models (LLMs) show promise on extracting unstructured data from EHRs, but it can be difficult to identify a single model that performs best on varied coding tasks. Further, clinical notes contain protected health information posing a challenge for the use of closed-source language models from commercial vendors. The identification of open-source LLMs that can be run within health organizations and exhibit high performance on SDOH tasks is an important issue to solve. Here, we introduce an intelligent routing system for SDOH coding that uses a language model router to direct medical record data to open-source LLMs that demonstrate optimal performance on specific SDOH codes. This intelligent routing system exhibits state of the art performance of 96.4% accuracy averaged across 13 codes, including homelessness and food insecurity, outperforming closed models such as GPT-4o. We leveraged a publicly-available, deidentified dataset of medical record notes to run the router, but we also introduce a synthetic data generation and validation paradigm to increase the scale of training data without needing privacy-protected medical records. Together, we demonstrate an architecture for intelligent routing of inputs to task-optimal language models to achieve high performance across a set of medical coding sub-tasks.

cs.AI↗

arXiv2024

Force Propagation in Active Cytoskeletal Networks

In biological systems, molecular-scale forces and motions are pivotal for enabling processes like motility, shape change, and replication. These forces and motions are organized, amplified, and transmitted across macroscopic scales by active materials such as the cytoskeleton, which drives micron-scale cellular movement and re-organization. Despite the integral role of active materials, understanding how molecular-scale interactions alter macroscopic structure and force propagation remains elusive. This knowledge gap presents challenges to the harnessing and regulation of such dynamics across diverse length scales. Here, we demonstrate how mediating the bundling of microtubules can shift active matter between a global force-transmitting phase and a local force-dissipating phase. A fivefold increase in microtubule effective length results in the transition from local to global phase with a hundredfold increase in velocity autocorrelation. Through theory and simulation, we identify signatures of a percolation-driven transition between the two phases. This provides evidence for how force propagation can be generated when local molecular interactions reach a sufficient length scale. We show that force propagation in the active matter system enables material transport. Consequently, we demonstrate that the global phase is capable of facilitating millimeter-scale human cell transport and manipulation, as well as powering the movement of aqueous droplets. These findings underscore the potential for designing active materials capable of force organization and transmission. Our results lay the foundation for further exploration into the organization and propagation of forces/stresses in biological systems, thereby paving the way for the engineering of active materials in synthetic biology and soft robotics.

cond-mat.soft↗

arXiv2023

Herd: Using multiple, smaller LLMs to match the performances of proprietary, large LLMs via an intelligent composer

Currently, over a thousand LLMs exist that are multi-purpose and are capable of performing real world tasks, including Q&A, text summarization, content generation, etc. However, accessibility, scale and reliability of free models prevents them from being widely deployed in everyday use cases. To address the first two issues of access and scale, organisations such as HuggingFace have created model repositories where users have uploaded model weights and quantized versions of models trained using different paradigms, as well as model cards describing their training process. While some models report performance on commonly used benchmarks, not all do, and interpreting the real world impact of trading off performance on a benchmark for model deployment cost, is unclear. Here, we show that a herd of open source models can match or exceed the performance of proprietary models via an intelligent router. We show that a Herd of open source models is able to match the accuracy of ChatGPT, despite being composed of models that are effectively 2.5x smaller. We show that in cases where GPT is not able to answer the query, Herd is able to identify a model that can, at least 40% of the time.

cs.AI↗

arXiv2023

What's the Magic Word? A Control Theory of LLM Prompting

Prompt engineering is crucial for deploying LLMs but is poorly understood mathematically. We formalize LLM systems as a class of discrete stochastic dynamical systems to explore prompt engineering through the lens of control theory. We offer a mathematical analysis of the limitations on the controllability of self-attention as a function of the singular values of the parameter matrices. We present complementary empirical results on the controllability of a panel of LLMs, including Falcon-7b, Llama-7b, and Falcon-40b. Given initial state $\mathbf x_0$ from Wikitext and prompts of length $k \leq 10$ tokens, we find that the "correct" next token is reachable at least 97% of the time, and that the top 75 most likely next tokens are reachable at least 85% of the time. Intriguingly, short prompt sequences can dramatically alter the likelihood of specific outputs, even making the least likely tokens become the most likely ones. This control-theoretic analysis of LLMs demonstrates the significant and poorly understood role of input sequences in steering output probabilities, offering a foundational perspective for enhancing language model system capabilities.

cs.CL↗

arXiv2023

Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models

The introduction of the transformer architecture and the self-attention mechanism has led to an explosive production of language models trained on specific downstream tasks and data domains. With over 200, 000 models in the Hugging Face ecosystem, users grapple with selecting and optimizing models to suit multifaceted workflows and data domains while addressing computational, security, and recency concerns. There is an urgent need for machine learning frameworks that can eliminate the burden of model selection and customization and unleash the incredible power of the vast emerging model library for end users. Here, we propose a context-aware routing system, Tryage, that leverages a language model router for optimal selection of expert models from a model library based on analysis of individual input prompts. Inspired by the thalamic router in the brain, Tryage employs a perceptive router to predict down-stream model performance on prompts and, then, makes a routing decision using an objective function that integrates performance predictions with user goals and constraints that are incorporated through flags (e.g., model size, model recency). Tryage allows users to explore a Pareto front and automatically trade-off between task accuracy and secondary goals including minimization of model size, recency, security, verbosity, and readability. Across heterogeneous data sets that include code, text, clinical data, and patents, the Tryage framework surpasses Gorilla and GPT3.5 turbo in dynamic model selection identifying the optimal model with an accuracy of 50.9% , compared to 23.6% by GPT 3.5 Turbo and 10.8% by Gorilla. Conceptually, Tryage demonstrates how routing models can be applied to program and control the behavior of multi-model LLM systems to maximize efficient use of the expanding and evolving language model ecosystem.

cs.LG↗

arXiv2023

Theoretical Limits of Energy Extraction in Active Fluids

Active materials form a class of far-from-equilibrium systems that are driven internally and exhibit self-organization which can be harnessed to perform mechanical work. Inspired by experiments on synthetic active networks we examine limits of work extraction from an active viscoelastic medium by analyzing the transport of a particle. The active viscoelastic material possesses an equilibrium density where the active and passive forces are balanced out. In one dimension, a gliding activation front (AF) that converts a passive to an active medium, provides active energy at a constant rate, which is injected into the system at one end and propagates to the other. We demonstrate that there exists a maximum velocity of the AF, above which the activated region fails to deliver the transport power. We hypothesize, and intuitively argue based on the limit cases, that the feasibility and the velocity of transport can be interpreted in terms of the velocity of an equilibration Domain Wall of the field, which is set by two parameters: a measure of activity, and the viscoelastic timescale. The phase diagram comprises Transport and No-Transport sectors, namely for any pair of the two parameters, there exists a threshold velocity of the AF above which the particle transport becomes impossible. Constructing the phase diagram we find that there are regions of the phase diagram for which the threshold velocity of the AF diverges. Larger viscoelastic timescale makes the transport region more accessible, and increases the transport velocity therein. Also, we find that increasing the velocity of AF results in larger extracted power but smaller transport coefficient; the ratio of the transport velocity and that of the AF. Our model provides a framework for understanding the energetics of transport phenomena in biology, and designing efficient mechanisms of transport in synthetic active materials.

cond-mat.soft↗

arXiv2022

Spatiotemporal patterning of extensile active stresses in microtubule-based active fluids

Active stresses, which are collectively generated by the motion of energy-consuming rod-like constituents, generate chaotic autonomous flows. Controlling active stresses in space and time is an essential prerequisite for controlling the intrinsically chaotic dynamics of extensile active fluids. We design single-headed kinesin molecular motors that exhibit optically enhanced clustering, and thus enable precise and repeatable spatial and temporal control of extensile active stresses. Such motors enable rapid, reversible switching between flowing and quiescent states. In turn, spatio-temporal patterning of the active stress controls the evolution of the ubiquitous bend-instability of extensile active fluids and determines its critical length dependence. Combining optically controlled clusters with conventional kinesin motors enables one-time switching from contractile to extensile active stresses. These results open a path towards real-time control of the autonomous flows generated by active fluids.

cond-mat.soft↗