Michael P Brenner | Searcharxiv

arXiv2025

CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning

Scientific problem-solving involves synthesizing information while applying expert knowledge. We introduce CURIE, a scientific long-Context Understanding,Reasoning and Information Extraction benchmark to measure the potential of Large Language Models (LLMs) in scientific problem-solving and assisting scientists in realistic workflows. This benchmark introduces ten challenging tasks with a total of 580 problems and solution pairs curated by experts in six disciplines - materials science, condensed matter physics, quantum computing, geospatial analysis, biodiversity, and proteins - covering both experimental and theoretical work-flows in science. We evaluate a range of closed and open LLMs on tasks in CURIE which requires domain expertise, comprehension of long in-context information,and multi-step reasoning. While Gemini Flash 2.0 and Claude-3 show consistent high comprehension across domains, the popular GPT-4o and command-R+ fail dramatically on protein sequencing tasks. With the best performance at 32% there is much room for improvement for all models. We hope that insights gained from CURIE can guide the future development of LLMs in sciences. Evaluation code and data are in https://github.com/google/curie

cs.CL↗

arXiv2022

Beyond Quality and Quantity: Contact Distribution Encodes Frictional Strength

Classically, the quantity of contact area $A_R$ between two bodies is considered a proxy for the force of friction. However, bond density across the interface - quality of contact - is also relevant, and contemporary debate often centers around the relative importance of these two factors. In this work, we demonstrate that a third factor, often overlooked, plays a significant role in static frictional strength: the distribution of contact. We perform static friction measurements, $μ$, on three pairs of solid blocks while imaging the contact plane. By using linear regression on hundreds of image-$μ$ pairs, we are able to predict future friction measurements with 3 to 7 times better accuracy than existing benchmarks, including total quantity of contact area. Our model has no access to quality of contact, and we therefore conclude that a large portion of the interfacial state is encoded in the spatial distribution of contact, rather than its quality or quantity.

cond-mat.soft↗