arXiv subjects

Nicholas Marshall

Publications and source records attributed to Nicholas Marshall.

6 recordsLinked to original sources

arXiv2025

First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations

Large language models (LLMs) and medical AI tools are routinely used by physicians and patients for medical advice, yet their clinical safety profiles remain poorly characterized. We present NOHARM (Numerous Options Harm Assessment for Risk in Medicine), a 1,100-task benchmark of primary care-to-specialist consultation cases to measure the frequency and severity of potentially harmful errors from LLM-generated medical consultation recommendations. NOHARM covers 10 specialties, with 12,747 expert annotations for 4,249 clinical management options. Across 20 notable LLMs and 4 widely used retrieval-augmented generation (RAG) clinical AI tools, direct application of recommendations carried potential for severe harm in up to 24.6% of cases, with errors of omission accounting for more than 80% of severe errors. Harm potential was not uniform across systems, with clinical AI tools outperforming generalist LLMs, and multi-agent AI teaming further improving performance in generalist models. In a randomized study of 101 U.S.-licensed generalist physicians, AI assistance improved physician performance compared to conventional resources. However, AI-assisted physicians frequently omitted valuable AI-generated recommendations and still scored lower than many AI systems alone. Had those recommendations been incorporated, combined human-AI responses would have outperformed both the human and AI system as used, suggesting complementary strengths and unrealized potential in human-AI teaming. Collectively, these results show that despite strong performance on medical knowledge benchmarks, widely used AI tools can produce medical consultation advice with the potential for severe harm, and highlight the need for explicit measurement of clinical safety. The benchmark and leaderboard are publicly available to support ongoing evaluation and improvement of AI systems used for clinical care.

cs.CY↗

arXiv2025

G-Net: A Provably Easy Construction of High-Accuracy Random Binary Neural Networks

We propose a novel randomized algorithm for constructing binary neural networks with tunable accuracy. This approach is motivated by hyperdimensional computing (HDC), which is a brain-inspired paradigm that leverages high-dimensional vector representations, offering efficient hardware implementation and robustness to model corruptions. Unlike traditional low-precision methods that use quantization, we consider binary embeddings of data as points in the hypercube equipped with the Hamming distance. We propose a novel family of floating-point neural networks, G-Nets, which are general enough to mimic standard network layers. Each floating-point G-Net has a randomized binary embedding, an embedded hyperdimensional (EHD) G-Net, that retains the accuracy of its floating-point counterparts, with theoretical guarantees, due to the concentration of measure. Empirically, our binary models match convolutional neural network accuracies and outperform prior HDC models by large margins, for example, we achieve almost 30\% higher accuracy on CIFAR-10 compared to prior HDC models. G-Nets are a theoretically justified bridge between neural networks and randomized binary neural networks, opening a new direction for constructing robust binary/quantized deep learning models. Our implementation is available at https://github.com/GNet2025/GNet.

cs.LG↗

arXiv2025

Momentum accelerated power iterations and the restarted Lanczos method

In this paper we compare two methods for finding extremal eigenvalues and eigenvectors: the restarted Lanczos method and momentum accelerated power iterations. The convergence of both methods is based on ratios of Chebyshev polynomials evaluated at subdominant and dominant eigenvalues; however, the convergence is not the same. Here we compare the theoretical convergence properties of both methods, and determine the relative regimes where each is more efficient. We further introduce a preconditioning technique for the restarted Lanczos method using momentum accelerated power iterations, and demonstrate its effectiveness. The theoretical results are backed up by numerical tests on benchmark problems.

math.NA↗

arXiv2025

A Multi-Phase Analysis of Blood Culture Stewardship: Machine Learning Prediction, Expert Recommendation Assessment, and LLM Automation

Blood cultures are often over ordered without clear justification, straining healthcare resources and contributing to inappropriate antibiotic use pressures worsened by the global shortage. In study of 135483 emergency department (ED) blood culture orders, we developed machine learning (ML) models to predict the risk of bacteremia using structured electronic health record (EHR) data and provider notes via a large language model (LLM). The structured models AUC improved from 0.76 to 0.79 with note embeddings and reached 0.81 with added diagnosis codes. Compared to an expert recommendation framework applied by human reviewers and an LLM-based pipeline, our ML approach offered higher specificity without compromising sensitivity. The recommendation framework achieved sensitivity 86%, specificity 57%, while the LLM maintained high sensitivity (96%) but over classified negatives, reducing specificity (16%). These findings demonstrate that ML models integrating structured and unstructured data can outperform consensus recommendations, enhancing diagnostic stewardship beyond existing standards of care.

cs.LG↗

arXiv2025

Antibiotic Resistance Microbiology Dataset (ARMD): A Resource for Antimicrobial Resistance from EHRs

The Antibiotic Resistance Microbiology Dataset (ARMD) is a de-identified resource derived from electronic health records (EHR) that facilitates research in antimicrobial resistance (AMR). ARMD encompasses big data from adult patients collected from over 15 years at two academic-affiliated hospitals, focusing on microbiological cultures, antibiotic susceptibilities, and associated clinical and demographic features. Key attributes include organism identification, susceptibility patterns for 55 antibiotics, implied susceptibility rules, and de-identified patient information. This dataset supports studies on antimicrobial stewardship, causal inference, and clinical decision-making. ARMD is designed to be reusable and interoperable, promoting collaboration and innovation in combating AMR. This paper describes the dataset's acquisition, structure, and utility while detailing its de-identification process.

q-bio.QM↗

arXiv2016

Extracting Geography from Trade Data

Understanding international trade is a fundamental problem in economics -- one standard approach is via what is commonly called the "gravity equation", which predicts the total amount of trade $F_ij$ between two countries $i$ and $j$ as $$ F_{ij} = G \frac{M_i M_j}{D_{ij}},$$ where $G$ is a constant, $M_i, M_j$ denote the "economic mass" (often simply the gross domestic product) and $D_{ij}$ the "distance" between countries $i$ and $j$, where "distance" is a complex notion that includes geographical, historical, linguistic and sociological components. We take the \textit{inverse} route and ask ourselves to which extent it is possible to reconstruct meaningful information about countries simply from knowing the bilateral trade volumes $F_{ij}$: indeed, we show that a remarkable amount of geopolitical information can be extracted. The main tool is a spectral decomposition of the Graph Laplacian as a tool to perform nonlinear dimensionality reduction. This may have further applications in economic analysis and provides a data-based approach to "trade distance".

q-fin.TR↗