Searcharxiv⌕ Search

arXiv subjects

Adji Bousso Dieng

Publications and source records attributed to Adji Bousso Dieng.

At least 19 recordsLinked to original sources

MOOD: Benchmarking Post-Hoc OOD Detection for Materials Property Prediction

Out-of-distribution (OOD) detection is essential for deploying machine learning models in high stakes scientific applications, yet methods that flag OOD inputs without retraining the model have been validated almost exclusively on image classification. We introduce MOOD (Materials Out-of-Distribution Detection), a benchmark for evaluating whether these post-hoc OOD detectors transfer to graph neural networks (GNNs) on materials property prediction. MOOD evaluates 15 post-hoc detectors and 2 supervised diagnostic probes across 4 GNN architectures, 5 MatBench tasks, and 15 physically motivated compositional and structural shifts, yielding 264 evaluation settings and over 13,000 AUROC measurements across three encoder training seeds. Our evaluation reveals three key findings. First, current post-hoc detectors perform near chance; all 15 detectors achieve an AUROC around 0.5. Second, detector failures decompose into two regimes: method bottlenecks and encoder bottlenecks. Method bottlenecks arise when supervised probes can separate in-distribution (ID) from OOD samples but unsupervised detectors fail, meaning that the embeddings contain OOD signal that current detectors do not extract. Encoder bottlenecks arise when even supervised probes fail to separate ID from OOD samples, indicating that the relevant shift is not accessible in the frozen GNN representation. Third, when OOD prediction error exceeds ID error, the AUROC of distance-based and density-based detectors correlates with the size of that gap. Collectively, these findings indicate that reliable OOD detection in scientific domains requires representation learning strategies that explicitly preserve OOD relevant signal, particularly for tasks where current GNN architectures encode no extractable signal at all.

cs.LG↗

Vendi Anomaly Scores for Efficient and Accurate Anomaly Detection

Anomaly detection (AD) is a critical for trustworthy machine learning and scientific discovery. Existing methods typically define anomalies through local density, isolation heuristics, or distance from a fitted distribution, making them sensitive to neighborhood scale, partitioning choices, or restrictive distributional assumptions. In this work, we introduce a new paradigm by formulating anomaly detection from a diversity perspective. We propose the $\textit{Vendi Anomaly Score}$ (VAS), which detects anomalies by quantifying how much a sample changes the diversity of the dataset, as measured by the Vendi score, when removed. VAS captures both local redundancy and global data structure without relying on density estimation or partitioning heuristics. Moreover, VAS requires no hyperparameter tuning, instead adapting automatically to the spectral structure of the data. VAS is non-parametric and scales linearly with dataset size. Across 148 benchmark embedding datasets derived from 10 source datasets and 4 embedding architectures, VAS achieves state-of-the-art AD performance. We further validate VAS on large-scale ImageNet experiments, where it remains robust as both contamination rate and dataset scale increase.

cs.LG↗

Diversity-Aware Active Learning Builds Materials Datasets for Current and Future Objectives

Materials science data collection can be expensive, making dataset reuse and long-term utility important for future discovery. In practice, data collection often prioritizes properties of current interest while other properties are excluded from the acquisition process. This can produce datasets that are well suited for current objectives but lose information needed for properties of future interest. Here, we present how to construct datasets that remain informative for both. We develop and evaluate acquisition frameworks that combine predictive disagreement with diversity in material features, predicted outcomes, or both. We test these approaches across four DFT datasets and an experimental thermoelectric dataset. Diversity inclusion in dataset construction can reduce present and future objectives trade-off by preserving a broader range of materials and property behavior during acquisition. Across all datasets, the Non-dominated Sorting Genetic Algorithm II with disagreements and feature diversity (NSGA-II Dis. + diversity) framework developed in this work provides the most robust balance between current and future objectives. It maintains prediction performance above random sampling for current objectives while mitigating bias on excluded properties and, in many cases, improving their performance beyond random sampling. This diversity-aware acquisition framework provides a practical route to building materials datasets with unbiased quality entries and mitigating cold-start limitations in subsequent modeling and discovery campaigns.

cond-mat.mtrl-sci↗

Vendi Novelty Scores for Out-of-Distribution Detection

Out-of-distribution (OOD) detection is critical for the safe deployment of machine learning systems. Existing post-hoc detectors typically rely on model confidence scores or likelihood estimates in feature space, often under restrictive distributional assumptions. In this work, we introduce a third paradigm and formulate OOD detection from a diversity perspective. We propose the Vendi Novelty Score (VNS), an OOD detector based on the Vendi Scores (VS), a family of similarity-based diversity metrics. VNS quantifies how much a test sample increases the VS of the in-distribution feature set, providing a principled notion of novelty that does not require density modeling. VNS is linear-time, non-parametric, and naturally combines class-conditional (local) and dataset-level (global) novelty signals. Across multiple image classification benchmarks and network architectures, VNS achieves state-of-the-art OOD detection performance. Remarkably, VNS retains this performance when computed using only 1% of the training data, enabling deployment in memory- or access-constrained settings.

cs.LG↗

Vendi Information Gain for Active Learning and its Application to Ecology

While monitoring biodiversity through camera traps has become an important endeavor for ecological research, identifying species in the captured image data remains a major bottleneck due to limited labeling resources. Active learning -- a machine learning paradigm that selects the most informative data to label and train a predictive model -- offers a promising solution, but typically focuses on uncertainty in the individual predictions without considering uncertainty across the entire dataset. We introduce a new active learning policy, Vendi information gain (VIG), that selects images based on their impact on dataset-wide prediction uncertainty, capturing both informativeness and diversity. We applied VIG to the Snapshot Serengeti dataset and compared it against common active learning methods. VIG needs only 3% of the available data to reach 75% accuracy, a level that baselines require more than 10% of the data to achieve. With 10% of the data, VIG attains 88% predictive accuracy, 12% higher than the best of the baselines. This improvement in performance is consistent across metrics and batch sizes, and we show that VIG also collects more diverse data in the feature space. VIG has broad applicability beyond ecology, and our results highlight its value for biodiversity monitoring in data-limited environments.

cs.LG↗

Generative AI for Biosciences: Emerging Threats and Roadmap to Biosecurity

The rapid adoption of generative artificial intelligence (GenAI) in the biosciences is transforming biotechnology, medicine, and synthetic biology. Yet this advancement is intrinsically linked to new vulnerabilities, as GenAI lowers the barrier to misuse and introduces novel biosecurity threats, such as generating synthetic viral proteins or toxins. These dual-use risks are often overlooked, as existing safety guardrails remain fragile and can be circumvented through deceptive prompts or jailbreak techniques. In this Perspective, we first outline the current state of GenAI in the biosciences and emerging threat vectors ranging from jailbreak attacks and privacy risks to the dual-use challenges posed by autonomous AI agents. We then examine urgent gaps in regulation and oversight, drawing on insights from 130 expert interviews across academia, government, industry, and policy. A large majority ($\approx 76$\%) expressed concern over AI misuse in biology, and 74\% called for the development of new governance frameworks. Finally, we explore technical pathways to mitigation, advocating a multi-layered approach to GenAI safety. These defenses include rigorous data filtering, alignment with ethical principles during development, and real-time monitoring to block harmful requests. Together, these strategies provide a blueprint for embedding security throughout the GenAI lifecycle. As GenAI becomes integrated into the biosciences, safeguarding this frontier requires an immediate commitment to both adaptive governance and secure-by-design technologies.

cs.CR↗

Are neural scaling laws leading quantum chemistry astray?

Neural scaling laws are driving the machine learning community toward training ever-larger foundation models across domains, assuring high accuracy and transferable representations for extrapolative tasks. We test this promise in quantum chemistry by scaling model capacity and training data from quantum chemical calculations. As a generalization task, we evaluate the resulting models' predictions of the bond dissociation energy of neutral H$_2$, the simplest possible molecule. We find that, regardless of dataset size or model capacity, models trained only on stable structures fail dramatically to even qualitatively reproduce the H$_2$ energy curve. Only when compressed and stretched geometries are explicitly included in training do the predictions roughly resemble the correct shape. Nonetheless, the largest foundation models trained on the largest and most diverse datasets containing dissociating diatomics exhibit serious failures on simple diatomic molecules. Most strikingly, they cannot reproduce the trivial repulsive energy curve of two bare protons, revealing their failure to learn the basic Coulomb's law involved in electronic structure theory. These results suggest that scaling alone is insufficient for building reliable quantum chemical models.

physics.chem-ph↗

Applications of the Vendi score in genomic epidemiology

The Vendi score (VS), a diversity metric recently conceived in the context of machine learning, with applications in a wide range of fields, has a few distinct advantages over the metrics commonly used in ecology. It is classification-independent, incorporates abundance information, and has a tunable sensitivity to rare/abundant types. Using rich COVID-19 sequence data as a paradigm, we develop methods for applying the VS to time-resolved sequence data. We show how the VS allows for characterization of the overall diversity of circulating viruses and for discernment of emerging variants prior to formal identification. Furthermore, applying the VS to phylogenetic trees provides a convenient overview of within-clade diversity which can aid viral variant detection.

q-bio.PE↗

A Unified and Predictive Measure of Functional Diversity

Despite the critical role of functional diversity (FD) in understanding ecological systems and processes, its robust quantification remains a significant challenge. A long-held view in the field is that it is not possible to capture its three facets -- functional richness, functional divergence, and functional evenness -- in a single index. This perspective has prompted recent proposals for FD measurement to use three separate indices, one for each aspect. Here, we challenge this paradigm by demonstrating that the probability-weighted Vendi Score (pVS), first introduced by Friedman and Dieng (2023), can serve as a powerful functional diversity index that can capture its three facets. We adapt pVS to functional ecology by defining it as the exponential of the Rényi entropy of the eigenvalues of the abundance-weighted trait similarity matrix. This formulation allows pVS to be applicable at any biological level. It can be defined at the species level, at which most existing FD metrics are defined, and at the individual level to naturally incorporate intraspecific trait variation (ITV) when detailed data are available. We theoretically and empirically demonstrate the robustness of pVS. We first mathematically prove it satisfies several essential desiderata for FD metrics, including invariance to functional redundancy, set monotonicity, distance monotonicity, and concavity. We then show that pVS consistently exhibits the expected ground-truth behavior on simulated ecosystem scenarios under which many FD metrics fail. By integrating abundances and trait similarities within a single, theoretically sound framework, pVS provides a generally applicable index for ecology.

q-bio.PE↗

Rethinking Ecological Measures Of Functional Diversity

Understanding functional diversity, the range and variability of species' roles and actions within their communities, is key to predicting and preserving the functions that sustain both nature and human well-being. In this paper, we provide a comprehensive review of the literature on functional diversity measurement. We begin by consolidating essential criteria that effective measures of functional diversity should meet. We then evaluate fifteen widely used functional diversity metrics against these criteria and assess their performance across six synthetic ecosystem scenarios where optimal behavior is known. Surprisingly, our analysis reveals that none of the widely used metrics fully satisfy all the established requirements, and all fail in at least one ecosystem scenario. In particular, we find that almost all metrics flagrantly violate set monotonicity and distance monotonicity, requirements that adding a novel species should increase diversity, and that the magnitude of that increase should grow with trait dissimilarity. We also find that metrics fail to decline when rare, functionally extreme species are lost, and even increase when a perfectly redundant species is added. These critical flaws leave them blind to the very biodiversity loss that functional diversity measures are intended to detect. Our findings underscore the urgent need to develop a new generation of functional diversity metrics that more accurately reflect ecological realities.

q-bio.PE↗

The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance

Current state-of-the-art dynamical models, such as Mamba, assume the same level of noisiness for all elements of a given sequence, which limits their performance on noisy temporal data. In this paper, we introduce the $α$-Alternator, a novel generative model for time-dependent data that dynamically adapts to the complexity introduced by varying noise levels in sequences. The $α$-Alternator leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to adjust, at each time step $t$, the influence of the sequence element at time $t$ and the latent representation of the dynamics up to that time step on the predicted future dynamics. This influence is captured by a parameter that is learned and shared across all sequences in a given dataset. The sign of this parameter determines the direction of influence. A negative value indicates a noisy dataset, where a sequence element that increases the VS is considered noisy, and the model relies more on the latent history when processing that element. Conversely, when the parameter is positive, a sequence element that increases the VS is considered informative, and the $α$-Alternator relies more on this new input than on the latent history when updating its predicted latent dynamics. The $α$-Alternator is trained using a combination of observation masking and Alternator loss minimization. Masking simulates varying noise levels in sequences, enabling the model to be more robust to these fluctuations and improving its performance in trajectory prediction, imputation, and forecasting. Our experimental results demonstrate that the $α$-Alternator outperforms both Alternators and state-of-the-art state-space models across neural decoding and time-series forecasting benchmarks.

cs.LG↗

Vendi-RAG: Adaptively Trading-Off Diversity And Quality Significantly Improves Retrieval Augmented Generation With LLMs

Retrieval-augmented generation (RAG) enhances large language models (LLMs) for domain-specific question-answering (QA) tasks by leveraging external knowledge sources. However, traditional RAG systems primarily focus on relevance-based retrieval and often struggle with redundancy, especially when reasoning requires connecting information from multiple sources. This paper introduces Vendi-RAG, a framework based on an iterative process that jointly optimizes retrieval diversity and answer quality. This joint optimization leads to significantly higher accuracy for multi-hop QA tasks. Vendi-RAG leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to promote semantic diversity in document retrieval. It then uses an LLM judge that evaluates candidate answers, generated after a reasoning step, and outputs a score that the retriever uses to balance relevance and diversity among the retrieved documents during each iteration. Experiments on three challenging datasets -- HotpotQA, MuSiQue, and 2WikiMultiHopQA -- demonstrate Vendi-RAG's effectiveness in multi-hop reasoning tasks. The framework achieves significant accuracy improvements over traditional single-step and multi-step RAG approaches, with accuracy increases reaching up to +4.2% on HotpotQA, +4.1% on 2WikiMultiHopQA, and +1.3% on MuSiQue compared to Adaptive-RAG, the current best baseline. The benefits of Vendi-RAG are even more pronounced as the number of retrieved documents increases. Finally, we evaluated Vendi-RAG across different LLM backbones, including GPT-3.5, GPT-4, and GPT-4o-mini, and observed consistent improvements, demonstrating that the framework's advantages are model-agnostic.

cs.CL↗

Alternators With Noise Models

Alternators have recently been introduced as a framework for modeling time-dependent data. They often outperform other popular frameworks, such as state-space models and diffusion models, on challenging time-series tasks. This paper introduces a new Alternator model, called Alternator++, which enhances the flexibility of traditional Alternators by explicitly modeling the noise terms used to sample the latent and observed trajectories, drawing on the idea of noise models from the diffusion modeling literature. Alternator++ optimizes the sum of the Alternator loss and a noise-matching loss. The latter forces the noise trajectories generated by the two noise models to approximate the noise trajectories that produce the observed and latent trajectories. We demonstrate the effectiveness of Alternator++ in tasks such as density estimation, time series imputation, and forecasting, showing that it outperforms several strong baselines, including Mambas, ScoreGrad, and Dyffusion.

cs.LG↗

Vendi Information Gain: An Alternative To Mutual Information For Science And Machine Learning

In his 1948 seminal paper A Mathematical Theory of Communication that birthed information theory, Claude Shannon introduced mutual information (MI), which he called "rate of transmission", as a way to quantify information gain (IG) and defined it as the difference between the marginal and conditional entropy of a random variable. While MI has become a standard tool in science and engineering, it has several shortcomings. First, MI is often intractable - it requires a density over samples with tractable Shannon entropy - and existing techniques for approximating it often fail, especially in high dimensions. Moreover, in settings where MI is tractable, its symmetry and insensitivity to sample similarity are undesirable. In this paper, we propose the Vendi Information Gain (VIG), a novel alternative to MI that leverages the Vendi scores, a flexible family of similarity-based diversity metrics. We call the logarithm of the VS the Vendi entropy and define VIG as the difference between the marginal and conditional Vendi entropy of a variable. Being based on the VS, VIG accounts for similarity. Furthermore, VIG generalizes MI and recovers it under the assumption that the samples are completely dissimilar. Importantly, VIG only requires samples and not a probability distribution over them. Finally, it is asymmetric, a desideratum for a good measure of IG that MI fails to meet. VIG extends information theory to settings where MI completely fails. For example, we use VIG to describe a novel, unified framework for active data acquisition, a popular paradigm of modern data-driven science. We demonstrate the advantages of VIG over MI in diverse applications, including in cognitive science to model human response times to external stimuli and in epidemiology to learn epidemic processes and identify disease hotspots in different countries via level-set estimation.

cs.IT↗

MuAPBEK: An Improved Analytical Kinetic Energy Density Functional for Quantum Chemistry

Orbital-free density functional theory (OFDFT) offers a true realization of the Hohenberg-Kohn theorems, enabling full quantum-mechanical studies of electronic systems based solely on electron densities. However, OFDFT remains limited by the difficulty of formulating accurate kinetic-energy density functionals. In this paper, we substantially enhance the accuracy of OFDFT energies and densities by tuning, during density initialization, the parameter $μ$ of the APBEK functional, which arises in the second-order gradient expansion of the kinetic energy for semiclassical neutral atoms. We augment this parameterized APBEK functional with two physically-motivated, non-empirical corrections derived from Kato's cusp condition and the virial theorem. The resulting functional, which we call MuAPBEK, is benchmarked against Kohn-Sham density functional theory (KSDFT) on atoms, organic molecules from the QM9 dataset, and the anti-malarial drug artemisinin. MuAPBEK achieves much lower energy errors than standard APBEK and Thomas-Fermi-von-Weizsacker functionals, even when the latter two are evaluated on converged KSDFT densities. Its mean absolute energy errors on atoms and molecules are 161 and 122 kcal/mol, respectively, indicating that MuAPBEK's errors do not scale with system size. MuAPBEK also yields accurate densities, with a mean integrated absolute density error of 1.8 electrons for molecules. Importantly, one step of our density optimization scheme is at least ten times faster than a single KSDFT self-consistent field cycle and exhibits a lower-order computational time complexity of $O(N^{1.96})$ with respect to system size, $N$. Our results indicate that highly-accurate OFDFT for large-scale quantum simulations beyond the practical limits of KSDFT is within reach.

physics.chem-ph↗

Building Machine Learning Challenges for Anomaly Detection in Science

Scientific discoveries are often made by finding a pattern or object that was not predicted by the known rules of science. Oftentimes, these anomalous events or objects that do not conform to the norms are an indication that the rules of science governing the data are incomplete, and something new needs to be present to explain these unexpected outliers. The challenge of finding anomalies can be confounding since it requires codifying a complete knowledge of the known scientific behaviors and then projecting these known behaviors on the data to look for deviations. When utilizing machine learning, this presents a particular challenge since we require that the model not only understands scientific data perfectly but also recognizes when the data is inconsistent and out of the scope of its trained behavior. In this paper, we present three datasets aimed at developing machine learning-based anomaly detection for disparate scientific domains covering astrophysics, genomics, and polar science. We present the different datasets along with a scheme to make machine learning challenges around the three datasets findable, accessible, interoperable, and reusable (FAIR). Furthermore, we present an approach that generalizes to future machine learning challenges, enabling the possibility of large, more compute-intensive challenges that can ultimately lead to scientific discovery.

cs.LG↗

Behavior-Inspired Neural Networks for Relational Inference

From pedestrians to Kuramoto oscillators, interactions between agents govern how dynamical systems evolve in space and time. Discovering how these agents relate to each other has the potential to improve our understanding of the often complex dynamics that underlie these systems. Recent works learn to categorize relationships between agents based on observations of their physical behavior. These approaches model relationship categories as outcomes of a categorical distribution which is limiting and contrary to real-world systems, where relationship categories often intermingle and interact. In this work, we introduce a level of abstraction between the observable behavior of agents and the latent categories that determine their behavior. To do this, we learn a mapping from agent observations to agent preferences for a set of latent categories. The learned preferences and inter-agent proximity are integrated in a nonlinear opinion dynamics model, which allows us to naturally identify mutually exclusive categories, predict an agent's evolution in time, and control an agent's behavior. Through extensive experiments, we demonstrate the utility of our model for learning interpretable categories, and the efficacy of our model for long-horizon trajectory prediction.

cs.LG↗

The Vendiscope: An Algorithmic Microscope For Data Collections

The evolution of microscopy, beginning with its invention in the late 16th century, has continuously enhanced our ability to explore and understand the microscopic world, enabling increasingly detailed observations of structures and phenomena. In parallel, the rise of data-driven science has underscored the need for sophisticated methods to explore and understand the composition of complex data collections. This paper introduces the Vendiscope, the first algorithmic microscope designed to extend traditional microscopy to computational analysis. The Vendiscope leverages the Vendi scores -- a family of differentiable diversity metrics rooted in ecology and quantum mechanics -- and assigns weights to data points based on their contribution to the overall diversity of the collection. These weights enable high-resolution data analysis at scale. We demonstrate this across biology, materials science, and machine learning (ML). We analyzed the $250$ million protein sequences in the protein universe, discovering that over $200$ million are near-duplicates and that AlphaFold fails on proteins with Gene Ontology (GO) functions that contribute most to diversity. Applying the Vendiscope to the Materials Project database led to similar findings: more than $85\%$ of the crystals with formation energy data are near-duplicates and ML models perform poorly on materials that enhance diversity. Additionally, the Vendiscope can be used to study phenomena such as memorization in generative models. We used the Vendiscope to identify memorized training samples from $13$ different generative models and found that the best-performing ones often memorize the training samples that contribute least to diversity. Our findings demonstrate that the Vendiscope can serve as a powerful tool for data-driven science.

cs.LG↗