arXiv subjects

Alpha A. Lee

Publications and source records attributed to Alpha A. Lee.

At least 19 recordsLinked to original sources

arXiv2024

Matbench Discovery -- A framework to evaluate machine learning crystal stability predictions

The rapid adoption of machine learning (ML) in domain sciences necessitates best practices and standardized benchmarking for performance evaluation. We present Matbench Discovery, an evaluation framework for ML energy models, applied as pre-filters for high-throughput searches of stable inorganic crystals. This framework addresses the disconnect between thermodynamic stability and formation energy, as well as retrospective vs. prospective benchmarking in materials discovery. We release a Python package to support model submissions and maintain an online leaderboard, offering insights into performance trade-offs. To identify the best-performing ML methodologies for materials discovery, we benchmarked various approaches, including random forests, graph neural networks (GNNs), one-shot predictors, iterative Bayesian optimizers, and universal interatomic potentials (UIP). Our initial results rank models by test set F1 scores for thermodynamic stability prediction: EquiformerV2 + DeNS > Orb > SevenNet > MACE > CHGNet > M3GNet > ALIGNN > MEGNet > CGCNN > CGCNN+P > Wrenformer > BOWSR > Voronoi fingerprint random forest. UIPs emerge as the top performers, achieving F1 scores of 0.57-0.82 and discovery acceleration factors (DAF) of up to 6x on the first 10k stable predictions compared to random selection. We also identify a misalignment between regression metrics and task-relevant classification metrics. Accurate regressors can yield high false-positive rates near the decision boundary at 0 eV/atom above the convex hull. Our results demonstrate UIPs' ability to optimize computational budget allocation for expanding materials databases. However, their limitations remain underexplored in traditional benchmarks. We advocate for task-based evaluation frameworks, as implemented here, to address these limitations and advance ML-guided materials discovery.

cond-mat.mtrl-sci↗

arXiv2023

GAUCHE: A Library for Gaussian Processes in Chemistry

We introduce GAUCHE, a library for GAUssian processes in CHEmistry. Gaussian processes have long been a cornerstone of probabilistic machine learning, affording particular advantages for uncertainty quantification and Bayesian optimisation. Extending Gaussian processes to chemical representations, however, is nontrivial, necessitating kernels defined over structured inputs such as graphs, strings and bit vectors. By defining such kernels in GAUCHE, we seek to open the door to powerful tools for uncertainty quantification and Bayesian optimisation in chemistry. Motivated by scenarios frequently encountered in experimental chemistry, we showcase applications for GAUCHE in molecular discovery and chemical reaction optimisation. The codebase is made available at https://github.com/leojklarner/gauche

physics.chem-ph↗

arXiv2022

Inferring global dynamics from local structure in liquid electrolytes

Ion transport in concentrated electrolytes plays a fundamental role in electrochemical systems such as lithium ion batteries. Nonetheless, the mechanism of transport amid strong ion-ion interactions remains enigmatic. A key question is whether the dynamics of ion transport can be predicted by the local static structure alone, and if so what are the key structural motifs that determine transport. In this paper, we show that machine learning can successfully decompose global conductivity into the spatio-temporal average of local, instantaneous ionic contributions, and relate this ``local molar conductivity" field to the local ionic environment. Our machine learning model accurately predicts the molar conductivity of electrolyte systems that were not part of the training set, suggesting that the dynamics of ion transport is predictable from local static structure. Further, through analysing this machine-learned local conductivity field, we observe that fluctuations in local conductivity at high concentration are negatively correlated with total molar conductivity. Surprisingly, these fluctuations arise due to a long tail distribution of low conductivity ions, rather than distinct ion pairs, and are spatially correlated through both like- and unlike-charge interactions. More broadly, our approach shows how machine learning can aid the understanding of complex soft matter systems, by learning a function that attributes global collective properties to local, atomistic contributions.

cond-mat.soft↗

arXiv2022

Achieving Robustness to Aleatoric Uncertainty with Heteroscedastic Bayesian Optimisation

Bayesian optimisation is a sample-efficient search methodology that holds great promise for accelerating drug and materials discovery programs. A frequently-overlooked modelling consideration in Bayesian optimisation strategies however, is the representation of heteroscedastic aleatoric uncertainty. In many practical applications it is desirable to identify inputs with low aleatoric noise, an example of which might be a material composition which consistently displays robust properties in response to a noisy fabrication process. In this paper, we propose a heteroscedastic Bayesian optimisation scheme capable of representing and minimising aleatoric noise across the input space. Our scheme employs a heteroscedastic Gaussian process (GP) surrogate model in conjunction with two straightforward adaptations of existing acquisition functions. First, we extend the augmented expected improvement (AEI) heuristic to the heteroscedastic setting and second, we introduce the aleatoric noise-penalised expected improvement (ANPEI) heuristic. Both methodologies are capable of penalising aleatoric noise in the suggestions and yield improved performance relative to homoscedastic Bayesian optimisation and random sampling on toy problems as well as on two real-world scientific datasets. Code is available at: \url{https://github.com/Ryan-Rhys/Heteroscedastic-BO}

stat.ML↗

arXiv2022

Rapid Discovery of Stable Materials by Coordinate-free Coarse Graining

A fundamental challenge in materials science pertains to elucidating the relationship between stoichiometry, stability, structure, and property. Recent advances have shown that machine learning can be used to learn such relationships, allowing the stability and functional properties of materials to be accurately predicted. However, most of these approaches use atomic coordinates as input and are thus bottle-necked by crystal structure identification when investigating novel materials. Our approach solves this bottleneck by coarse-graining the infinite search space of atomic coordinates into a combinatorially enumerable search space. The key idea is to use Wyckoff representations -- coordinate-free sets of symmetry-related positions in a crystal -- as the input to a machine learning model. Our model demonstrates exceptionally high precision in discovering new theoretically stable materials, identifying 1,569 materials that lie below the known convex hull of previously calculated materials from just 5,675 ab-initio calculations. Our approach opens up fundamental advances in computational materials discovery.

cond-mat.mtrl-sci↗

arXiv2021

Materials Informatics Reveals Unexplored Structure Space in Cuprate Superconductors

High-temperature superconducting cuprates have the potential to be transformative in a wide range of energy applications. In this work we analyse the corpus of historical data about cuprates using materials informatics and re-examine how their structures are related to their critical temperatures (Tc). The available data is highly clustered and no single database contains all the features of interest to properly examine trends. To work around these issues we employ a linear calibration approach that allows us to utilise multiple data sources -- combining fine resolution data for which the Tc is unknown with coarse resolution data where it is known. The hybrid data set constructed enables us to explore the trends in Tc with the apical and in-plane copper-oxygen distances. We show that large regions of the materials space have yet to be explored and highlight how novel experiments relying on nano-engineering of the crystal structure may enable us to explore such new regions. Based on the trends identified we propose that single layer Bi-based cuprates are good candidate systems for such experiments.

cond-mat.supr-con↗

arXiv2021

Modelling the Multiwavelength Variability of Mrk 335 using Gaussian Processes

The optical and UV variability of the majority of AGN may be related to the reprocessing of rapidly-changing X-ray emission from a more compact region near the central black hole. Such a reprocessing model would be characterised by lags between X-ray and optical/UV emission due to differences in light travel time. Observationally however, such lag features have been difficult to detect due to gaps in the lightcurves introduced through factors such as source visibility or limited telescope time. In this work, Gaussian process regression is employed to interpolate the gaps in the Swift X-ray and UV lightcurves of the narrow-line Seyfert 1 galaxy Mrk 335. In a simulation study of five commonly-employed analytic Gaussian process kernels, we conclude that the Matern 1/2 and rational quadratic kernels yield the most well-specified models for the X-ray and UVW2 bands of Mrk 335. In analysing the structure functions of the Gaussian process lightcurves, we obtain a broken power law with a break point at 125 days in the UVW2 band. In the X-ray band, the structure function of the Gaussian process lightcurve is consistent with a power law in the case of the rational quadratic kernel whilst a broken power law with a breakpoint at 66 days is obtained from the Matern 1/2 kernel. The subsequent cross-correlation analysis is consistent with previous studies and furthermore, shows tentative evidence for a broad X-ray-UV lag feature of up to 30 days in the lag-frequency spectrum where the significance of the lag depends on the choice of Gaussian process kernel.

astro-ph.HE↗

arXiv2021

Dataset Bias in the Natural Sciences: A Case Study in Chemical Reaction Prediction and Synthesis Design

Datasets in the Natural Sciences are often curated with the goal of aiding scientific understanding and hence may not always be in a form that facilitates the application of machine learning. In this paper, we identify three trends within the fields of chemical reaction prediction and synthesis design that require a change in direction. First, the manner in which reaction datasets are split into reactants and reagents encourages testing models in an unrealistically generous manner. Second, we highlight the prevalence of mislabelled data, and suggest that the focus should be on outlier removal rather than data fitting only. Lastly, we discuss the problem of reagent prediction, in addition to reactant prediction, in order to solve the full synthesis design problem, highlighting the mismatch between what machine learning solves and what a lab chemist would need. Our critiques are also relevant to the burgeoning field of using machine learning to accelerate progress in experimental Natural Sciences, where datasets are often split in a biased way, are highly noisy, and contextual variables that are not evident from the data strongly influence the outcome of experiments.

physics.chem-ph↗

arXiv2021

Machine learnt approximations to the bridge function yield improved closures for the Ornstein-Zernike equation

A key challenge for soft materials design and coarse-graining simulations is determining interaction potentials between components that give rise to desired condensed-phase structures. In theory, the Ornstein-Zernike equation provides an elegant framework for solving this inverse problem. Pioneering work in liquid state theory derived analytical closures for the framework. However, these analytical closures are approximations, valid only for specific classes of interaction potentials. In this work, we combine the physics of liquid state theory with machine learning to infer a closure directly from simulation data. The resulting closure is more accurate than commonly used closures across a broad range of interaction potentials. We show for two examples of a prototypical inverse design problem, fitting a coarse-grained simulation potential, that our approach leads to improved one-step inversion.

cond-mat.soft↗

arXiv2020

Bayesian unsupervised learning reveals hidden structure in concentrated electrolytes

Electrolytes play an important role in a plethora of applications ranging from energy storage to biomaterials. Notwithstanding this, the structure of concentrated electrolytes remains enigmatic. Many theoretical approaches attempt to model the concentrated electrolytes by introducing the idea of ion pairs, with ions either being tightly `paired' with a counter-ion, or `free' to screen charge. In this study we reframe the problem into the language of computational statistics, and test the null hypothesis that all ions share the same local environment. Applying the framework to molecular dynamics simulations, we show that this null hypothesis is not supported by data. Our statistical technique suggests the presence of distinct local ionic environments; surprisingly, these differences arise in like charge correlations rather than unlike charge attraction. The resulting fraction of particles in non-aggregated environments shows a universal scaling behaviour across different background dielectric constants and ionic concentrations.

cond-mat.soft↗

arXiv2020

Investigating 3D Atomic Environments for Enhanced QSAR

Predicting bioactivity and physical properties of molecules is a longstanding challenge in drug design. Most approaches use molecular descriptors based on a 2D representation of molecules as a graph of atoms and bonds, abstracting away the molecular shape. A difficulty in accounting for 3D shape is in designing molecular descriptors can precisely capture molecular shape while remaining invariant to rotations/translations. We describe a novel alignment-free 3D QSAR method using Smooth Overlap of Atomic Positions (SOAP), a well-established formalism developed for interpolating potential energy surfaces. We show that this approach rigorously describes local 3D atomic environments to compare molecular shapes in a principled manner. This method performs competitively with traditional fingerprint-based approaches as well as state-of-the-art graph neural networks on pIC$_{50}$ ligand-binding prediction in both random and scaffold split scenarios. We illustrate the utility of SOAP descriptors by showing that its inclusion in ensembling diverse representations statistically improves performance, demonstrating that incorporating 3D atomic environments could lead to enhanced QSAR for cheminformatics.

q-bio.QM↗

arXiv2020

Predicting materials properties without crystal structure: Deep representation learning from stoichiometry

Machine learning has the potential to accelerate materials discovery by accurately predicting materials properties at a low computational cost. However, the model inputs remain a key stumbling block. Current methods typically use descriptors constructed from knowledge of either the full crystal structure -- therefore only applicable to materials with already characterised structures -- or structure-agnostic fixed-length representations hand-engineered from the stoichiometry. We develop a machine learning approach that takes only the stoichiometry as input and automatically learns appropriate and systematically improvable descriptors from data. Our key insight is to treat the stoichiometric formula as a dense weighted graph between elements. Compared to the state of the art for structure-agnostic methods, our approach achieves lower errors with less data.

physics.comp-ph↗

arXiv2020

Materials Graph Transformer predicts the outcomes of inorganic reactions with reliable uncertainties

A common bottleneck for materials discovery is synthesis. While recent methodological advances have resulted in major improvements in the ability to predicatively design novel materials, researchers often still rely on trial-and-error approaches for determining synthesis procedures. In this work, we develop a model that predicts the major product of solid-state reactions. The cardinal feature of this approach is the construction of fixed-length, learned representations of reactions. Precursors are represented as nodes on a `reaction graph', and message-passing operations between nodes are used to embody the interactions between precursors in the reaction mixture. Through an ablation study, it is shown that this framework not only outperforms less physically-motivated baseline methods but also more reliably assesses the uncertainty in its predictions.

physics.comp-ph↗

arXiv2019

Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learning

Predicting bioactivity and physical properties of small molecules is a central challenge in drug discovery. Deep learning is becoming the method of choice but studies to date focus on mean accuracy as the main metric. However, to replace costly and mission-critical experiments by models, a high mean accuracy is not enough: Outliers can derail a discovery campaign, thus models need reliably predict when it will fail, even when the training data is biased; experiments are expensive, thus models need to be data-efficient and suggest informative training sets using active learning. We show that uncertainty quantification and active learning can be achieved by Bayesian semi-supervised graph convolutional neural networks. The Bayesian approach estimates uncertainty in a statistically principled way through sampling from the posterior distribution. Semi-supervised learning disentangles representation learning and regression, keeping uncertainty estimates accurate in the low data limit and allowing the model to start active learning from a small initial pool of training data. Our study highlights the promise of Bayesian deep learning for chemistry.

cs.LG↗

arXiv2019

Validating the Validation: Reanalyzing a large-scale comparison of Deep Learning and Machine Learning models for bioactivity prediction

Machine learning methods may have the potential to significantly accelerate drug discovery. However, the increasing rate of new methodological approaches being published in the literature raises the fundamental question of how models should be benchmarked and validated. We reanalyze the data generated by a recently published large-scale comparison of machine learning models for bioactivity prediction and arrive at a somewhat different conclusion. We show that the performance of support vector machines is competitive with that of deep learning methods. Additionally, using a series of numerical experiments, we question the relevance of area under the receiver operating characteristic curve as a metric in virtual screening, and instead suggest that area under the precision-recall curve should be used in conjunction with the receiver operating characteristic. Our numerical experiments also highlight challenges in estimating the uncertainty in model performance via scaffold-split nested cross validation.

cs.LG↗

arXiv2018

Screening Lengths in Ionic Fluids

The decay of correlations in ionic fluids is a classical problem in soft matter physics that underpins applications ranging from controlling colloidal self-assembly to batteries and supercapacitors. The conventional wisdom, based on analyzing a solvent-free electrolyte model, suggests that all correlation functions between species decay with a common decay length in the asymptotic far field limit. Nonetheless, a solvent is present in many electrolyte systems. We show using an analytical theory and molecular dynamics simulations that multiple decay lengths can coexist in the asymptotic limit as well as at intermediate distances once a hard sphere solvent is considered. Our analysis provides an explanation for the recently observed discontinuous change in the structural force across a thin film of ionic liquid-solvent mixtures as the composition is varied, as well as reframes recent debates in the literature about the screening length in concentrated electrolytes.

cond-mat.stat-mech↗

arXiv2018

Geometry of energy landscapes and the optimizability of deep neural networks

Deep neural networks are workhorse models in machine learning with multiple layers of non-linear functions composed in series. Their loss function is highly non-convex, yet empirically even gradient descent minimisation is sufficient to arrive at accurate and predictive models. It is hitherto unknown why are deep neural networks easily optimizable. We analyze the energy landscape of a spin glass model of deep neural networks using random matrix theory and algebraic geometry. We analytically show that the multilayered structure holds the key to optimizability: Fixing the number of parameters and increasing network depth, the number of stationary points in the loss function decreases, minima become more clustered in parameter space, and the tradeoff between the depth and width of minima becomes less severe. Our analytical results are numerically verified through comparison with neural networks trained on a set of classical benchmark datasets. Our model uncovers generic design principles of machine learning models.

cond-mat.dis-nn↗

arXiv2018

Inverse Ising inference by combining Ornstein-Zernike theory with deep learning

Inferring a generative model from data is a fundamental problem in machine learning. It is well-known that the Ising model is the maximum entropy model for binary variables which reproduces the sample mean and pairwise correlations. Learning the parameters of the Ising model from data is the challenge. We establish an analogy between the inverse Ising problem and the Ornstein-Zernike formalism in liquid state physics. Rather than analytically deriving the closure relation, we use a deep neural network to learn the closure from simulations of the Ising model. We show, using simulations as well as biochemical datasets, that the deep neural network model outperforms systematic field-theoretic expansions, is more data-efficient than the pseudolikelihood method, and can generalize well beyond the parameter regime of the training data. The neural network is able to learn from synthetic data, which can be generated with relative ease, to give accurate predictions on real world datasets.

cond-mat.stat-mech↗