arXiv subjects

Luis M. Antunes

Publications and source records attributed to Luis M. Antunes.

4 recordsLinked to original sources

arXiv2026

Discovery and recovery of crystalline materials with property-conditioned transformers

Generative models have recently shown great promise for accelerating the design and discovery of new functional materials. Conditional generation enhances this capacity by allowing inverse design, where specific desired properties can be requested during the generation process. However, conditioning of transformer-based approaches, in particular, is constrained by discrete tokenisation schemes and the risk of catastrophic forgetting during fine-tuning. This work introduces CrystaLLM-π (property injection), a conditional autoregressive framework that integrates continuous property representations directly into the transformer's attention mechanism. Two architectures, Property-Key-Value (PKV) Prefix attention and PKV Residual attention, are presented. These methods bypass inefficient sequence-level tokenisation and preserve foundational knowledge from unsupervised pre-training on Crystallographic Information Files (CIFs) as textual input. We establish the efficacy of these mechanisms through systematic robustness studies and evaluate the framework's versatility across two distinct tasks. First, for structure recovery, the model processes high-dimensional, heterogeneous X-ray diffraction patterns, achieving structural accuracy competitive with specialised models and demonstrating applications to experimental structure recovery and polymorph differentiation. Second, for materials discovery, the model is fine-tuned on a specialised photovoltaic dataset to generate novel, stable candidates validated by Density Functional Theory (DFT). It implicitly learns to target optimal band gap regions for high photovoltaic efficiency, demonstrating a capability to map complex structure-property relationships. CrystaLLM-π provides a unified, flexible, and computationally efficient framework for inverse materials design.

cond-mat.mtrl-sci↗

arXiv2024

Crystal Structure Generation with Autoregressive Large Language Modeling

The generation of plausible crystal structures is often the first step in predicting the structure and properties of a material from its chemical composition. Quickly generating and predicting inorganic crystal structures is important for the discovery of new materials, which can target applications such as energy or electronic devices. However, most current methods for crystal structure prediction are computationally expensive, slowing the pace of innovation. Seeding structure prediction algorithms with quality generated candidates can overcome a major bottleneck. Here, we introduce CrystaLLM, a methodology for the versatile generation of crystal structures, based on the autoregressive large language modeling (LLM) of the Crystallographic Information File (CIF) format. Trained on millions of CIF files, CrystaLLM focuses on modeling crystal structures through text. CrystaLLM can produce plausible crystal structures for a wide range of inorganic compounds unseen in training, as demonstrated by ab initio simulations. The integration with predictors of formation energy permits the use of a Monte Carlo Tree Search algorithm to improve the generation of meaningful structures. Our approach challenges conventional representations of crystals, and demonstrates the potential of LLMs for learning effective 'world models' of crystal chemistry, which will lead to accelerated discovery and innovation in materials science.

cond-mat.mtrl-sci↗

arXiv2022

Predicting Thermoelectric Transport Properties from Composition with Attention-based Deep Learning

Thermoelectric materials can be used to construct devices which recycle waste heat into electricity. However, the best known thermoelectrics are based on rare, expensive or even toxic elements, which limits their widespread adoption. To enable deployment on global scales, new classes of effective thermoelectrics are thus required. $\textit{Ab initio}$ models of transport properties can help in the design of new thermoelectrics, but they are still too computationally expensive to be solely relied upon for high-throughput screening in the vast chemical space of all possible candidates. Here, we use models constructed with modern machine learning techniques to scan very large areas of inorganic materials space for novel thermoelectrics, using composition as an input. We employ an attention-based deep learning model, trained on data derived from $\textit{ab initio}$ calculations, to predict a material's Seebeck coefficient, electrical conductivity, and power factor over a range of temperatures and $\textit{n}$- or $\textit{p}$-type doping levels, with surprisingly good performance given the simplicity of the input, and with significantly lower computational cost. The results of applying the model to a space of known and hypothetical binary and ternary selenides reveal several materials that may represent promising thermoelectrics. Our study establishes a protocol for composition-based prediction of thermoelectric behaviour that can be easily enhanced as more accurate theoretical or experimental databases become available.

cond-mat.mtrl-sci↗

arXiv2021

Distributed Representations of Atoms and Materials for Machine Learning

The use of machine learning is becoming increasingly common in computational materials science. To build effective models of the chemistry of materials, useful machine-based representations of atoms and their compounds are required. We derive distributed representations of compounds from their chemical formulas only, via pooling operations of distributed representations of atoms. These compound representations are evaluated on ten different tasks, such as the prediction of formation energy and band gap, and are found to be competitive with existing benchmarks that make use of structure, and even superior in cases where only composition is available. Finally, we introduce a new approach for learning distributed representations of atoms, named SkipAtom, which makes use of the growing information in materials structure databases.

cond-mat.mtrl-sci↗