arXiv subjects

Jaewan Lee

Publications and source records attributed to Jaewan Lee.

4 recordsLinked to original sources

arXiv2025

CAST: Cross Attention based multimodal fusion of Structure and Text for materials property prediction

Recent advancements in graph neural networks (GNNs) have significantly enhanced the prediction of material properties by modeling crystal structures as graphs. However, GNNs often struggle to capture global structural characteristics, such as crystal systems, limiting their predictive performance. To overcome this issue, we propose CAST, a cross-attention-based multimodal model that integrates graph representations with textual descriptions of materials, effectively preserving critical structural and compositional information. Unlike previous approaches, such as CrysMMNet and MultiMat, which rely on aggregated material-level embeddings, CAST leverages cross-attention mechanisms to combine fine-grained graph node-level and text token-level features. Additionally, we introduce a masked node prediction pretraining strategy that further enhances the alignment between node and text embeddings. Our experimental results demonstrate that CAST outperforms existing baseline models across four key material properties-formation energy, band gap, bulk modulus, and shear modulus-with average relative MAE improvements ranging from 10.2% to 35.7%. Analysis of attention maps confirms the importance of pretraining in effectively aligning multimodal representations. This study underscores the potential of multimodal learning frameworks for developing more accurate and globally informed predictive models in materials science.

cs.LG↗

arXiv2025

Geometric Embedding Alignment via Curvature Matching in Transfer Learning

Geometrical interpretations of deep learning models offer insightful perspectives into their underlying mathematical structures. In this work, we introduce a novel approach that leverages differential geometry, particularly concepts from Riemannian geometry, to integrate multiple models into a unified transfer learning framework. By aligning the Ricci curvature of latent space of individual models, we construct an interrelated architecture, namely Geometric Embedding Alignment via cuRvature matching in transfer learning (GEAR), which ensures comprehensive geometric representation across datapoints. This framework enables the effective aggregation of knowledge from diverse sources, thereby improving performance on target tasks. We evaluate our model on 23 molecular task pairs sourced from various domains and demonstrate significant performance gains over existing benchmark model under both random (14.4%) and scaffold (8.3%) data splits.

cs.LG↗

arXiv2025

MolMole: Molecule Mining from Scientific Literature

The extraction of molecular structures and reaction data from scientific documents is challenging due to their varied, unstructured chemical formats and complex document layouts. To address this, we introduce MolMole, a vision-based deep learning framework that unifies molecule detection, reaction diagram parsing, and optical chemical structure recognition (OCSR) into a single pipeline for automating the extraction of chemical data directly from page-level documents. Recognizing the lack of a standard page-level benchmark and evaluation metric, we also present a testset of 550 pages annotated with molecule bounding boxes, reaction labels, and MOLfiles, along with a novel evaluation metric. Experimental results demonstrate that MolMole outperforms existing toolkits on both our benchmark and public datasets. The benchmark testset will be publicly available, and the MolMole toolkit will be accessible soon through an interactive demo on the LG AI Research website. For commercial inquiries, please contact us at \href{mailto:contact_ddu@lgresearch.ai}{contact\_ddu@lgresearch.ai}.

cs.LG↗

arXiv2024

Lattice Lingo: Effect of Textual Detail on Multimodal Learning for Property Prediction of Crystals

Most prediction models for crystal properties employ a unimodal perspective, with graph-based representations, overlooking important non-local information that affects crystal properties. Some recent studies explore the impact of integrating graph and textual information on crystal property predictions to provide the model with this "missing" information by concatenation of embeddings. However, such studies do not evaluate which type of textual information is actually beneficial. We concatenate graph representations with text representations derived from textual descriptions with varying levels of detail. These descriptions, generated using the Robocrystallographer package, encompass global (e.g., space group, crystal type), local (e.g., bond lengths, coordination environment), and semiglobal (e.g., connectivity, arrangements) information about the structures. Our approach investigates how augmenting graph-based information with various levels of textual detail influences the performance for predictions for shear modulus and bulk modulus. We demonstrate that while graph representations can capture local structural information, incorporating semiglobal textual information enhances model performance the most. Global information can support performance further in the presence of semiglobal information. Our findings suggest that the strategic inclusion of textual information can enhance property prediction, thereby advancing the design and discovery of advanced novel materials for battery electrodes, catalysts, etc.

cond-mat.mtrl-sci↗