arXiv subjects

Hiroyuki Kusuhara

Publications and source records attributed to Hiroyuki Kusuhara.

8 recordsLinked to original sources

arXiv2026

From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models

Understanding how chemical language models (CLMs) learn chemical meaning from molecular string representations, rather than only surface-level string patterns, is an important question in chemical representation learning and machine learning for chemistry. Chirality provides a demanding test case: enantiomers can differ greatly in pharmacological activity and toxicity, yet CLMs often struggle to distinguish chiral configurations reliably. Here we present Pan-CORE (Pan-Chemical Omniscale Representation Engine), a family of autoregressive Transformer-based encoder-decoder models for SMILES translation, and use high-temporal-resolution checkpoint analysis to investigate how chiral information is learned during training. Across all tested Pan-CORE variants, we observe a reproducible jump-up in which chiral-token accuracy rises abruptly after a long plateau, suggesting that chiral learning stagnation is not explained by model capacity alone and instead reflects the complexity of chiral constraints. Analyses of attention dynamics, residual-stream trajectories, and latent-space geometry support an encoder-centered mechanism in which chiral-token representations undergo transient destabilization and reconstruction, seen as a V-shaped drop and recovery in vector norm and directional stability, together with a clear reorganization of chiral molecular representations in the latent space. Encoder-decoder cross-evaluation further supports the encoder-centered nature of the transition, and targeted attention-head ablation identifies a small set of chiral-sensitive heads whose removal selectively reduces chiral-token accuracy even in the fully trained model. These findings show that SMILES translation can serve as a useful experimental system for mechanistic analysis of semantic emergence in CLMs, with implications for interpretable chemical representation learning.

cs.LG↗

arXiv2026

Exploring Multi-Objective Trade-offs in Reference Compound Selection for Validation Studies of Toxicity Assays

In chemical safety assessment, validation studies rely on reference compound lists to evaluate the applicability of alternative methods prior to regulatory acceptance. These lists are expected to cover multiple aspects, including chemical structure, physicochemical properties, and toxicity profiles. In practice, however, trade-offs among these aspects are typically addressed implicitly through expert judgment, making them difficult to examine systematically. Here, we formulate reference compound selection for toxicity assay validation as an explicit multi-objective design problem. We define three interpretable objectives capturing structural, physicochemical, and toxicity diversity, and employ a genetic algorithm as an exploratory tool to examine the trade-off structure and resulting Pareto-optimal solutions. Rather than prescribing optimal or recommended compound sets, this formulation enables systematic exploration of designs and explicit comparison of their positions within a common design space. As an illustrative application, we link existing assay datasets with expert-curated validation lists by treating ``selected as a reference compound`` as an annotation on the underlying compound pool. We show that expert-selected, random, and algorithmically generated compound lists occupy distinct regions of the design space. Furthermore, under an illustrative fixed modeling setup, different regions of the design space were associated with different observed evaluation outcomes, supporting the view that reference compound selection constitutes a structured dimension of evaluation design. Together, these results provide a methodological perspective for treating reference compound selection as an analyzable design object, complementing established expert-driven practices.

q-bio.QM↗

arXiv2026

Notation-level confounding: When inconsistent molecular notations mislead chemical language models

Chemical language models (CLMs) are increasingly used for molecular design and property prediction. Because these models learn from textual encodings of molecules, differences in how such encodings are generated may affect their behavior. In cheminformatics, the term canonical SMILES implies a single standardized notation, yet different toolkits define distinct canonicalization rules, yielding multiple canonical strings for the same molecule. To examine how this variability arises and why it matters, we surveyed 264 CLM papers in PubMed and found that about half did not specify their canonicalization procedure, limiting transparency and reproducibility. Using a molecular translation framework, we show that when multiple valid notations are mixed or left undocumented, inconsistent notations distort latent representations and, in some benchmarks, can spuriously inflate predictive accuracy, a phenomenon we term notation-level confounding. These findings demonstrate how subtle differences in SMILES generation can mislead CLMs and highlight the importance of explicitly reporting preprocessing tools and settings.

q-bio.QM↗

arXiv2024

A novel molecule generative model of VAE combined with Transformer for unseen structure generation

Recently, molecule generation using deep learning has been actively investigated in drug discovery. In this field, Transformer and VAE are widely used as powerful models, but they are rarely used in combination due to structural and performance mismatch of them. This study proposes a model that combines these two models through structural and parameter optimization in handling diverse molecules. The proposed model shows comparable performance to existing models in generating molecules, and showed by far superior performance in generating molecules with unseen structures. Another advantage of this VAE model is that it generates molecules from latent representation, and therefore properties of molecules can be easily predicted or conditioned with it, and indeed, we show that the latent representation of the model successfully predicts molecular properties. Ablation study suggested the advantage of VAE over other generative models like language model in generating novel molecules. It also indicated that the latent representation can be shortened to ~32 dimensional variables without loss of reconstruction, suggesting the possibility of a much smaller molecular descriptor or model than existing ones. This study is expected to provide a virtual chemical library containing a wide variety of compounds for virtual screening and to enable efficient screening.

q-bio.BM↗

arXiv2024

Difficulty in chirality recognition for Transformer architectures learning chemical structures from string

Recent years have seen rapid development of descriptor generation based on representation learning of extremely diverse molecules, especially those that apply natural language processing (NLP) models to SMILES, a literal representation of molecular structure. However, little research has been done on how these models understand chemical structure. To address this black box, we investigated the relationship between the learning progress of SMILES and chemical structure using a representative NLP model, the Transformer. We show that while the Transformer learns partial structures of molecules quickly, it requires extended training to understand overall structures. Consistently, the accuracy of molecular property predictions using descriptors generated from models at different learning steps was similar from the beginning to the end of training. Furthermore, we found that the Transformer requires particularly long training to learn chirality and sometimes stagnates with low performance due to misunderstanding of enantiomers. These findings are expected to deepen the understanding of NLP models in chemistry.

cs.LG↗

arXiv2023

NRBdMF: A recommendation algorithm for predicting drug effects considering directionality

Predicting the novel effects of drugs based on information about approved drugs can be regarded as a recommendation system. Matrix factorization is one of the most used recommendation systems and various algorithms have been devised for it. A literature survey and summary of existing algorithms for predicting drug effects demonstrated that most such methods, including neighborhood regularized logistic matrix factorization, which was the best performer in benchmark tests, used a binary matrix that considers only the presence or absence of interactions. However, drug effects are known to have two opposite aspects, such as side effects and therapeutic effects. In the present study, we proposed using neighborhood regularized bidirectional matrix factorization (NRBdMF) to predict drug effects by incorporating bidirectionality, which is a characteristic property of drug effects. We used this proposed method for predicting side effects using a matrix that considered the bidirectionality of drug effects, in which known side effects were assigned a positive label (plus 1) and known treatment effects were assigned a negative (minus 1) label. The NRBdMF model, which utilizes drug bidirectional information, achieved enrichment of side effects at the top and indications at the bottom of the prediction list. This first attempt to consider the bidirectional nature of drug effects using NRBdMF showed that it reduced false positives and produced a highly interpretable output.

q-bio.QM↗

arXiv2022

Investigation of a Data Split Strategy Involving the Time Axis in Adverse Event Prediction Using Machine Learning

Adverse events are a serious issue in drug development and many prediction methods using machine learning have been developed. The random split cross-validation is the de facto standard for model building and evaluation in machine learning, but care should be taken in adverse event prediction because this approach does not match to the real-world situation. The time split, which uses the time axis, is considered suitable for real-world prediction. However, the differences in model performance obtained using the time and random splits are not clear due to the lack of the comparable studies. To understand the differences, we compared the model performance between the time and random splits using nine types of compound information as input, eight adverse events as targets, and six machine learning algorithms. The random split showed higher area under the curve values than did the time split for six of eight targets. The chemical spaces of the training and test datasets of the time split were similar, suggesting that the concept of applicability domain is insufficient to explain the differences derived from the splitting. The area under the curve differences were smaller for the protein interaction than for the other datasets. Subsequent detailed analyses suggested the danger of confounding in the use of knowledge-based information in the time split. These findings indicate the importance of understanding the differences between the time and random splits in adverse event prediction and strongly suggest that appropriate use of the splitting strategies and interpretation of results are necessary for the real-world prediction of adverse events. We provide analysis code and datasets used in the present study (https://github.com/mizuno-group/AE_prediction).

cs.LG↗

arXiv2021

Extraction of diverse gene groups with individual relationship from gene co-expression networks

Motivation: Modules in gene coexpression networks (GCN) can be regarded as gene groups with individual relationships. No studies have optimized module detection methods to extract diverse gene groups from GCN, especially for data from clinical specimens. Results: Here, we optimized the flow from transcriptome data to gene modules, aiming to cover diverse gene relationships. We found the prediction accuracy of relationships in benchmark networks of non-mammalian was not always suitable for evaluating gene relationships of human and employed network based metrics. We also proposed a module detection method involving a combination of graphical embedding and recursive partitioning, and confirmed its stable and high performance in biological plausibility of gene groupings. Analysis of differentially ex-pressed genes of several reported cancers using the extracted modules successfully added relational information consistent with previous reports, confirming the usefulness of our framework.

q-bio.MN↗