arXiv subjects

Kan Hatakeyama-Sato

Publications and source records attributed to Kan Hatakeyama-Sato.

7 recordsLinked to original sources

arXiv2026

JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation

We introduce JMed48k, a multi-profession Japanese healthcare licensing benchmark for evaluating vision-language models. Built from official PDF materials released by the Japanese Ministry of Health, Labour and Welfare, JMed48k contains 48,862 exam questions and 20,142 images from 11 national licensing examinations between 2005 and 2025, with visual content annotated under an 8-type taxonomy. From this corpus, we derive JMed48k-Eval, a recent five-year evaluation subset with 12,484 scored questions, including 9,905 text-only questions and 2,579 questions with images. We evaluate 21 proprietary, open-source, and medical-specific models, reporting text-only and with-image performance separately. Because these subsets contain different questions, we further introduce a paired image-removal audit that evaluates questions with images before and after removing visual content to explore four answer-transition states. The audit shows that proprietary and open source models gain substantially from images, whereas medical-specific systems show limited observable use of visual evidence, with many correct answers persisting after image removal. Even among proprietary models, the net image-removal effect varies sevenfold across professions, from +5.7 points on Physician questions to +39.8 points on Public Health Nurse questions. We release JMed48k to support reproducible, profession-stratified evaluation of vision-language models in medical licensing settings.

cs.CV↗

arXiv2026

JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models

As Large Language Models (LLMs) are increasingly deployed in healthcare field, it becomes essential to carefully evaluate their medical safety before clinical use. However, existing safety benchmarks remain predominantly English-centric, and test with only single-turn prompts despite multi-turn clinical consultations. To address these gaps, we introduce JMedEthicBench, the first multi-turn conversational benchmark for evaluating medical safety of LLMs for Japanese healthcare. Our benchmark is based on 67 guidelines from the Japan Medical Association and contains over 50,000 adversarial conversations generated using seven automatically discovered jailbreak strategies. Using a dual-LLM scoring protocol, we evaluate 27 models and find that commercial models maintain robust safety while medical-specialized models exhibit increased vulnerability. Furthermore, safety scores decline significantly across conversation turns (median: 9.5 to 5.0, $p < 0.001$). Cross-lingual evaluation on both Japanese and English versions of our benchmark reveals that medical model vulnerabilities persist across languages, indicating inherent alignment limitations rather than language-specific factors. These findings suggest that domain-specific fine-tuning may accidentally weaken safety mechanisms and that multi-turn interactions represent a distinct threat surface requiring dedicated alignment strategies.

cs.CL↗

arXiv2025

Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Research

This review explores the potential of foundation models to advance laboratory automation in the materials and chemical sciences. It emphasizes the dual roles of these models: cognitive functions for experimental planning and data analysis, and physical functions for hardware operations. While traditional laboratory automation has relied heavily on specialized, rigid systems, foundation models offer adaptability through their general-purpose intelligence and multimodal capabilities. Recent advancements have demonstrated the feasibility of using large language models (LLMs) and multimodal robotic systems to handle complex and dynamic laboratory tasks. However, significant challenges remain, including precision manipulation of hardware, integration of multimodal data, and ensuring operational safety. This paper outlines a roadmap highlighting future directions, advocating for close interdisciplinary collaboration, benchmark establishment, and strategic human-AI integration to realize fully autonomous experimental laboratories.

cs.RO↗

arXiv2024

SPACIER: On-Demand Polymer Design with Fully Automated All-Atom Classical Molecular Dynamics Integrated into Machine Learning Pipelines

Machine learning has rapidly advanced the design and discovery of new materials with targeted applications in various systems. First-principles calculations and other computer experiments have been integrated into material design pipelines to address the lack of experimental data and the limitations of interpolative machine learning predictors. However, the enormous computational costs and technical challenges of automating computer experiments for polymeric materials have limited the availability of open-source automated polymer design systems that integrate molecular simulations and machine learning. We developed SPACIER, an open-source software program that integrates RadonPy, a Python library for fully automated polymer property calculations based on all-atom classical molecular dynamics into a Bayesian optimization-based polymer design system to overcome these challenges. As a proof-of-concept study, we successfully synthesized optical polymers that surpass the Pareto boundary formed by the tradeoff between the refractive index and Abbe number.

cond-mat.mtrl-sci↗

arXiv2023

Teaching Specific Scientific Knowledge into Large Language Models through Additional Training

Through additional training, we explore embedding specialized scientific knowledge into the Llama 2 Large Language Model (LLM). Key findings reveal that effective knowledge integration requires reading texts from multiple perspectives, especially in instructional formats. We utilize text augmentation to tackle the scarcity of specialized texts, including style conversions and translations. Hyperparameter optimization proves crucial, with different size models (7b, 13b, and 70b) reasonably undergoing additional training. Validating our methods, we construct a dataset of 65,000 scientific papers. Although we have succeeded in partially embedding knowledge, the study highlights the complexities and limitations of incorporating specialized information into LLMs, suggesting areas for further improvement.

cs.CL↗

arXiv2021

A Generative Model for Extrapolation Prediction in Materials Informatics

We report a deep generative model for regression tasks in materials informatics. The model is introduced as a component of a data imputer, and predicts more than 20 diverse experimental properties of organic molecules. The imputer is designed to predict material properties by "imagining" the missing data in the database, enabling the use of incomplete material data. Even removing 60% of the data does not diminish the prediction accuracy in a model task. Moreover, the model excels at extrapolation prediction, where target values of the test data are out of the range of the training data. Such extrapolation has been regarded as an essential technique for exploring novel materials, but has hardly been studied to date due to its difficulty. We demonstrate that the prediction performance can be improved by >30% by using the imputer compared with traditional linear regression and boosting models. The benefit becomes especially pronounced with few records for an experimental property (< 100 cases) when prediction would be difficult by conventional methods. The presented approach can be used to more efficiently explore functional materials and break through previous performance limits.

physics.comp-ph↗

arXiv2020

Tackling the challenge of a huge materials science search space with quantum-inspired annealing

Efficient screening of chemicals is essential for exploring new materials. However, the search space is astronomically large, making calculations with conventional computers infeasible. For example, an $N$-component system of organic molecules generates >$10^{60N}$ candidates. Here, a quantum-inspired annealing machine is used to tackle the challenge of the large search space. The prototype system extracts candidate chemicals and their composites with desirable parameters, such as melting temperature and ionic conductivity. The system can be at least $10^4$-$10^7$ times faster than conventional approaches. Such exponential acceleration is critical for exploring the enormous search space in virtual screening.

physics.comp-ph↗