Searcharxiv⌕ Search

arXiv subjects

Isshu Lee

Publications and source records attributed to Isshu Lee.

2 recordsLinked to original sources

NEBench R1: a Reasoning Benchmark for Evaluating Nuclear Domain Competence in Large Language Models

Large language models (LLMs) are increasingly being explored for knowledge-intensive and safety-critical applications in nuclear engineering. However, existing nuclear-domain benchmarks provide limited content coverage or focus primarily on answer accuracy without systematically evaluating the quality of the underlying reasoning. This study introduces NEBench R1, a reasoning-oriented benchmark comprising 7,546 unique multiple-choice questions extracted from U.S. Nuclear Regulatory Commission (NRC) reactor operator licensing examinations covering 18 nuclear facilities. Each question is classified according to the NRC Knowledge and Abilities catalogs and accompanied by an official explanation serving as a reference for reasoning evaluation. Six LLMs were evaluated, with overall accuracy ranging from 34.0% to 73.9% across three distinct performance tiers. To assess reasoning quality, an LLM-based judge evaluated correctly answered responses using a multidimensional rubric. Although responses generally demonstrated strong problem comprehension, they scored lower in factual and procedural accuracy and distractor analysis, indicating that correct answers were not necessarily supported by technically sound justifications. Response clarity was also weakly indicative of technical quality. Furthermore, longer reasoning traces were associated with lower accuracy in both frontier reasoning models, suggesting that reasoning length may reflect question difficulty rather than improved problem-solving performance. Reasoning-token consumption may therefore provide an additional signal for identifying potentially unreliable responses. NEBench R1 establishes a benchmark for evaluating both answer accuracy and reasoning quality, providing a foundation for assessing the domain competence and reliability of LLMs in nuclear engineering.

cs.AI↗

Machine Learning-based Correlation of Charpy Impact Properties Between Sub-sized and Standard-sized Specimens for Nuclear Structural Materials

Reliable correlations of Charpy impact test results between sub-sized and full-sized specimens are essential for structural integrity assessments, particularly in nuclear applications, where spatial constraints and limited material volume restrict specimen size. Although standards such as ASTM A370 and BS 7910 provide guidance on conversion methodologies, and numerous analytical correlation methods have been proposed in prior studies, these approaches generally have limited accuracy and their applicability is often constrained to specific materials, treatment conditions, and specimen geometries. In this study, a Machine Learning (ML)-based framework is proposed for correlating Charpy impact properties across specimen sizes. The proposed approach maps absorbed energy values across the full ductile-to-brittle transition region by applying a temperature shift combined with scaled residual projection, to align sub-sized test data with full-sized response. From the resulting temperature-energy profiles, the correlated values for upper shelf energy (USE) and ductile-to-brittle transition temperature (DBTT) are extracted by fitting data with a hyperbolic tangent model. The framework is validated using a dataset comprising 389 matched sub-sized and full-sized Charpy impact tests on SA533B steel. This ML-based approach demonstrates an improved correlation performance relative to conventional analytical methods, achieving R2 values of 0.942 for USE and 0.892 for DBTT. The trained ML models do not require access to full-sized Charpy data during inference, making this approach suitable for material surveillance programs, accelerated irradiation testing, and other applications involving small-size Charpy impact testing.

cs.LG↗