Hitoshi Koshiba | Searcharxiv

arXiv2026

Analysis of Potential Generative AI Use in Abstracts of KAKENHI-Funded Projects

This study analyzes the extent to which abstracts of projects funded under the Scientific Research (C) category of the Grants-in-Aid for Scientific Research (KAKENHI) were classified as AI-generated. The analysis covers projects funded over the five-year period from FY2022 to FY2026. The number of abstracts classified as AI-generated began to increase in FY2025, and approximately 20% of the abstracts were classified as AI-generated in FY2026. Although the proportions varied to some extent, abstracts classified as AI-generated were observed in many review categories. These findings indicate that the use of generative AI has begun to spread across many research fields represented in Scientific Research (C).

cs.DL↗

arXiv2024

Exploring the applicability of Large Language Models to citation context analysis

Unlike traditional citation analysis -- which assumes that all citations in a paper are equivalent -- citation context analysis considers the contextual information of individual citations. However, citation context analysis requires creating large amounts of data through annotation, which hinders the widespread use of this methodology. This study explored the applicability of Large Language Models (LLMs) -- particularly ChatGPT -- to citation context analysis by comparing LLMs and human annotation results. The results show that the LLMs annotation is as good as or better than the human annotation in terms of consistency but poor in terms of predictive performance. Thus, having LLMs immediately replace human annotators in citation context analysis is inappropriate. However, the annotation results obtained by LLMs can be used as reference information when narrowing the annotation results obtained by multiple human annotators to one, or LLMs can be used as one of the annotators when it is difficult to prepare sufficient human annotators. This study provides basic findings important for the future development of citation context analyses.

cs.DL↗