SearcharxivSearch

arXiv subjects

Jiyuan Ye

Publications and source records attributed to Jiyuan Ye.

2 recordsLinked to original sources

Ontology-Guided Multi-Agent Extraction of Evaluation Objects from Academic Review Texts: Evidence from Chinese Library and Information Science

Academic reviews, scholarly commentaries, and book reviews serve as sources of evaluative statements about theories, methods, literature, institutions, and policies, providing valuable evidence for scholarly evaluation. Existing scientific entity extraction methods mainly target research articles and are less effective for evaluation objects, which are often abstract, context-dependent, and characterized by ambiguous type boundaries. This study proposes an ontology-guided multi-agent framework for evaluation object extraction. The framework combines candidate discovery, ontology-constrained classification, and domain review. Experimental results show that it achieves a Precision of 90.33%, Recall of 84.55%, Entity-level F1 of 87.34%, Strict Typed F1 of 79.78%, and Type Accuracy of 91.35%, substantially outperforming rule-based and zero-shot baselines. Ablation results indicate that the multi-agent workflow improves recall and stability, while ontology-based boundary constraints enhance fine-grained classification and reduce category confusion. The framework supports the structured utilization of evaluative scholarly texts and provides methodological support for evidence-based research evaluation and STI mining.

cs.CL

Beyond the Binary: The System of All-round Evaluation of Research and Its Practices in China

The lack of a macro-level, systematic evaluation theory to guide the implementation of evaluation practices has become a key bottleneck in the reform of global research evaluation systems. By reviewing the historical development of research evaluation, this paper highlights the current binary opposition between qualitative and quantitative methods in evaluation practices. This paper introduces the System of All-round Evaluation of Research (SAER), a framework that integrates form, content, and utility evaluations with six key elements. SAER offers a theoretical breakthrough by transcending the binary, providing a comprehensive foundation for global evaluation reforms. The comprehensive system proposes a trinity of three evaluation dimensions, combined with six evaluation elements, which would help academic evaluators and researchers reconcile binary oppositions in evaluation methods. The system highlights the dialectical wisdom and experience embedded in Chinese research evaluation theory, offering valuable insights and references for the reform and advancement of global research evaluation systems.

cs.SE