Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.23318

On some unresolved issues in aggregate level comparative research evaluation

Paul Donner
View original source ↗Read source document ↗Export metadata ↓

Abstract

This short conceptual paper recalls to the attention of the scientometrics community some long-standing open methodological problems and proposes new directions that might be taken in renewed engagements with them. These problems are the differential dependency of the global reference score on the included units' observed scores and the dilution of impact measurement accuracy due to collaboration, which causes a global convergence in country citation impact due to international collaborative research. Advantages and disadvantages of existing and new solutions are discussed.

Explore related subjects

cs.DL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Paul Donner. 2026-10-16. On some unresolved issues in aggregate level comparative research evaluation. https://arxiv.org/abs/2610.23318

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Multiplex coupling: Connecting the intellectual base and the topics of a research field

Two documents can be close because they cite the same works or because they write about the same things, and the two kinds of proximity often disagree. Science mapping reads them in separate maps or merges them, as hybrid clustering does; either way the disagreement is lost. We introduce multiplex coupling, a framework that couples the same documents by their references and by their terms, keeps the two layers separate and analyses their relation. Similarities are standardised against a null model, and the agreement between the layers is measured on random pairs of documents. Consensus clustering partitions each layer into roots, groups sharing an intellectual base, and themes, groups sharing a topic. A test on the roots $\times$ themes table identifies branching roots and convergent themes, and pairs of roots are followed over time. Every structure is reported with its persistence across numbers of neighbours, which on synthetic data identifies most false branching. Applications to Web of Science, Scopus and OpenAlex collections show that the two layers carry largely complementary information, and reveal substantive and methodological bases that have grown into several topics. The framework is implemented in bibliometrix.

cs.DL↗
arXiv2026

Faster alone vs. further together: The speed and impact of researchers' responses to societal crises

Throughout the 21st century, society encountered numerous crises that demanded quick and widespread action. In the present work, we documented researchers' responses to 25 of these crises, analyzing 1,537,284 published responses from 3,262,152 researchers spanning 225 countries and territories. An analysis of publication speed and citation patterns yielded support for a proverbial claim: "to go fast, go alone; to go far, go together". Solo authors were quicker to publish responses to societal crises -- but their responses received less attention in scholarly, news, and policy venues. Growth towards large-scale collaboration was associated with increased impact; However, this team size advantage was characterized by diminishing returns and significantly varied across sampled crises. Furthermore, pluralistic machine learning modeling highlights the importance of considering additional context, such as government effectiveness and the expertise of team leaders. Taken together, results facilitate evidence-based conversations regarding how researchers optimize speed and impact in the midst of societal crises.

cs.DL↗
arXiv2026

General Classification for Digital Knowledge: Algorithm, Identifiers and a First Benchmark

Digital libraries and knowledge graphs need a way to organize very large collections so that related works are close and any work is quick to reach. The classifications libraries inherited for this, such as LCC and DDC, are maintained by editorial staff, are partly closed, and were not built against a stated, measurable objective. We propose the opposite: a general classification that is fitted, under the constraints that make a classification usable on a shelf or in a browse list, to the same quantity that a companion survey and benchmark (Awad, 2026) ranks existing schemes by. The objective is J = D + 0.005 N, where D is the normalized Dasgupta cost of the class tree on a fixed similarity graph (Dasgupta, 2016) and N is an expected navigation cost in bits with a per-level charge. Two constraints are hard: every class is a contiguous run of one fixed shelf order (the optimal-alphabetic-tree constraint of Hu and Tucker, 1971), and new works are filed by splitting leaf classes only, so that every existing class keeps its code, label and parent. On a corpus of 10,000 arXiv papers the fitted prototype has 110 hand-labelled classes and reaches cost 0.1722 in sample (average linkage 0.1518; arXiv's own categories 0.3333; the best existing scheme reached through the same lexical classifier 0.6414, from the survey). Filing 500 library books as new works created 16 classes and changed no existing identifier. Fit, filing and scoring share one embedding: the held-out skill of 0.922 falls to 0.896 when the same fit is scored on the graphs of the two other models, and on books the average falls from 0.806 to 0.731. The scheme has also seen only arXiv, and labels for inserted classes are weak. We state those limits plainly, release the scheme as CC0 open data, and ask whether a learned, open, order-preserving classification can become a general one.

cs.DL↗