arXiv subjects

Alonso Rodriguez-Navarro

Publications and source records attributed to Alonso Rodriguez-Navarro.

13 recordsLinked to original sources

arXiv2026

Sum of rank ratios: an alternative to percentiles for research assessment, from groundbreaking to mainstream research

Assessing research that pushes the boundaries of knowledge is challenging because such work is extremely infrequent, accounting for only about 0.01 per cent of all research outputs. Consequently, knowledge about how to evaluate this type of research is far more limited than the well established methods used to assess more common research outcomes. This study addresses this gap by using a rank based approach in which each paper is assigned a unique value equal to the ratio between its local and global ranks. The cumulative value of these ratios, starting from the most cited paper, provides the evaluative basis, and the Rn index described here, using 10 rank ratios, appears to be the best option. Although research assessment based on global ranks was originally developed to evaluate the largest contributors to groundbreaking knowledge, namely, the USA and China, which account for most of the most cited papers, the Rn index has broader applications. This study demonstrates that it is also a better option than the number of top 10 per cent or top 1per cent highly cited papers, which are the most common indicators used to evaluate countries that seldom or never produce cutting-edge research that pushes the boundaries of knowledge. In all cases, the Rn index reflects the highest quality science produced by each country. Furthermore, the Rn index can be easily calculated without specialized training in bibliometrics and is insignificantly affected by ties in citation counts.

cs.DL↗

arXiv2024

The Rn-index: a more accurate variant of the Rk-index

The contribution to pushing the boundaries of knowledge is a critical metric for evaluating the research performance of countries and institutions, which in many cases is not revealed by common bibliometric indicators. The Rk-index was specifically designed to assess such contributions, and the Rn-index is a variant that corrects the weakness of the Rk-index, particularly in the evaluation of countries that produce a high proportion of global advancements. This is the case of the USA and China in many technological fields. Additionally, the Rn-index is simple to calculate and understand, as it involves only summing the ratios between the local and global ranks of papers, ordered by their citation count. Moreover, the Rn-index may also be fractionally counted.

cs.DL↗

arXiv2024

Countries pushing the boundaries of knowledge: the US dominance, China rise, and the EU stagnation

Knowing which countries contribute the most to pushing the boundaries of knowledge in science and technology has social and political importance. However, common citation metrics do not adequately measure this contribution. This measure requires more stringent metrics appropriate for the highly influential breakthrough papers that push the boundaries of knowledge, which are very highly cited but very rare. Here I used the recently described Rk index, specifically designed to address this issue. I applied this index to 25 countries and the EU across 10 key research topics, five technological and five biomedical, studying domestic and international collaborative papers independently. In technological topics, the Rk indices of domestic papers show that overall, the USA, China, and the EU are leaders; other countries are clearly behind. The USA is notably ahead of China, and the EU is far behind China. The same approach to biomedical topics shows an overwhelming dominance of the USA and that the EU is ahead of China. The analysis of internationally collaborative papers further demonstrates the US dominance. These results conflict with current country rankings based on less stringent indicators.

cs.DL↗

arXiv2023

Uncertain research country rankings. Should we continue producing uncertain rankings?

Purpose: Citation-based assessments of countries' research capabilities often misrepresent their ability to achieve breakthrough advancements. These assessments commonly classify Japan as a developing country, which contradicts its prominent scientific standing. The purpose of this study is to investigate the underlying causes of such inaccurate assessments and to propose methods for conducting more reliable evaluations. Design/methodology/approach: The study evaluates the effectiveness of top-percentile citation metrics as indicators of breakthrough research. Using case studies of selected countries and research topics, the study examines how deviations from lognormal citation distributions impact the accuracy of these percentile indicators. A similar analysis is conducted using university data from the Leiden Ranking to investigate citation distribution deviations at the institutional level. Findings: The study finds that inflated lower tails in citation distributions lead to undervaluation of research capabilities in advanced technological countries, as captured by some percentile indicators. Conversely, research-intensive universities exhibit the opposite trend: a reduced lower tail relative to the upper tail, which causes percentile indicators to overestimate their actual research capacity. Research limitations: The descriptions are mathematical facts that are self-evident. Practical implications: Due to variations in citation patterns across countries and institutions, the Ptop 10%/P and Ptop 1%/P ratios are not universal predictors of breakthrough research. Evaluations should move away from these metrics. Relying on inappropriate citation-based measures could lead to poor decision-making in research policy, undermining the effectiveness of research strategies and their outcomes.

cs.DL↗

arXiv2023

Rank analysis of most cited publications, a new approach for research assessments

Citation metrics are the best tools for research assessments. However, current metrics may be misleading in research systems that pursue simultaneously different goals, such as the advance of science and incremental innovations, because their publications have different citation distributions. We estimate the contribution to the progress of knowledge by studying only a limited number of the most cited papers, which are dominated by publications pursuing this progress. To field-normalize the metrics, we substitute the number of citations by the rank position of papers from one country in the global list of papers. Using synthetic series of lognormally distributed numbers, we developed the Rk-index, which is calculated from the global ranks of the 10 highest numbers in each series, and demonstrate its equivalence to the number of papers in top percentiles, P_top0.1% and P_top0.01% . In real cases, the Rk-index is simple and easy to calculate, and evaluates the contribution to the progress of knowledge better than less stringent metrics. Although further research is needed, rank analysis of the most cited papers is a promising approach for research evaluation. It is also demonstrated that, for this purpose, domestic and collaborative papers should be studied independently.

cs.DL↗

arXiv2023

The extreme upper tail of Japan's citation distribution reveals its research success

A number of indications, such as the number of Nobel Prize winners, show Japan to be a scientifically advanced country. However, standard bibliometric indicators place Japan as a scientifically developing country. The present study is based on the conjecture that scientific publications from Japan belong to two different populations: one originates from studies that advance science and includes highly cited papers, while the other is formed by poorly cited papers with almost zero probability of being highly cited. Although these two categories of papers cannot be easily identified and separated, the scientific level of Japan can be tested by studying the extreme upper tail of the citation distribution of all scientific articles. In contrast to standard bibliometric indicators, which are calculated from the total number of papers or from sets of papers in which the two categories of papers are mixed, in the extreme upper tail, only papers that are addressed to the advance of science will be present. Based on the extreme upper tail, Japan belongs to the group of scientifically advanced countries and is significantly different from countries with a low scientific level. The number of Clarivate Citation laureates also supports our hypothesis that some citation-based metrics do not reveal the high scientific level of Japan. Our findings suggest that Japan is an extreme case of inaccuracy of some citation metrics; the same drawback might affect other countries, although to a lesser degree.

cs.DL↗

arXiv2022

Common bibliometric approaches fail to assess correctly the number of important scientific advances for most countries and institutions

Although not explicitly declared, most research rankings of countries and institutions are supposed to reveal their contribution to the advancement of knowledge. However, such advances are based on very highly cited publications with very low frequency, which can only very exceptionally be counted with statistical reliability. Percentile indicators enable calculations of the probability or frequency of such rare publications using counts of much more frequent publications; the general rule is that rankings based on the number of top 10% or 1% cited publications (Ptop 10%, Ptop 1%) will also be valid for the rare publications that push the boundaries of knowledge. Japan and its universities are exceptions, as their frequent Nobel Prizes contradicts their low Ptop 10% and Ptop 1%. We explain that this occurs because, in single research fields, the singularity of percentile indicators holds only for research groups that are homogeneous in their aims and efficiency. Correct calculations for ranking countries and institutions should add the results of their homogeneous groups, instead of considering all publications as a single set. Although based on Japan, our findings have a general character. Common predictions of scientific advances based on Ptop 10% might be severalfold lower than correct calculations.

cs.DL↗

arXiv2021

The link between countries' economic and scientific wealth has a complex dependence on technological activity and research policy

We studied the research performance of 69 countries by considering two different types of new knowledge: incremental (normal) and fundamental (radical). In principle, these two types of new knowledge should be assessed at two very different levels of citations, but we demonstrate that a simpler assessment can be performed based on the total number of papers (P) and the ratio of the number of papers in the global top 10% of most cited papers divided to the total number of papers (Ptop 10%/P). P represents the quantity, whereas the Ptop 10%/P ratio represents the efficiency. In ideal countries, P and the Ptop 10%/P ratio are linked to the gross domestic product (GDP) and GDP the per capita, respectively. Only countries with high Ptop 10%/P ratios participate actively in the creation of fundamental new knowledge and have Noble laureates. In real countries, the link between economic and scientific wealth can be modified by the technological activity and the research policy. We discuss how technological activity may decrease the Ptop 10%/P ratio while only slightly affecting the capacity to create fundamental new knowledge; in such countries, many papers may report incremental innovations that do not drive the advancement of knowledge. Japan is the clearest example of this, although there are many less extreme examples. Independently of technological activity, research policy has a strong influence on the Ptop 10%/P ratio, which may be higher or lower than expected from the GDP per capita depending on the success of the research policy.

cs.DL↗

arXiv2021

Validated research assessment based on highly cited researchers

Bibliometrics provides accurate, cheap and simple descriptions of research systems and should lay the foundations for research policy. However, disconnections between bibliometric knowledge and research policy frequently misguide the research policy in many countries. A way of correcting these disconnections might come from the use of simple indicators of research performance. One such simple indicator is the number of highly cited researchers, which can be used under the assumption that a research system that produces and employs many highly cited researchers will be more successful than others with fewer of them. Here, we validate the use of the number of highly cited researchers (Ioannidis et al. 2020; PLoS Biol 18(10): e3000918) for research assessment at the country level and determine a country ranking of research success. We also demonstrate that the number of highly cited researchers reported by Clarivate Analytics is also an indicator of the research success of countries. The formal difference between the numbers of highly cited researchers according to Ionannidis et al. and Clarivate Analytics is that evaluations based on these two lists of highly cited researchers are approximately equivalent to evaluations based on the top 5% and 0.05% of highly cited papers, respectively. Moreover, the Clarivate Analytics indicator is flawed in some countries.

cs.DL↗

arXiv2021

La evaluacion de la investigacion: Espana suspende

The research assessments of countries or institutions should reveal their contribution to the advancement of science. Taking into consideration the correlation that exists between scientific impact and number of citations, research assessments can be based on citation counts. However, it is crucial to perform the counts in the heavy tail of citation distributions. Following this heavy tail approach, all measurements performed in Spanish universities show that their research is inefficient because the proportion of highly cited papers is much lower than in the most scientifically advanced countries. The same conclusion is reached when considering the number of researchers in the Ioannidis, Boyack, and Baas lists of highly cited researchers.

cs.DL↗

arXiv2018

Probability and expected frequency of breakthroughs - a robust method of research assessment based on the double rank property of citation distributions

In research policy, effective measures that lead to improvements in the generation of knowledge must be based on reliable methods of research assessment, but for many countries and institutions this is not the case. Publication and citation analyses can be used to estimate the part played by countries and institutions in the global progress of knowledge, but a concrete method of estimation is far from evident. The challenge arises because publications that report real progress of knowledge form an extremely low proportion of all publications; in most countries and institutions such contributions appear less than once per year. One way to overcome this difficulty is to calculate probabilities instead of counting the rare events on which scientific progress is based. This study reviews and summarizes several recent publications, and adds new results that demonstrate that the citation distribution of normal publications allows the probability of the infrequent events that support the progress of knowledge to be calculated.

cs.DL↗

arXiv2018

Technological research in the EU is less efficient than the world average. EU research policy risks Europeans' future

We have studied the efficiency of research in the EU by a percentile-based citation approach that analyzes the distribution of country papers among the world papers. Going up in the citation scale, the frequency of papers from efficient countries increases while the frequency from inefficient countries decreases. In the percentile-based approach, this trend, which is permanent at any citation level, is measured by the ep index that equals the Ptop 1%/Ptop 10% ratio. By using the ep index we demonstrate that EU research on fast-evolving technological topics is less efficient than the world average and that the EU is far from being able to compete with the most advanced countries. The ep index also shows that the USA is well ahead of the EU in both fast- and slow-evolving technologies, which suggests that the advantage of the USA over the EU in innovation is due to low research efficiency in the EU. In accord with some previous studies, our results show that the European Commission's ongoing claims about the excellence of EU research are based on a wrong diagnosis. The EU must focus its research policy on the improvement of its inefficient research. Otherwise, the future of Europeans is at risk.

cs.DL↗

arXiv2018

Research assessment by percentile-based double rank analysis

In the double rank analysis of research publications, the local rank position of a country or institution publication is expressed as a function of the world rank position. Excluding some highly or lowly cited publications, the double rank plot fits well with a power law, which can be explained because citations for local and world publications follow lognormal distributions. We report here that the distribution of the number of country or institution publications in world percentiles is a double rank distribution that can be fitted to a power law. Only the data points in high percentiles deviate from it when the local and world $μ$ parameters of the lognormal distributions are very different. The likelihood of publishing very highly cited papers can be calculated from the power law that can be fitted either to the upper tail of the citation distribution or to the percentile-based double rank distribution. The great advantage of the latter method is that it has universal application, because it is based on all publications and not just on highly cited publications. Furthermore, this method extends the application of the well-established percentile approach to very low percentiles where breakthroughs are reported but paper counts cannot be performed.

cs.DL↗