SearcharxivSearch

arXiv subjects

Manolis Antonoyiannakis

Publications and source records attributed to Manolis Antonoyiannakis.

9 recordsLinked to original sources

$Φ$ index: A standardized scale-independent and field-normalized citation indicator

The Impact Factor (IF), despite its widespread use, suffers from well-known biases that remain incompletely addressed in practice -- most notably its sensitivity to journal size and its lack of field normalization. Because of size sensitivity, a randomly formed journal of $n$ papers can attain a range of IF values that decreases sharply with size, as $\sim 1/\sqrt{n}$. The Central Limit Theorem, which underlies this effect, also allows us to correct for it by standardizing citation averages for scale and field in a manner analogous to calculating the $z$-score in statistics. We thus introduce the $Φ$ (Phi) index, defined as $Φ= (f - μ)\sqrt{n}/σ$, where $f$ is a journal's average citation count (akin to the IF), $n$ its publication count, and $μ, σ$ the mean and standard deviation of citations in its field. Applying the $Φ$ index to 12,173 journals in Clarivate's Journal Citation Reports, we obtain rankings that correct for size bias and elevate journals from underrepresented fields such as mathematics, law, and history. We validate the $Φ$ index via a Monte Carlo random sample test, which we propose as a standard diagnostic for any citation indicator. The methodology extends readily to departments, universities, and countries.

cs.DL

Global method for gender profile estimation from distribution of first names

As social issues related to gender bias attract closer scrutiny, accurate tools to determine the gender profile of large groups become essential. When explicit data is unavailable, gender is often inferred from names. Current methods follow a strategy whereby individuals of the group, one by one, are assigned a gender label or probability based on gender-name correlations observed in the population at large. We show that this strategy is logically inconsistent and has practical shortcomings, the most notable of which is the systematic underestimation of gender bias. We introduce a global inference strategy that estimates gender composition according to the context of the full list of names. The tool suffers from no intrinsic methodological effects, is robust against errors, easily implemented, and computationally light.

stat.AP

Does publicity in the science press drive citations? A vindication of peer review

We study how publicity, in the form of highlighting in the science press, affects the citations of research papers. After a brief review of prior work, we analyze papers published in Physical Review Letters (PRL) that are highlighted across eight different platforms. Using multiple linear regression we identify how each platform contributes to citations. We also analyze how frequently the highlighted papers end up in the top 1% cited papers in their field. We find that the strongest predictors of medium-term citation impact -- up to 7 years post-publication -- are Viewpoints in Physics, followed by Research Highlights in Nature, Editors' Suggestions in PRL, and Research Highlights in Nature Physics. Our key conclusions are that (a) highlighting for importance identifies a citation advantage, which (b) is stratified according to the degree of vetting during peer review (internal and external to the journal). This implies that we can view highlighting platforms as predictors of citation accrual, with varying degrees of strength that mirror each platform's vetting level.

cs.DL

Does Publicity in the Science Press Drive Citations?

We study how publicity in the science press, in the form of highlighting, affects the citations of research papers. Using multiple linear regression, we quantify the citation advantage associated with several highlighting platforms for papers published in Physical Review Letters (PRL) from 2008-2018. We thus find that the strongest predictor of citation accrual is a Viewpoint in Physics magazine, followed by a Research Highlight in Nature, an Editors' Suggestion in PRL, and a Research Highlight in Nature Physics. A similar hierarchical pattern is found when we search for extreme, not average, citation accrual, in the form a paper being listed among the top-1% cited papers in physics by Clarivate Analytics. The citation advantage of each highlighting platform is stratified according to the degree of vetting for importance that the manuscript received during peer review. This implies that we can view highlighting platforms as predictors of citation accrual, with varying degrees of strength that mirror each platform's vetting level.

cs.DL

Impact Factor volatility to a single paper: A comprehensive analysis

We study how a single paper affects the Impact Factor (IF) by analyzing data from 3,088,511 papers published in 11639 journals in the 2017 Journal Citation Reports of Clarivate Analytics. We find that IFs are highly volatile. For example, the top-cited paper of 381 journals caused their IF to increase by more than 0.5 points, while for 818 journals the relative increase exceeded 25%. And one in 10 journals had their IF boosted by more than 50% by their top three cited papers. Because the single-paper effect on the IF is inversely proportional to journal size, small journals are rewarded much more strongly than large journals for a highly-cited paper, while they are penalized more for a low-cited paper, especially if their IF is high. This skewed reward mechanism incentivizes high-IF journals to stay small, to remain competitive in rankings. We discuss the implications for breakthrough papers appearing in prestigious journals. We question the reliability of IF rankings given the high IF sensitivity to a few papers for thousands of journals.

cs.DL

How a Single Paper Affects the Impact Factor: Implications for Scholarly Publishing

Because the Impact Factor (IF) is an average quantity and most journals are small, IFs are volatile. We study how a single paper affects the IF using data from 11639 journals in the 2017 Journal Citation Reports. We define as volatility the IF gain (or loss) caused by a single paper, and this is inversely proportional to journal size. We find high volatilities for hundreds of journals annually due to their top-cited paper: whether it is a highly-cited paper in a small journal, or a moderately (or even low) cited paper in a small and low-cited journal. For example, 1218 journals had their most cited paper boost their IF by more than 20%, while for 231 journals the boost exceeded 50%. We find that small journals are rewarded much more than large journals for publishing a highly-cited paper, and are also penalized more for publishing a low-cited paper, especially if they have a high IF. This produces a strong incentive for prestigious, high-IF journals to stay small, to remain competitive in IF rankings. We discuss the implications for breakthrough papers to appear in prestigious journals. We also question the practice of ranking journals by IF given this uneven reward mechanism.

cs.DL

Impact Factors and the Central Limit Theorem: Why Citation Averages Are Scale Dependent

Citation averages, and Impact Factors (IFs) in particular, are sensitive to sample size. We apply the Central Limit Theorem (CLT) to IFs to understand their scale-dependent behavior. For a journal of $n$ randomly selected papers from a population of all papers, we expect from the CLT that its IF fluctuates around the population average $μ$, and spans a range of values proportional to $σ/\sqrt[]{n}$, where $σ^2$ is the variance of the population's citation distribution. The $1/\sqrt[]{n}$ dependence has profound implications for IF rankings: The larger a journal, the narrower the range around $μ$ where its IF lies. IF rankings therefore allocate an unfair advantage to smaller journals in the high IF ranks, and to larger journals in the low IF ranks. We expect a scale-dependent stratification of journals in IF rankings, whereby small journals occupy top, middle, and bottom ranks; mid-sized journals occupy middle ranks; and very large journals have IFs that asymptotically approach $μ$. We confirm these predictions by analyzing (i) 166,498 IF \& journal-size data pairs in the 1997--2016 Journal Citation Reports of Clarivate Analytics, (ii) the top-cited portion of 276,000 physics papers published in 2014--2015, and (iii) the citation distributions of an arbitrarily sampled list of physics journals. We conclude that the CLT is a good predictor of the IF range of actual journals, while sustained deviations from its predictions are a mark of true, non-random, citation impact. IF rankings are thus misleading unless one compares like-sized journals or adjusts for these effects. We propose the $Φ$ index, a rescaled IF adjusted for size, which can be generalized to account also for different citation practices across research fields. Our methodology applies also to citation averages used to compare research fields, university departments or countries in various rankings.

physics.soc-ph

Highlighting Impact and the Impact of Highlighting: PRB Editors' Suggestions

Associate Editor Manolis Antonoyiannakis discusses the highlighting, as Editors' Suggestions, of a small percentage of the papers published each week. We highlight papers primarily for their importance and impact in their respective fields, or because we find them particularly interesting or elegant. It turns out that the additional layer of scrutiny involved in the selection of papers as Editors' Suggestions is associated with a significantly elevated and sustained citation impact.

physics.soc-ph

Acceptance Rates in Physical Review Letters: No Seasonal Bias

Are editorial decisions biased? A recent discussion in Learned Publishing has focused on one aspect of potential bias in editorial decisions, namely seasonal (e.g., monthly) variations in acceptance rates of research journals. In this letter, we contribute to the discussion by analyzing data from Physical Review Letters (PRL), a journal published by the American Physical Society. We studied the 190,106 papers submitted to PRL from January 1990 until September 2012. No statistically significant variations were found in the monthly acceptance rates. We conclude that the time of year that the authors of a paper submit their work to PRL has no effect on the fate of the paper through the review process.

physics.soc-ph