arXiv subjects

Lutz Bornmann

Publications and source records attributed to Lutz Bornmann.

171 records · Page 10Linked to original sources

arXiv2011

Integrated Impact Indicators (I3) compared with Impact Factors (IFs): An alternative research design with policy implications

In bibliometrics, the association of "impact" with central-tendency statistics is mistaken. Impacts add up, and citation curves should therefore be integrated instead of averaged. For example, the journals MIS Quarterly and JASIST differ by a factor of two in terms of their respective impact factors (IF), but the journal with the lower IF has the higher impact. Using percentile ranks (e.g., top-1%, top-10%, etc.), an integrated impact indicator (I3) can be based on integration of the citation curves, but after normalization of the citation curves to the same scale. The results across document sets can be compared as percentages of the total impact of a reference set. Total number of citations, however, should not be used instead because the shape of the citation curves is then not appreciated. I3 can be applied to any document set and any citation window. The results of the integration (summation) are fully decomposable in terms of journals or instititutional units such as nations, universities, etc., because percentile ranks are determined at the paper level. In this study, we first compare I3 with IFs for the journals in two ISI Subject Categories ("Information Science & Library Science" and "Multidisciplinary Sciences"). The LIS set is additionally decomposed in terms of nations. Policy implications of this possible paradigm shift in citation impact analysis are specified.

cs.DL↗

arXiv2011

Mapping excellence in the geography of science: An approach based on Scopus data

As research becomes an ever more globalized activity, there is growing interest in national and international comparisons of standards and quality in different countries and regions. A sign for this trend is the increasing interest in rankings of universities according to their research performance, both inside but also outside the scientific environment. New methods presented in this paper, enable us to map centers of excellence around the world using programs that are freely available. Based on Scopus data, field-specific excellence can be identified and agglomerated in regions and cities where recently highly-cited papers were published. Differences in performance rates can be visualized on the map using colors and sizes of the marks.

cs.DL↗

arXiv2011

The detection of "hot regions" in the geography of science: A visualization approach by using density maps

Spatial scientometrics has attracted a lot of attention in the very recent past. The visualization methods (density maps) presented in this paper allow for an analysis revealing regions of excellence around the world using computer programs that are freely available. Based on Scopus and Web of Science data, field-specific and field-overlapping scientific excellence can be identified in broader regions (worldwide or for a specific continent) where high quality papers (highly cited papers or papers published in Nature or Science) were published. We used a geographic information system to produce our density maps. We also briefly discuss the use of Google Earth.

cs.DL↗

arXiv2010

How fractional counting affects the Impact Factor: Normalization in terms of differences in citation potentials among fields of science

The ISI-Impact Factors suffer from a number of drawbacks, among them the statistics-why should one use the mean and not the median?-and the incomparability among fields of science because of systematic differences in citation behavior among fields. Can these drawbacks be counteracted by counting citation weights fractionally instead of using whole numbers in the numerators? (i) Fractional citation counts are normalized in terms of the citing sources and thus would take into account differences in citation behavior among fields of science. (ii) Differences in the resulting distributions can be tested statistically for their significance at different levels of aggregation. (iii) Fractional counting can be generalized to any document set including journals or groups of journals, and thus the significance of differences among both small and large sets can be tested. A list of fractionally counted Impact Factors for 2008 is available online at http://www.leydesdorff.net/weighted_if/weighted_if.xls. The in-between group variance among the thirteen fields of science identified in the U.S. Science and Engineering Indicators is not statistically significant after this normalization. Although citation behavior differs largely between disciplines, the reflection of these differences in fractionally counted citation distributions could not be used as a reliable instrument for the classification.

cs.DL↗

arXiv2009

A Meta-evaluation of Scientific Research Proposals: Different Ways of Comparing Rejected to Awarded Applications

Combining different data sets with information on grant and fellowship applications submitted to two renowned funding agencies, we are able to compare their funding decisions (award and rejection) with scientometric performance indicators across two fields of science (life sciences and social sciences). The data sets involve 671 applications in social sciences and 668 applications in life sciences. In both fields, awarded applicants perform on average better than all rejected applicants. If only the most preeminent rejected applicants are considered in both fields, they score better than the awardees on citation impact. With regard to productivity we find differences between the fields: While the awardees in life sciences outperform on average the most preeminent rejected applicants, the situation is reversed in social sciences.

cs.CY↗

arXiv2009

Citation Environment of Angewandte Chemie

Recently, aggregated journal-journal citation networks were made accessible from the perspective of each journal included in the Science Citation Index see (http://www.leydesdorff.net/). The local matrices can be used to inspect the relevant citation environment of a journal using statistical analysis and visualization techniques from social network analysis. The inspection gives an answer to the question what the local impact of this and other journals in the environment is. In this study the citation environment of Angewandte Chemie was analysed. Angewandte Chemie is one of the prime chemistry journals in the world. Its environment was compared with that of the Journal of the American Chemical Society. The results of the environment analyses give a detailed insight into the field-embeddedness of Angewandte Chemie. The impacts of the German and international editions of this journal are compared.

cs.DL↗

arXiv2009

New approaches for increasing the reliability of the h index research performance measurement

In the year 2005 Jorge Hirsch introduced the h index for quantifying the research output of scientists. Today, the h index is a widely accepted indicator of research performance. The h index has been criticized for its insufficient reliability - the ability to discriminate reliably between meaningful amounts of research performance. Taking as an example an extensive data set with bibliometric data on scientists working in the field of molecular biology, we compute h2 lower, h2 upper, and sRM values and present them as complementary approaches that improve the reliability of the h index research performance measurement.

stat.AP↗

arXiv2007

Gender differences in grant peer review: A meta-analysis

Narrative reviews of peer review research have concluded that there is negligible evidence of gender bias in the awarding of grants based on peer review. Here, we report the findings of a meta-analysis of 21 studies providing, to the contrary, evidence of robust gender differences in grant award procedures. Even though the estimates of the gender effect vary substantially from study to study, the model estimation shows that all in all, among grant applicants men have statistically significant greater odds of receiving grants than women by about 7%.

math.ST↗

arXiv2006

Latent Markov modelling applied to grant peer review

In the grant peer review process we can distinguish various evaluation stages in which assessors judge applications on a rating scale. Research on this process that considers its multi-stage character scarcely exists. In this study we analyze 1954 applications for doctoral and post-doctoral fellowships of the Boehringer Ingelheim Fonds (B.I.F.), assessed in three stages (first: evaluation by an external reviewer; second: internal evaluation by a staff member; third: final decision by the B.I.F. Board of Trustees). The results show that an application only has a chance of approval if it was recommended for support in the first evaluation stage. Therefore, a form of triage or pre-screening seems desirable. We found differences in transition probabilities from one stage to the other for doctoral applicants submitted by males and females.

math.ST↗