arXiv subjects

Lutz Bornmann

Publications and source records attributed to Lutz Bornmann.

At least 37 records · Page 2Linked to original sources

arXiv2022

Empirical analysis of recent temporal dynamics of research fields: Annual publications in chemistry and related areas as an example

Changes in the number of publications in a certain field might reflect the dynamic of scientific progress in this field, since an increase in the number of publications can be interpreted as an increase in the field-specific knowledge. In this paper, we present a methodological approach to analyse the dynamics of science on lower aggregation levels, i.e., the level of research fields. Our trend analysis approach is able to uncover very recent trends, and the methods used to study the trends are simple to understand for the possible recipients of the results. In order to demonstrate the trend analysis approach, we focused in this study on the annual number of publications (and patents) in chemistry (and related areas) between 2014 and 2020 identifying those fields in chemistry with the highest dynamics (largest rates of change in publication counts). The study is based on the mono-disciplinary literature database CAplus. Our results reveal that the number of publications in the CAplus database is increasing since many years. Research regarding optical phenomena and electrochemical technologies was found to be among the emerging topics in recent years.

cs.DL↗

arXiv2022

Identification of young talented individuals in the natural and life sciences using bibliometric data

Identification of young talented individuals is not an easy task. Citation-based data usually need too long to accrue. In this study, we proposed a method based on bibliometric data for the identification of young talented individuals. Three different indicators and their combinations were used. An older cohort with their first publication between 1999 and 2003 was used to find the most suitable indicator combination. For the validation step, citation impact on the level of individual papers was used. The best performing indicator combination was applied to the time period 2007-2011 for identifying young talented individuals who published their first paper within this time period. We produced a set of 46,200 potential talented individuals.

cs.DL↗

arXiv2022

Revolutions in science: The proposal of an approach for the identification of most important researchers, institutions, and countries based on Reference Publication Year Spectroscopy (RPYS)

RPYS is a bibliometric method originally introduced in order to reveal the historical roots of research topics or fields. RPYS does not identify the most highly cited papers of the publication set being studied (as is usually done by bibliometric analyses in research evaluation), but instead it indicates most frequently referenced publications - each within a specific reference publication year. In this study, we propose to use the method to identify important researchers, institutions and countries in the context of breakthrough research. To demonstrate our approach, we focus on research on physical modeling of Earth's climate and the prediction of global warming as an example. Klaus Hasselmann and Syukuro Manabe were both honored with the Nobel Prize in 2021 for their fundamental contributions to this research. Our results reveal that RPYS is able to identify most important researchers, institutions, and countries. For example, all the relevant authors' institutions are located in the USA. These institutions are either research centers of two US National Research Administrations (NASA and NOAA) or universities: the University of Arizona, Princeton University, the Massachusetts Institute of Technology (MIT), and the University of Stony Brook.

cs.DL↗

arXiv2022

How relevant is climate change research for climate change policy? An empirical analysis based on Overton data

Climate change is an ongoing topic in nearly all areas of society since many years. A discussion of climate change without referring to scientific results is not imaginable. This is especially the case for policies since action on the macro scale is required to avoid costly consequences for society. In this study, we deal with the question of how research on climate change and policy are connected. In 2019, the new Overton database of policy documents was released including links to research papers that are cited by policy documents. The use of results and recommendations from research on climate change might be reflected in citations of scientific papers in policy documents. Although we suspect a lot of uncertainty related to the coverage of policy documents in Overton, there seems to be an impact of international climate policy cycles on policy document publication. We observe local peaks in climate policy documents around major decisions in international climate diplomacy. Our results point out that IGOs and think tanks -- with a focus on climate change -- have published more climate change policy documents than expected. We found that climate change papers that are cited in climate change policy documents received significantly more citations on average than climate change papers that are not cited in these documents. Both areas of society (science and policy) focus on similar climate change research fields: biology, earth sciences, engineering, and disease sciences. Based on these and other empirical results in this study, we propose a simple model of policy impact considering a chain of different document types: the chain starts with scientific assessment reports (systematic reviews) that lead via science communication documents (policy briefs, policy reports or plain language summaries) and government reports to legislative documents.

physics.soc-ph↗

arXiv2022

Reference Publication Year Spectroscopy (RPYS) in practice: A software tutorial

In course of the organization of Workshop III entitled "Cited References Analysis Using CRExplorer" at the International Conference of the International Society for Scientometrics and Informetrics (ISSI2021), we have prepared three reference publication year spectroscopy (RPYS) analyses: (i) papers published in Journal of Informetrics; (ii) papers regarding the topic altmetrics; and (iii) papers published by Ludo Waltman (we selected this researcher since he received the Derek de Solla Price Memorial Medal during the ISSI2021 conference). The first RPYS analysis has been presented live at the workshop and the second and third RPYS analyses have been left to the participants for undertaking after the workshop. Here, we present the results for all three RPYS analyses. The three analyses have shown quite different seminal papers with a few overlaps. Many of the foundational papers in the field of scientometrics (e.g., distributions of publications and citations, citation network and co-citation analyses, and citation analysis with the aim of impact measurement and research evaluation) were retrieved as seminal papers of the papers published in Journal of Informetrics. Mainly papers with discussions of the deficiencies of citation-based impact measurements and comparisons between altmetrics and citations were retrieved as seminal papers of the topic altmetrics. The RPYS analysis of the paper set published by Ludo Waltman mainly retrieved papers about network analyses, citation relations, and citation impact measurement.

cs.DL↗

arXiv2021

Scores of a specific field-normalized indicator calculated with different approaches of field-categorization: Are the scores different or similar?

Usage of field-normalized citation scores is a bibliometric standard. Different methods for field-normalization are in use, but also the choice of field-classification system determines the resulting field-normalized citation scores. Using Web of Science data, we calculated field-normalized citation scores using the same formula but different field-classification systems to answer the question if the resulting scores are different or similar. Six field-classification systems were used: three based on citation relations, one on semantic similarity scores (i.e., a topical relatedness measure), one on journal sets, and one on intellectual classifications. Systems based on journal sets and intellectual classifications agree on at least the moderate level. Two out of the three sets based on citation relations also agree on at least the moderate level. Larger differences were observed for the third data set based on citation relations and semantic similarity scores. The main policy implication is that normalized citation impact scores or rankings based on them should not be compared without deeper knowledge of the classification systems that were used to derive these values or rankings.

cs.DL↗

arXiv2021

Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases

Growth of science is a prevalent issue in science of science studies. In recent years, two new bibliographic databases have been introduced which can be used to study growth processes in science from centuries back: Dimensions from Digital Science and Microsoft Academic. In this study, we used publication data from these new databases and added publication data from two established databases (Web of Science from Clarivate Analytics and Scopus from Elsevier) to investigate scientific growth processes from the beginning of the modern science system until today. We estimated regression models that included simultaneously the publication counts from the four databases. The results of the unrestricted growth of science calculations show that the overall growth rate amounts to 4.10% with a doubling time of 17.3 years. As the comparison of various segmented regression models in the current study revealed, the model with five segments fits the publication data best. We demonstrated that these segments with different growth rates can be interpreted very well, since they are related to either phases of economic (e.g., industrialization) and / or political developments (e.g., Second World War). In this study, we additionally analyzed scientific growth in two broad fields (Physical and Technical Sciences as well as Life Sciences) and the relationship of scientific and economic growth in UK. The comparison between the two fields revealed only slight differences. The comparison of the British economic and scientific growth rates showed that the economic growth rate is slightly lower than the scientific growth rate.

cs.DL↗

arXiv2021

Investigating Dissemination of Scientific Information on Twitter: A Study of Topic Networks in Opioid Publications

One way to assess a certain aspect of the value of scientific research is to measure the attention it receives on social media. While previous research has mostly focused on the "number of mentions" of scientific research on social media, the current study applies "topic networks" to measure public attention to scientific research on Twitter. Topic networks are the networks of co-occurring author keywords in scholarly publications and networks of co-occurring hashtags in the tweets mentioning those scholarly publications. This study investigates which topics in opioid scholarly publications have received public attention on Twitter. Additionally, it investigates whether the topic networks generated from the publications tweeted by all accounts (bot and non-bot accounts) differ from those generated by non-bot accounts. Our analysis is based on a set of opioid scholarly publications from 2011 to 2019 and the tweets associated with them. We use co-occurrence network analysis to generate topic networks. Results indicated that Twitter users have mostly used generic terms to discuss opioid publications, such as "Opioid," "Pain," "Addiction," "Treatment," "Analgesics," "Abuse," "Overdose," and "Disorders." Results confirm that topic networks provide a legitimate method to visualize public discussions of health-related scholarly publications and how Twitter users discuss health-related scientific research differently from the scientific community. There was a substantial overlap between the topic networks based on the tweets by all accounts and non-bot accounts. This result indicates that it might not be necessary to exclude bot accounts for generating topic networks as they have a negligible impact on the results.

cs.IR↗

arXiv2021

Report on Workshop III "Cited References Analysis Using CRExplorer" at the 18th International Conference of the International Society for Scientometrics and Informetrics (ISSI2021)

We have organized Workshop III entitled "Cited References Analysis Using CRExplorer" at ISSI2021. Here, we report and reflect on this workshop. The aim of this workshop was to bring beginners, practitioners, and experts in cited references analyses together. A mixture of presentations and an interactive part was intended to provide benefits for all kinds of scientometricians with an interest in cited references analyses.

cs.DL↗

arXiv2021

Heat Waves -- a hot topic in climate change research

Research on heat waves (periods of excessively hot weather, which may be accompanied by high humidity) is a newly emerging research topic within the field of climate change research with high relevance for the whole of society. In this study, we analyzed the rapidly growing scientific literature dealing with heat waves. No summarizing overview has been published on this literature hitherto. We developed a suitable search query to retrieve the relevant literature covered by the Web of Science (WoS) as complete as possible and to exclude irrelevant literature (n = 8,011 papers). The time-evolution of the publications shows that research dealing with heat waves is a highly dynamic research topic, doubling within about 5 years. An analysis of the thematic content reveals the most severe heat wave events within the recent decades (1995 and 2003), the cities and countries/regions affected (United States, Europe, and Australia), and the ecological and medical impacts (drought, urban heat islands, excess hospital admissions, and mortality). Risk estimation and future strategies for adaptation to hot weather are major political issues. We identified 104 citation classics which include fundamental early works of research on heat waves and more recent works (which are characterized by a relatively strong connection to climate change).

cs.DL↗

arXiv2021

Quantum technology 2.0 -- topics and contributing countries from 1980 to 2018

The second quantum technological revolution started around 1980 with the control of single quantum particles and their interaction on an individual basis. These experimental achievements enabled physicists and engineers to utilize long-known quantum features - especially superposition and entanglement of single quantum states - for a whole range of practical applications. We use a publication set of 54,598 papers from the Web of Science published between 1980 and 2018 to investigate the time development of four main subfields of quantum technology in terms of numbers and shares of publication as well as the occurrence of topics and their relation to the 25 top contributing countries. Three successive time periods are distinguished in the analyses by their short doubling times in relation to the whole Web of Science. The periods can be characterized by the publication of pioneering works, the exploration of research topics, and the maturing of quantum technology, respectively. Compared to the US, China has a far over proportional contribution to the worldwide publication output, but not in the segment of highly-cited papers.

cs.DL↗

arXiv2021

Is culture related to strong science? An empirical investigation

National culture is among those societal factors which could influence research and innovation activities. In this study, we investigated the associations of two national culture models with citation impact of nations (measured by the proportion of papers belonging to the 10% and 1% most cited papers in the corresponding fields, PPtop 10% and PPtop 1%). Bivariate statistical analyses showed that of six Hofstede's national culture dimensions (HNCD), uncertainty avoidance and power distance had a statistically significant negative associa-tion, while individualism and indulgence had a statistically significant positive associationwith both citation impact indicators. The study also revealed that of two Inglehart-Welzel cultural values (IWCV), the value survival versus self-expression is statistically significantly related to citation impact indicators. We additionally calculated multiple regression analyses controlling for the possible effects of confounding factors including national self-citations, international co-authorships, invest-ments in research and development, international migrant stock, number of researchers ofeach nation, language, and productivity. The results revealed that the statistically significant associations of HNCD with citation impact indicators disappeared. But the statistically significant relationship between survivals versus self-expression values and citation impact indicators remained stable even after controlling for the confounding variables. Thus, the freedom of expression and trust in society might contribute to better scholarly communication systems, higher level of international collaborations, and further quality research.

cs.DL↗

arXiv2021

Mapping the impact of papers on various status groups: A new excellence mapping tool based on citation and reader scores

In over five years, Bornmann, Stefaner, de Moya Anegon, and Mutz (2014) and Bornmann, Stefaner, de Moya Anegon, and Mutz (2014, 2015) have published several releases of the www.excellencemapping.net tool revealing (clusters of) excellent institutions worldwide based on citation data. With the new release, a completely revised tool has been published. It is not only based on citation data (bibliometrics), but also Mendeley data (altmetrics). Thus, the institutional impact measurement of the tool has been expanded by focusing on additional status groups besides researchers such as students and librarians. Furthermore, the visualization of the data has been completely updated by improving the operability for the user and including new features such as institutional profile pages. In this paper, we describe the datasets for the current excellencemapping.net tool and the indicators applied. Furthermore, the underlying statistics for the tool and the use of the web application are explained.

cs.DL↗

arXiv2020

A Decade of In-text Citation Analysis based on Natural Language Processing and Machine Learning Techniques: An overview of empirical studies

Citation analysis is one of the most frequently used methods in research evaluation. We are seeing significant growth in citation analysis through bibliometric metadata, primarily due to the availability of citation databases such as the Web of Science, Scopus, Google Scholar, Microsoft Academic, and Dimensions. Due to better access to full-text publication corpora in recent years, information scientists have gone far beyond traditional bibliometrics by tapping into advancements in full-text data processing techniques to measure the impact of scientific publications in contextual terms. This has led to technical developments in citation context and content analysis, citation classifications, citation sentiment analysis, citation summarisation, and citation-based recommendation. This article aims to narratively review the studies on these developments. Its primary focus is on publications that have used natural language processing and machine learning techniques to analyse citations.

cs.DL↗

arXiv2020

Bibliometrics-based heuristics: What is their definition and how can they be studied?

When scientists study the phenomena they are interested in, they apply sound methods and base their work on theoretical considerations. In contrast, when the fruits of their research is being evaluated, basic scientific standards do not seem to matter. Instead, simplistic bibliometric indicators (i.e., publications and citation counts) are, paradoxically, both widely used and criticized without any methodological and theoretical framework that would serve to ground both use and critique. Yet, Bornmann and Marewski [1] proposed such a framework recently. They developed bibliometrics-based heuristics (BBHs) based on the fast-and-frugal heuristics approach [2] to decision making, in order to conceptually understand and empirically investigate the quantitative evaluation of research as well as to effectively train end-users of bibliometrics (e.g., science managers, scientists). Heuristics are decision strategies that use part of the available information and ignore the rest. By exploiting the statistical structure of task environments, they can aid to make accurate, fast, effortless, and cost-efficient decisions without that trade-offs are incurred. Because of their simplicity, heuristics are easy to understand and communicate, enhancing the transparency of decision processes. In this commentary, we explain several BBHs and discuss how such heuristics can be employed in practice (using the evaluation of applicants for funding programs as one example). Furthermore, we outline why heuristics can perform well, and how they and their fit to task environments can be studied. In pointing to the potential of research on BBHs and to the risks that come with an under-researched, mindless usage of bibliometrics, this commentary contributes to make research evaluation more scientific.

cs.DL↗

arXiv2020

Convergent validity of several indicators measuring disruptiveness with milestone assignments to physics papers by experts

This study focuses on a recently introduced type of indicator measuring disruptiveness in science. Disruptive research diverges from current lines of research by opening up new lines. In the current study, we included the initially proposed indicator of this new type (Wu, Wang, & Evans, 2019) and several variants with DI1: DI5, DI1n, DI5n, and DEP. Since indicators should measure what they propose to measure, we investigated the convergent validity of the indicators. We used a list of milestone papers, selected and published by editors of Physical Review Letters, and investigated whether this human (experts - based list is related to values of the several disruption indicators variants and - if so - which variants show the highest correlation with expert judgements. We used bivariate statistics, multiple regression models, and (coarsened) exact matching (CEM) to investigate the convergent validity of the indicators. The results show that the indicators correlate differently with the milestone paper assignments by the editors. It is not the initially proposed disruption index that performed best (DI1), but the variant DI5 which has been introduced by Bornmann, Devarakonda, Tekles, and Chacko (2019). In the CEM analysis of this study, the DEP variant - introduced by Bu, Waltman, and Huang (2019) - also showed favorable results.

cs.DL↗

arXiv2020

Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching

Field-normalization of citations is bibliometric standard. Despite the observed differences in citation counts between fields, the question remains how strong fields influence citation rates beyond the effect of attributes or factors possibly influencing citations (FICs). We considered several FICs such as number of pages and number of co-authors in this study. We wondered whether there is a separate field-effect besides other effects (e.g., from numbers of pages and co-authors). To find an answer on the question in this study, we applied inverse-probability of treatment weighting (IPW). Using Web of Science data (a sample of 308,231 articles), we investigated whether mean differences among subject categories in citation rates still remain, even if the subject categories are made comparable in the field-related attributes (e.g., comparable of co-authors, comparable number of pages) by IPW. In a diagnostic step of our statistical analyses, we considered propensity scores as covariates in regression analyses to examine whether the differences between the fields in FICs vanish. The results revealed that the differences did not completely vanish but were strongly reduced. We received similar results when we calculated mean value differences of the fields after IPW representing the causal or unconfounded field effects on citations. However, field differences in citation rates remain. The results point out that field-normalization seems to be a prerequisite for citation analysis and cannot be replaced by the consideration of any set of FICs in citation analyses.

cs.DL↗

arXiv2020

An Evaluation of Percentile Measures of Citation Impact, and a Proposal for Making Them Better

Percentiles are statistics pointing to the standing of a paper's citation impact relative to other papers in a given citation distribution. Percentile Ranks (PRs) often play an important role in evaluating the impact of scholars, institutions, and lines of study. Because PRs are so important for the assessment of scholarly impact, and because citation practices differ greatly across time and fields, various percentile approaches have been proposed to time- and field-normalize citations. Unfortunately, current popular methods often face significant problems in time- and field-normalization, including when papers are assigned to multiple fields or have been published by more than one unit (e.g., researchers or countries). They also face problems for estimating citation counts (CCs) for pre-defined PRs (e.g., the 90th PR). We offer a series of guidelines and procedures that, we argue, address these problems and others and provide a superior means to make the use of percentile methods more accurate and informative. In particular, we introduce two approaches, CP-IN and CP-EX, that should be preferred in bibliometric studies because they consider the complete citation distribution. Both approaches are based on cumulative frequencies in percentages (CPs). The paper further shows how bar graphs and beamplots can present PRs in a more meaningful and accurate manner.

cs.DL↗