arXiv subjects

Lutz Bornmann

Publications and source records attributed to Lutz Bornmann.

At least 109 records · Page 6Linked to original sources

arXiv2016

Policy documents as sources for measuring societal impact: How often is climate change research mentioned in policy-related documents?

In the current UK Research Excellence Framework (REF) and the Excellence in Research for Australia (ERA) societal impact measurements are inherent parts of the national evaluation systems. In this study, we deal with a relatively new form of societal impact measurements. Recently, Altmetric - a start-up providing publication level metrics - started to make data for publications available which have been mentioned in policy documents. We regard this data source as an interesting possibility to specifically measure the (societal) impact of research. Using a comprehensive dataset with publications on climate change as an example, we study the usefulness of the new data source for impact measurement. Only 1.2% (n=2,341) out of 191,276 publications on climate change in the dataset have at least one policy mention. We further reveal that papers published in Nature and Science as well as from the areas "Earth and related environmental sciences" and "Social and economic geography" are especially relevant in the policy context. Given the low coverage of the climate change literature in policy documents, this study can be only a first attempt to study this new source of altmetric data. Further empirical studies are necessary in upcoming years, because mentions in policy documents are of special interest in the use of altmetric data for measuring target-oriented the broader impact of research.

physics.soc-ph↗

arXiv2016

Citations: Indicators of Quality? The Impact Fallacy

We argue that citation is a composed indicator: short-term citations can be considered as currency at the research front, whereas long-term citations can contribute to the codification of knowledge claims into concept symbols. Knowledge claims at the research front are more likely to be transitory and are therefore problematic as indicators of quality. Citation impact studies focus on short-term citation, and therefore tend to measure not epistemic quality, but involvement in current discourses in which contributions are positioned by referencing. We explore this argument using three case studies: (1) citations of the journal Soziale Welt as an example of a venue that tends not to publish papers at a research front, unlike, for example, JACS; (2) Robert Merton as a concept symbol across theories of citation; and (3) the Multi-RPYS ("Multi-Referenced Publication Year Spectroscopy") of the journals Scientometrics, Gene, and Soziale Welt. We show empirically that the measurement of "quality" in terms of citations can further be qualified: short-term citation currency at the research front can be distinguished from longer-term processes of incorporation and codification of knowledge claims into bodies of knowledge. The recently introduced Multi-RPYS can be used to distinguish between short-term and long-term impacts.

cs.DL↗

arXiv2016

Construction of a Pragmatic Base Line for Journal Classifications and Maps Based on Aggregated Journal-Journal Citation Relations

A number of journal classification systems have been developed in bibliometrics since the launch of the Citation Indices by the Institute of Scientific Information (ISI) in the 1960s. These systems are used to normalize citation counts with respect to field-specific citation patterns. The best known system is the so-called "Web-of-Science Subject Categories" (WCs). In other systems papers are classified by algorithmic solutions. Using the Journal Citation Reports 2014 of the Science Citation Index and the Social Science Citation Index (n of journals = 11,149), we examine options for developing a new system based on journal classifications into subject categories using aggregated journal-journal citation data. Combining routines in VOSviewer and Pajek, a tree-like classification is developed. At each level one can generate a map of science for all the journals subsumed under a category. Nine major fields are distinguished at the top level. Further decomposition of the social sciences is pursued for the sake of example with a focus on journals in information science (LIS) and science studies (STS). The new classification system improves on alternative options by avoiding the problem of randomness in each run that has made algorithmic solutions hitherto irreproducible. Limitations of the new system are discussed (e.g. the classification of multi-disciplinary journals). The system's usefulness for field-normalization in bibliometrics should be explored in future studies.

cs.DL↗

arXiv2016

New features of CitedReferencesExplorer (CRExplorer)

CRExplorer version 1.6.7 was released on July 5, 2016. This version includes the following new features and improvements: Scopus: Using "File" - "Import" - "Scopus", CRExplorer reads files from Scopus. The file format "CSV" (including citations, abstracts and references) should be chosen in Scopus for downloading records. Export facilities: Using "File" - "Export" - "Scopus", CRExplorer exports files in the Scopus format. Using "File" - "Export" - "Web of Science", CRExplorer exports files in the Web of Science format. These files can be imported in other bibliometric programs (e.g. VOSviewer). Space bar: Select a specific cited reference in the cited references table, press the space bar, and all bibliographic details of the CR are shown. Internal file format: Using "File" - "Save", working files are saved in the internal file format "*.cre". The files include all data including matching results and manual matching corrections. The files can be opened by using "File" - "Open".

cs.DL↗

arXiv2016

Climate Change Research in View of Bibliometrics

This bibliometric study of a large publication set dealing with research on climate change aims at mapping the relevant literature from a bibliometric perspective and presents a multitude of quantitative data: (1) The growth of the overall publication output as well as (2) of some major subfields, (3) the contributing journals and countries as well as their citation impact, and (4) a title word analysis aiming to illustrate the time evolution and relative importance of specific research topics. The study is based on 222,060 papers published between 1980 and 2014. The total number of papers shows a strong increase with a doubling every 5-6 years. Continental biomass related research is the major subfield, closely followed by climate modeling. Research dealing with adaptation, mitigation, risks, and vulnerability of global warming is comparatively small, but their share of papers increased exponentially since 2005. Research on vulnerability and on adaptation published the largest proportion of very important papers. Research on climate change is quantitatively dominated by the USA, followed by the UK, Germany, and Canada. The citation-based indicators exhibit consistently that the UK has produced the largest proportion of high impact papers compared to the other countries (having published more than 10,000 papers). The title word analysis shows that the term climate change comes forward with time. Furthermore, the term impact arises and points to research dealing with the various effects of climate change. Finally, the term model and related terms prominently appear independent of time, indicating the high relevance of climate modeling.

cs.DL↗

arXiv2016

Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator

In this paper, a new field-normalized indicator is introduced, which is rooted in early insights in bibliometrics, and is compared with several established field-normalized indicators (e.g. the mean normalized citation score, MNCS, and indicators based on percentile approaches). Garfield (1979) emphasizes that bare citation counts from different fields cannot be compared for evaluative purposes, because the "citation potential" can vary significantly between the fields. Garfield (1979) suggests that "the most accurate measure of citation potential is the average number of references per paper published in a given field". Based on this suggestion, the new indicator is basically defined as follows: the citation count of a focal paper is divided by the mean number of cited references in a field to normalize citations. The new indicator is called citation score normalized by cited references (CSNCR). The theoretical analysis of the CSNCR shows that it has the properties of consistency and homogeneous normalization. The close relation of the new indicator to the MNCS is discussed. The empirical comparison of the CSNCR with other field-normalized indicators shows that it is slightly poorer able to field-normalize citation counts than other cited-side normalized indicators (e.g. the MNCS), but its results are favorable compared to two citing-side indicator variants (SNCS indicators). Taken as a whole, the results of this study confirm the ability of established indicators to field-normalize citations.

cs.DL↗

arXiv2016

Efficiency of research performance and the glass researcher

Abramo and D'Angelo (in press) doubt the validity of established size-independent indicators measuring citation impact and plead in favor of measuring scientific efficiency (by using the Fractional Scientific Strength indicator). This note is intended to comment on some questionable and a few favorable approaches in the paper by Abramo and D'Angelo (in press).

cs.DL↗

arXiv2016

Referenced Publication Year Spectroscopy (RPYS) and Algorithmic Historiography: The Bibliometric Reconstruction of András Schubert's Œuvre

Referenced Publication Year Spectroscopy (RPYS) was recently introduced as a method to analyze the historical roots of research fields and groups or institutions. RPYS maps the distribution of the publication years of the cited references in a document set. In this study, we apply this methodology to the œuvre of an individual researcher on the occasion of a Festschrift for András Schubert's 70th birthday. We discuss the different options of RPYS in relation to one another (e.g. Multi-RPYS), and in relation to the longer-term research program of algorithmic historiography (e.g., HistCite) based on Schubert's publications (n=172) and cited references therein as a bibliographic domain in scientometrics. Main path analysis and Multi-RPYS of the citation network are used to show the changes and continuities in Schubert's intellectual career. Diachronic and static decomposition of a document set can lead to different results, while the analytically distinguishable lines of research may overlap and interact over time, and intermittent.

cs.DL↗

arXiv2016

The "Tournaments" Metaphor in Citation Impact Studies: Power-Weakness Ratios (PWR) as a Journal Indicator

Ramanujacharyulu's (1964) Power-Weakness Ratio (PWR) measures impact by recursively multiplying the citation matrix by itself until convergence is reached in both the cited and citing dimensions; the quotient of these values is defined as PWR, whereby "cited" is considered as power and "citing" as weakness. Analytically, PWR is an attractive candidate for measuring journal impact because of its symmetrical handling of the rows and columns in the asymmetrical citation matrix, its recursive algorithm, and its mathematical elegance. In this study, PWR is discussed and critically assessed in relation to other size-independent recursive metrics. A test using the set of 83 journals in "information and library science" (according to the Web-of-Science categorization) converged, but did not provide interpretable results. Further decomposition of this set into homogeneous sub-graphs shows that--like most other journal indicators--PWR can perhaps be used within homogeneous sets, but not across citation communities.

cs.DL↗

arXiv2016

Measuring impact in research evaluations: A thorough discussion of methods for, effects of, and problems with impact measurements

Impact of science is one of the most important topics in scientometrics. Recent developments show a fundamental change in impact measurements from impact on science to impact on society. Since impact measurement is currently in a state of far reaching changes, this paper describes recent developments and facing problems in this area. For that the results of key publications (dealing with impact measurement) are discussed. The paper discusses how impact is generally measured within science and beyond (section 2), which effects impact measurements have on the science system (section 3), and which problems are associated with impact measurement (section 4). The problems associated with impact measurement constitute the focus of this paper: Science is marked by inequality, random chance, anomalies, the right to make mistakes, unpredictability, and a high significance of extreme events, which might distort impact measurements. Scientometricians as the producer of impact scores and decision makers as their consumers should be aware of these problems and should consider them in the generation and interpretation of bibliometric results, respectively.

cs.DL↗

arXiv2016

Introducing CitedReferencesExplorer (CRExplorer): A program for Reference Publication Year Spectroscopy with Cited References Standardization

We introduce a new tool - the CitedReferencesExplorer (CRExplorer, www.crexplorer.net) - which can be used to disambiguate and analyze the cited references (CRs) of a publication set downloaded from the Web of Science (WoS). The tool is especially suitable to identify those publications which have been frequently cited by the researchers in a field and thereby to study for example the historical roots of a research field or topic. CRExplorer simplifies the identification of key publications by enabling the user to work with both a graph for identifying most frequently cited reference publication years (RPYs) and the list of references for the RPYs which have been most frequently cited. A further focus of the program is on the standardization of CRs. It is a serious problem in bibliometrics that there are several variants of the same CR in the WoS. In this study, CRExplorer is used to study the CRs of all papers published in the Journal of Informetrics. The analyses focus on the most important papers published between 1980 and 1990.

cs.DL↗

arXiv2016

Detecting the historical roots of tribology research: a bibliometric analysis

In this study, the historical roots of tribology are investigated using a newly developed scientometric method called Referenced Publication Years Spectroscopy. The study is based on cited references in tribology research publications. The Science Citation Index Expanded is used as data source. The results show that RPYS has the potential to identify the important publications : Most of the publications which have been identified in this study as highly cited (referenced) publications are landmark publications in the field of tribology.

cs.DL↗

arXiv2016

Is collaboration among scientists related to the citation impact of papers because their quality increases with collaboration? An analysis based on data from F1000Prime and normalized citation scores

In recent years, the relationship of collaboration among scientists and the citation impact of papers have been frequently investigated. Most of the studies show that the two variables are closely related: an increasing collaboration activity (measured in terms of number of authors, number of affiliations, and number of countries) is associated with an increased citation impact. However, it is not clear whether the increased citation impact is based on the higher quality of papers which profit from more than one scientist giving expert input or other (citation-specific) factors. Thus, the current study addresses this question by using two comprehensive datasets with publications (in the biomedical area) including quality assessments by experts (F1000Prime member scores) and citation data for the publications. The study is based on nearly 10,000 papers. Robust regression models are used to investigate the relationship between number of authors, number of affiliations, and number of countries, respectively, and citation impact - controlling for the papers' quality (measured by F1000Prime expert ratings). The results point out that the effect of collaboration activities on impact is largely independent of the papers' quality. The citation advantage is apparently not quality-related; citation specific factors (e.g. self-citations) seem to be important here.

cs.DL↗

arXiv2016

Relative Citation Ratio (RCR): An empirical attempt to study a new field-normalized bibliometric indicator

Hutchins, Yuan, M., and Santangelo (2015) proposed the Relative Citation Ratio (RCR) as a new field-normalized impact indicator. This study investigates the RCR by correlating it on the level of single publications with established field-normalized indicators and assessments of the publications by peers. We find that the RCR correlates highly with established field-normalized indicators, but the correlation between RCR and peer assessments is only low to medium.

cs.DL↗

arXiv2016

Excellence networks in science: A Web-based application based on Bayesian multilevel logistic regression (BMLR) for the identification of institutions collaborating successfully

In this study we present an application which can be accessed via www.excellence-networks.net and which represents networks of scientific institutions worldwide. The application is based on papers (articles, reviews and conference papers) published between 2007 and 2011. It uses (network) data, on which the SCImago Institutions Ranking is based (Scopus data from Elsevier). Using this data, institutional networks have been estimated with statistical models (Bayesian multilevel logistic regression, BMLR) for a number of Scopus subject areas. Within single subject areas, we have investigated and visualized how successfully overall an institution (reference institution) has collaborated (compared to all the other institutions in a subject area), and with which other institutions (network institutions) a reference institution has collaborated particularly successfully. The "best paper rate" (statistically estimated) was used as an indicator for evaluating the collaboration success of an institution. This gives the proportion of highly cited papers from an institution, and is considered generally as an indicator for measuring impact in bibliometrics.

cs.DL↗

arXiv2015

Sampling Issues in Bibliometric Analysis

Bibliometricians face several issues when drawing and analyzing samples of citation records for their research. Drawing samples that are too small may make it difficult or impossible for studies to achieve their goals, while drawing samples that are too large may drain resources that could be better used for other purposes. This paper considers three common situations and offers advice for dealing with each. First, an entire population of records is available for an institution. We argue that, even though all records have been collected, the use of inferential statistics, significance testing, and confidence intervals is both common and desirable. Second, because of limited resources or other factors, a sample of records needs to be drawn. We demonstrate how power analyses can be used to determine in advance how large the sample needs to be to achieve the study's goals. Third, the sample size may already be determined, either because the data have already been collected or because resources are limited. We show how power analyses can again be used to determine how large effects need to be in order to find effects that are statistically significant. Such information can then help bibliometricians to develop reasonable expectations as to what their analysis can accomplish. While we focus on issues of interest to bibliometricians, our recommendations and procedures can easily be adapted for other fields of study.

stat.AP↗

arXiv2015

Networks of reader and country status: An analysis of Mendeley reader statistics

The number of papers published in journals indexed by the Web of Science core collection is steadily increasing. In recent years, nearly two million new papers were published each year; somewhat more than one million papers when primary research papers are considered only (articles and reviews are the document types where primary research is usually reported or reviewed). However, who reads these papers? More precisely, which groups of researchers from which (self-assigned) scientific disciplines and countries are reading these papers? Is it possible to visualize readership patterns for certain countries, scientific disciplines, or academic status groups? One popular method to answer these questions is a network analysis. In this study, we analyze Mendeley readership data of a set of 1,133,224 articles and 64,960 reviews with publication year 2012 to generate three different kinds of networks: (1) The network based on disciplinary affiliations of Mendeley readers contains four groups: (i) biology, (ii) social science and humanities (including relevant computer science), (iii) bio-medical sciences, and (iv) natural science and engineering. In all four groups, the category with the addition "miscellaneous" prevails. (2) The network of co-readers in terms of professional status shows that a common interest in papers is mainly shared among PhD students, Master's students, and postdocs. (3) The country network focusses on global readership patterns: a group of 53 nations is identified as core to the scientific enterprise, including Russia and China as well as two thirds of the OECD (Organisation for Economic Co-operation and Development) countries.

cs.DL↗

arXiv2015

Replicability and the public/private divide

In a recent letter, Carlos Vilchez-Roman criticizes Bornmann et al. (2015) for using data which cannot be reproduced without access to an in-house version of the Web-of-Science (WoS) at the Max Planck Digital Libraries (MPDL, Munich). We agree with the norm of replicability and therefore returned to our data. Is the problem only a practical one of automation or does the in-house processing add analytical value to the data? Is the newly emerging situation in any sense different from a further professionalization of the field? In our opinion, a political economy of science indicators has in the meantime emerged with a competitive dynamic that affects the intellectual organization of the field.

cs.DL↗