arXiv subjects

Lutz Bornmann

Publications and source records attributed to Lutz Bornmann.

At least 91 records · Page 5Linked to original sources

arXiv2017

The number of linked references of publications in Microsoft Academic in comparison with the Web of Science

In the context of a comprehensive Microsoft Academic (MA) study, we explored in an initial step the quality of linked references data in MA in comparison with Web of Science (WoS). Linked references are the backbone of bibliometrics, because they are the basis of the times cited information in citation indexes. We found that the concordance of linked references between MA and WoS ranges from weak to non-existent for the full sample (publications of the University of Zurich with less than 50 linked references in MA). An analysis with a sample restricted to less than 50 linked references in WoS showed a strong agreement between linked references in MA and WoS.

cs.DL↗

arXiv2017

The geography of references in elite articles: What countries contribute to the archives of knowledge

This study is intended to find an answer for the question on which national "shoulders" the worldwide top-level research stands. Traditionally, national scientific standings are evaluated in terms of the number of citations to their papers. We raise a different question: instead of analyzing the citations to the countries' articles (the forward view), we examine referenced publications from specific countries cited in the most elite publications (the backward-citing-view). "Elite publications" are operationalized as the top-1% most-highly cited articles. Using the articles published during the years 2004 to 2013, we examine the research referenced in these works. Our results confirm the well-known fact that China has emerged to become a major player in science. However, China still belongs to the low contributors when countries are ranked as contributors to the cited references in top-1% articles. Using this perspective, the results do not point to a decreasing trend for the USA; in fact, the USA exceeds expectations (compared to its publication share) in terms of contributions to cited references in the top-1% articles. Switzerland, Sweden, and the Netherlands also are shown at the top of the list. However, the results for Germany are lower than statistically expected.

cs.DL↗

arXiv2017

Reference Publication Year Spectroscopy (RPYS) of Eugene Garfield's publications

Which studies, theories, and ideas have influenced Eugene Garfield's scientific work? Recently, the method reference publication year spectroscopy (RPYS) has been introduced, which can be used to answer this and related questions. Since then, several studies have been published dealing with the historical roots of research fields and scientists. The program CRExplorer (http://www.crexplorer.net) was specifically developed for RPYS. In this study, we use this program to investigate the historical roots of Eugene Garfield's oeuvre.

cs.DL↗

arXiv2017

Visualizing the context of citations referencing papers published by Eugene Garfield: A new type of keyword co-occurrence analysis

During Eugene Garfield's (EG's) lengthy career as information scientist, he published about 1,500 papers. In this study, we use the impressive oeuvre of EG to introduce a new type of bibliometric networks: keyword co-occurrences networks based on the context of citations, which are referenced in a certain paper set (here: the papers published by EG). The citation context is defined by the words which are located around a specific citation. We retrieved the citation context from Microsoft Academic. To interpret and compare the results of the new network type, we generated two further networks: co-occurrence networks which are based on title and abstract keywords from (1) EG's papers and (2) the papers citing EG's publications. The comparison of the three networks suggests that papers of EG and citation contexts of papers citing EG are semantically more closely related to each other than to titles and abstracts of papers citing EG. This result accords with the use of citations in research evaluation that is based on the premise that citations reflect the cognitive influence of the cited on the citing publication.

cs.DL↗

arXiv2017

Normalization of zero-inflated data: An empirical analysis of a new indicator family

Recently, two new indicators (Equalized Mean-based Normalized Proportion Cited, EMNPC, and Mean-based Normalized Proportion Cited, MNPC) were proposed which are intended for sparse data. We propose a third indicator (Mantel-Haenszel quotient, MHq) belonging to the same indicator family. The MHq is based on the MH analysis - an established method for polling the data from multiple 2x2 contingency tables based on different subgroups. We test (using citations and assessments by peers) if the three indicators can distinguish between different quality levels as defined on the basis of the assessments by peers (convergent validity). We find that the indicator MHq is able to distinguish between the quality levels in most cases while MNPC and EMNPC are not.

cs.DL↗

arXiv2017

Can the Journal Impact Factor Be Used as a Criterion for the Selection of Junior Researchers? A Large-Scale Empirical Study Based on ResearcherID Data

Early in researchers' careers, it is difficult to assess how good their work is or how important or influential the scholars will eventually be. Hence, funding agencies, academic departments, and others often use the Journal Impact Factor (JIF) of where the authors have published to assess their work and provide resources and rewards for future work. The use of JIFs in this way has been heavily criticized, however. Using a large data set with many thousands of publication profiles of individual researchers, this study tests the ability of the JIF (in its normalized variant) to identify, at the beginning of their careers, those candidates who will be successful in the long run. Instead of bare JIFs and citation counts, the metrics used here are standardized according to Web of Science subject categories and publication years. The results of the study indicate that the JIF (in its normalized variant) is able to discriminate between researchers who published papers later on with a citation impact above or below average in a field and publication year - not only in the short term, but also in the long term. However, the low to medium effect sizes of the results also indicate that the JIF (in its normalized variant) should not be used as the sole criterion for identifying later success: other criteria, such as the novelty and significance of the specific research, academic distinctions, and the reputation of previous institutions, should also be considered.

cs.DL↗

arXiv2017

Betweenness and Diversity in Journal Citation Networks as Measures of Interdisciplinarity -- A Tribute to Eugene Garfield --

Journals were central to Eugene Garfield's research interests. Among other things, journals are considered as units of analysis for bibliographic databases such as the Web of Science (WoS) and Scopus. In addition to disciplinary classifications of journals, journal citation patterns span networks across boundaries to variable extents. Using betweenness centrality (BC) and diversity, we elaborate on the question of how to distinguish and rank journals in terms of interdisciplinarity. Interdisciplinarity, however, is difficult to operationalize in the absence of an operational definition of disciplines, the diversity of a unit of analysis is sample-dependent. BC can be considered as a measure of multi-disciplinarity. Diversity of co-citation in a citing document has been considered as an indicator of knowledge integration, but an author can also generate trans-disciplinary--that is, non-disciplined--variation by citing sources from other disciplines. Diversity in the bibliographic coupling among citing documents can analogously be considered as diffusion of knowledge across disciplines. Because the citation networks in the cited direction reflect both structure and variation, diversity in this direction is perhaps the best available measure of interdisciplinarity at the journal level. Furthermore, diversity is based on a summation and can therefore be decomposed, differences among (sub)sets can be tested for statistical significance. In an appendix, a general-purpose routine for measuring diversity in networks is provided.

cs.DL↗

arXiv2017

Measuring field-normalized impact of papers on specific societal groups: An altmetrics study based on Mendeley data

Bibliometrics is successful in measuring impact, because the target is clearly defined: the publishing scientist who is still active and working. Thus, citations are a target-oriented metric which measures impact on science. In contrast, societal impact measurements based on altmetrics are as a rule intended to measure impact in a broad sense on all areas of society (e.g. science, culture, politics, and economics). This tendency is especially reflected in the efforts to design composite indicators (e.g. the Altmetric attention score). We deem appropriate that not only the impact measurement using citations is target-oriented (citations measure the impact of papers on scientists), but also the measurement of impact using altmetrics. Impact measurements only make sense, if the target group - the recipient of academic papers - is clearly defined. Thus, we extend in this study the field-normalized reader impact indicator proposed by us in an earlier study, which is based on Mendeley data (the mean normalized reader score, MNRS), to a target-oriented field-normalized impact indicator (e.g., MNRS_ED measures reader impact on the sector of educational donation, i.e., teaching). This indicator can show - as demonstrated in empirical examples - the ability of journals, countries, and academic institutions to publish papers which are below or above the average impact of papers on a specific sector in society (e.g., the educational or teaching sector). For example, the method allows to measure the impact of scientific papers on students - controlling for the field in which the papers have been published and their publication year.

cs.DL↗

arXiv2017

t factor: A metric for measuring impact on Twitter

Based on the definition of the well-known h index we propose a t factor for measuring the impact of publications (and other entities) on Twitter. The new index combines tweet and retweet data in a balanced way whereby retweets are seen as data reflecting the impact of initial tweets. The t factor is defined as follows: A unit (single publication, journal, researcher, research group etc.) has factor t if t of its Nt tweets have at least t retweets each and the other (Nt-t) tweets have <=t retweets each.

cs.DL↗

arXiv2017

Generating Clustered Journal Maps: An Automated System for Hierarchical Classification

Journal maps and classifications for 11,359 journals listed in the combined Journal Citation Reports 2015 of the Science and Social Sciences Citation Indexes are provided at http://www.leydesdorff.net/jcr15. A routine using VOSviewer for integrating the journal mapping and their hierarchical clustering is also made available. In this short communication, we provide background on the journal mapping/clustering and an explanation and instructions about the routine. We compare 2015 journal maps with those for 2014 and show the delineations among fields and subfields to be sensitive to fluctuations. Labels for fields and sub-fields are not provided by the routine, but can be added by an analyst for pragmatic or intellectual reasons. The routine provides a means for testing one's assumptions against a baseline without claiming authority, clusters of related journals can be visualized to understand communities. The routine is generic and can be used for any 1-mode network.

cs.DL↗

arXiv2016

The Journal Impact Factor Should Not Be Discarded

The Journal Impact Factor (JIF) has been heavily criticized over decades. This opinion piece argues that the JIF should not be demonized. It still can be employed for research evaluation purposes by carefully considering the context and academic environment.

cs.DL↗

arXiv2016

How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data

In this short communication, we provide an overview of a relatively newly provided source of altmetrics data which could possibly be used for societal impact measurements in scientometrics. Recently, Altmetric - a start-up providing publication level metrics - started to make data for publications available which have been mentioned in policy-related documents. Using data from Altmetric, we study how many papers indexed in the Web of Science (WoS) are mentioned in policy-related documents. We find that less than 0.5% of the papers published in different subject categories are mentioned at least once in policy-related documents. Based on our results, we recommend that the analysis of (WoS) publications with at least one policy-related mention is repeated regularly (annually). Mentions in policy-related documents should not be used for impact measurement until new policy-related sites are tracked.

cs.DL↗

arXiv2016

Skewness of citation impact data and covariates of citation distributions: A large-scale empirical analysis based on Web of Science data

Using percentile shares, one can visualize and analyze the skewness in bibliometric data across disciplines and over time. The resulting figures can be intuitively interpreted and are more suitable for detailed analysis of the effects of independent and control variables on distributions than regression analysis. We show this by using percentile shares to analyze so-called "factors influencing citation impact" (FICs; e.g., the impact factor of the publishing journal) across year and disciplines. All articles (n= 2,961,789) covered by WoS in 1990 (n= 637,301), 2000 (n= 919,485), and 2010 (n= 1,405,003) are used. In 2010, nearly half of the citation impact is accounted for by the 10% most-frequently cited papers; the skewness is largest in the humanities (68.5% in the top-10% layer) and lowest in agricultural sciences (40.6%). The comparison of the effects of the different FICs (the number of cited references, number of authors, number of pages, and JIF) on citation impact shows that JIF has indeed the strongest correlations with the citation scores. However, the correlation between FICs and citation impact is lower, if citations are normalized instead of using raw citation counts.

cs.DL↗

arXiv2016

"Smart Girls" versus "Sleeping Beauties" in the Sciences: The Identification of Instant and Delayed Recognition by Using the Citation Angle

In recent years, a number of studies have introduced methods for identifying papers with delayed recognition (so called "sleeping beauties", SBs) or have presented single publications as cases of SBs. Most recently, Ke et al. (2015) proposed the so called "beauty coefficient" (denoted as B) to quantify how much a given paper can be considered as a paper with delayed recognition. In this study, the new term "smart girl" (SG) is suggested to differentiate instant credit or "flashes in the pan" from SBs. While SG and SB are qualitatively defined, the dynamic citation angle \b{eta} is introduced in this study as a simple way for identifying SGs and SBs quantitatively - complementing the beauty coefficient B. The citation angles for all articles from 1980 (n=166870) in natural sciences are calculated for identifying SGs and SBs and their extent. We reveal that about 3% of the articles are typical SGs and about 0.1% typical SBs. The potential advantages of the citation angle approach are explained.

cs.DL↗

arXiv2016

Expected values in percentile indicators

PP(top x%) is the proportion of papers of a unit (e.g. an institution or a group of researchers), which belongs to the x% most frequently cited papers in the corresponding fields and publication years. It has been proposed that x% of papers can be expected which belongs to the x% most frequently cited papers. In this Letter to the Editor we will present the results of an empirical test whether we can really have this expectation and how strong the deviations from the expected values are when many random samples are drawn from the database.

cs.DL↗

arXiv2016

Which early works are cited most frequently in climate change research literature? A bibliometric approach based on Reference Publication Year Spectroscopy

This bibliometric analysis focuses on the general history of climate change research and, more specifically, on the discovery of the greenhouse effect. First, the Reference Publication Year Spectroscopy (RPYS) is applied to a large publication set on climate change of 222,060 papers published between 1980 and 2014. The references cited therein were extracted and analyzed with regard to publications, which are cited most frequently. Second, a new method for establishing a more subject-specific publication set for applying RPYS (based on the co-citations of a marker reference) is proposed (RPYS-CO). The RPYS of the climate change literature focuses on the history of climate change research in total. We identified 35 highly-cited publications across all disciplines, which include fundamental early scientific works of the 19th century (with a weak connection to climate change) and some cornerstones of science with a stronger connection to climate change. By using the Arrhenius (1896) paper as a RPYS-CO marker paper, we selected only publications specifically discussing the discovery of the greenhouse effect and the role of carbon dioxide. Also, we focused on the time period 1800-1850 to reveal the contributions of J.B.J Fourier in terms of cited references. Using different RPYS approaches in this study, we were able to identify the complete range of works of the celebrated icons as well as many less known works relevant for the history of climate change research. The analyses confirmed the potential of the RPYS method for historical studies: Seminal papers are detected on the basis of the references cited by the overall community without any further assumptions.

cs.DL↗

arXiv2016

Professional and Citizen Bibliometrics: Complementarities and ambivalences in the development and use of indicators

Bibliometric indicators such as journal impact factors, h-indices, and total citation counts are algorithmic artifacts that can be used in research evaluation and management. These artifacts have no meaning by themselves, but receive their meaning from attributions in institutional practices. We distinguish four main stakeholders in these practices: (1) producers of bibliometric data and indicators; (2) bibliometricians who develop and test indicators; (3) research managers who apply the indicators; and (4) the scientists being evaluated with potentially competing career interests. These different positions may lead to different and sometimes conflicting perspectives on the meaning and value of the indicators. The indicators can thus be considered as boundary objects which are socially constructed in translations among these perspectives. This paper proposes an analytical clarification by listing an informed set of (sometimes unsolved) problems in bibliometrics which can also shed light on the tension between simple but invalid indicators that are widely used (e.g., the h-index) and more sophisticated indicators that are not used or cannot be used in evaluation practices because they are not transparent for users, cannot be calculated, or are difficult to interpret.

cs.DL↗

arXiv2016

Cited References and Medical Subject Headings (MeSH) as Two Different Knowledge Representations: Clustering and Mappings at the Paper Level

For the biomedical sciences, the Medical Subject Headings (MeSH) make available a rich feature which cannot currently be merged properly with widely used citing/cited data. Here, we provide methods and routines that make MeSH terms amenable to broader usage in the study of science indicators: using Web-of-Science (WoS) data, one can generate the matrix of citing versus cited documents; using PubMed/MEDLINE data, a matrix of the citing documents versus MeSH terms can be generated analogously. The two matrices can also be reorganized into a 2-mode matrix of MeSH terms versus cited references. Using the abbreviated journal names in the references, one can, for example, address the question whether MeSH terms can be used as an alternative to WoS Subject Categories for the purpose of normalizing citation data. We explore the applicability of the routines in the case of a research program about the amyloid cascade hypothesis in Alzheimer's disease (AD). One conclusion is that referenced journals provide archival structures, whereas MeSH terms indicate mainly variation (including novelty) at the research front. Furthermore, we explore the option of using the citing/cited matrix for main-path analysis as a by-product of the software.

cs.DL↗