The Past

Abstract

The first part of the book will discuss the past of altmetrics, its origins, its scientific roots, and its connection with bibliometrics and webometrics. In many aspects the past of altmetrics is also the past of bibliometrics and webometrics, but it needs to be emphasized that the beginning of altmetrics does not mean the end of bibliometrics or webometrics. The three research areas are developing side by side, learning from each other, complementing each other. This part of the book will give an overview of scholarly communication and the research methods involved in "counting, measuring, and weighing" it, namely bibliometrics and more recently, after the advent of the web, webometrics to analyze scholarly communication on the web. The shortcomings and pitfalls of bibliometrics in research evaluation will be discussed and the current standards and practices for most reliable bibliometric analyses will be presented. With that, the technical developments and societal changes that paved the way for altmetrics will be presented. This part will end by focusing on developments in social media, which, as an increasingly important place for scholarly communication, has made altmetrics possible.

Keywords

scholarly communication

bibliometrics

scientometrics

informetrics

methods

bibliometric laws

citation analysis

co-citation analysis

co-word analysis

social network

Goodhart's law: When a measure becomes the target, it ceases to be a good measure(Goodhart, 1975)

1 Introduction

From increasing competition of funding comes the need to find relevant and reliable metrics to help funders decide the most deserving targets for increasingly limited finances. Different aspects of scholars' and researchers' work are being assessed to find researchers with the most potential to make an impact on science and society as a whole. Faculty members' work usually consists of three tasks or three different areas of activities: (1) research, (2) teaching, and (3) public outreach. Productivity in research can be measured from the number of research publications, but in order to assess the quality of that research some other means are required. Teaching activities could be assessed, for instance, from the number of courses, tutoring hours, and student feedback, and activities related to public outreach could be measured from the number of public talks given or appearances in news media. Nevertheless, usually only research activities matter, or at least, matter most.

Some researchers focus full heartedly on research, while others may have more passion for teaching or public outreach. The expectations on research productivity may even vary from one discipline to the other (Dewett & DeNisi, 2004). Yet when evaluating researchers' productivity and performance, most emphasis is usually placed on research activities. This of course depends to some extent on the purpose of the evaluation, but in general, even candidates for an academic position with mostly teaching duties are in the Humboldtian university system mainly evaluated on their publications. Evaluation of researchers has traditionally focused very tightly around the formal research publications, which have of course benefited more productive researchers, at the cost of those scholars that have focused more on teaching or public outreach. This usually means that bad teachers with great publication records are chosen over great teachers with bad publication records.

As the number of researchers and the number of their research products have grown significantly in the last decades, it is increasingly difficult to assess the work of researchers, yet alone the work of research groups, universities, or disciplines, by peer review alone. In peer review the work of a researcher or researchers is evaluated by their peers, preferably experts in the same area of research, who familiarize themselves with the research products of those being evaluated and then give their assessment, depending on the purpose of the evaluation. While this might be possible to do when assessing a couple of candidates for a faculty position, it is not possible to do at any larger scale. It would simply take too long and be too costly to perform. The peer review therefore needs help from methods that can assess large quantities of research products quickly. Enter bibliometrics, the research area that investigates literature using quantitative methods.

In line with the increasing need for quantitative methods for research assessment, interest towards bibliometrics has also increased. The number of scientific publications about bibliometrics has increased almost 100-fold in the last 20 years, from only 97 publications indexed by the Web of Science in 1994, to a staggering 8731 publications in 2014 (Figure 1.1). This alone demonstrates the immense increase in the interest towards bibliometrics, and, with that, research assessment in recent years.

Numerous bibliometric indicators reveal to us a story about the desire to quantitatively assess different aspects of research work and scholarly communication, aspects that go beyond productivity alone. More recently that story includes new data sources and disruptive ideas that may change scholarly communication and research assessment for good. These ideas are the result of some technical developments and earlier research about science in itself and about scholarly communication as the backbone of scientific development.

2 Scholarly Communication

Robert K. Merton wrote in 1942 that "the institutional goal of science is the extension of certified knowledge." A single researcher's formal contributions to the extension of this certified knowledge or common knowledge are his or her scientific publications and in every discipline the formal scientific publications are of utmost importance. It is also through publications that the researcher positions himself or herself in the fields of science and it is through the publications that the researcher claims intellectual ownership of his or her ideas, of his or her contributions to the extension of knowledge. These contributions can, however, also be of an informal nature, such as conversations during coffee breaks at conferences or more recently, for instance, on Twitter or other social networking sites. They can also take other forms, such as datasets and software. Both the formal and informal contributions to the common knowledge are part of scholarly communication, but they also define and position the researcher in relation and in comparison to other researchers.

Scholarly communication is the process that starts with a research idea that may be acquired from reading the work of other researchers and that certainly builds upon the work of others. This is followed by the research work and writing of the manuscript, and ends with a formal scientific publication that through peer review is accepted as an "extension of certified knowledge" (Figure 2.1), or, in fact, ends with the scientific knowledge being used in society in, for instance, policy-making or product development. All discussions related to the research idea, presentations and seminars, online or offline, between the moment of getting the research idea and when the manuscript is published as a scientific publication, are part of either informal or formal scholarly communication. Although there are many aspects to scholarly communication, the formal scientific publication is crucial in every discipline. Kircz (1998) writes that "the scientific article is the object around which the whole fabric of writing, publishing, and reading is centered." While the scientific article is the repository of knowledge, citations place the article in time and connect it to earlier research.

By referencing earlier work the author gives credit to those before him and shows how his work builds upon that of other researchers. Bernal (1971) wrote that "the methods of the scientist would be of little avail if he had not at his disposal an immense stock of previous knowledge and experience." Bernal (1971) continues by describing the cumulative nature of science as "an ever-growing body of knowledge built of sequences of the reflections and ideas, but even more of the experience and actions, of a great stream of thinkers and workers." With that Bernal eloquently summarizes how a researcher is dependent on the work of those before him and how this intellectual debt or connection to earlier work is indicated through citations. Similarly, assuming that the publication is good enough, it will be used and recognized by other researchers who give credit to it by referencing it in new scientific publications, starting the process of scholarly communication all over again (Figure 2.1).

The references given and citations received are the links between earlier research and current research. They draw up the timeline and map developments of science. They are also part of the scientific reward system that stems from the...