Interview: Open Access Preprints Boost Article Citations and Mentions

we were talking with Nicholas Fraser

In a recently published study (PDF) a team of researchers of ZBW – Leibniz Information Centre for Economics and GESIS – Leibniz Institute for the Social Sciences analysed if preprints are associated with increased citations and sharings of journal articles. Nicholas Fraser, postdoctoral researcher at ZBW – Leibniz Information Centre for Economics, explains their findings to us.

What was the topic of your recently published research on preprints?

Our new study was undertaken as part of the “OASE” project, where we are aiming to understand how open access to scientific publications influences their impact, in terms of citations or indicators of engagement on online platforms (collectively termed “altmetrics”). In this study we were interested in determining whether preprints, as a form of open access, are associated with increased citations and altmetric indicators, and what factors might be driving this effect. Similar studies have been carried out before, but they have focused on arXiv which is a well-established preprint repository in the Physics and Mathematics communities.

The findings have not yet been transferred to other research areas where depositing preprints is a relatively new practice – we decided to test this by looking at bioRxiv, a rapidly growing preprint server aimed at researchers in the biological sciences.

So, what happens if journal articles are first being published as preprints?

There were two main effects we observed in our study. The first is that when a preprint is posted on bioRxiv, it is shared widely online (for instance on social media, such as Twitter). More surprisingly, we also found that preprints are often being cited directly in the scientific literature, despite the fact that they have never been ‘approved’ by a journal peer review process. Such findings may add to discussions on the role of peer review in maintaining the integrity of the scientific record.

The second main effect we observed is that once a preprint has been peer-reviewed and published in a journal, the journal article subsequently receives more citations and more online attention than articles published at the same time and in the same journals that were never submitted as a preprint. Some of the past studies on arXiv have suggested that this ‘citation advantage’ is a short-lived effect, where the citation rates of articles submitted as preprints receive a short-term boost in citations, but long-term citation rates do not show any difference. In our study we found a different effect – even after three years following publication in a journal, citation rates of those papers which were first posted on bioRxiv remained more than fifty percent higher than those that were not.

What else did you find out?

A number of other interesting results came out of our study. For example, when looking at indicators of online sharing, we found that preprints and their associated journal articles are shared in similar amounts on Twitter and on blogs, but preprints were covered much less in mainstream news articles and on Wikipedia than journal articles. An interpretation of this is that scientists are comfortable sharing results in ‘informal’ social networks, but that ‘formal’ media outlets are more careful to share only peer-reviewed research.

We also tried to account for other factors that might influence the relationship between the preprint status of an article and its citations or altmetrics, such as the seniority or country of an author. In doing this we found a number of unexpected results, for example, female authors were underrepresented in bioRxiv submissions when compared to papers not submitted to bioRxiv. The reason for this is not yet clear, but should warrant future research. Ultimately, we found that even when all of these factors were taken into account, articles that were submitted as preprints were still much more highly cited and shared online than those not submitted as preprints.

Can the results be generalized? Can you transfer them to other research disciplines?

In general our results agree with previous studies based on arXiv, showing that papers receive more citations if they are posted as preprints. I would stop short of saying this can be generalized to all research disciplines, as different communities have very different publishing and sharing behaviours. Certainly this could be the focus of future work, to investigate the effect in other subject areas and preprint repositories. Over the past few years there has been an explosion in the number of new preprint repositories, largely driven by the Open Science Framework (OSF), so as these repositories mature in the coming years they will become interesting targets for bibliometrics researchers.

How strong is the influence of the research topic? Are preprints on “hot” current topics being more cited or mentioned?

We did not specifically investigate the influence of ‘hot’ topics in this paper, although we did look at the role of the journal impact factor in influencing citations and altmetrics. One might expect that more exciting new topics tend to be published in the highest impact journals which try to select for the most novel findings, and so the preprint citation and altmetric advantage may be driven primarily by those few high impact journals. Our results did not support this though, and found that the effect applies generally across the spectrum of journals. What is clear from our results is that posting a preprint significantly speeds up the time to public dissemination of a study compared to the traditional journal publication route. In fast-moving, ‘hot’ fields, scientists will surely want to take advantage of this to stake their claim to discoveries as early as possible, and so preprints may represent a selection of authors’ most cutting-edge or highest quality research. Trying to account for this potential bias in author selection of articles to submit as preprints will also be a challenge for future research.

More information:
Preprints boost article citations and mentions – The benefits of showing your hand early.