Open science: Managing Research Data Using the ZBW Journal Data Archive

Domestic and foreign research funding bodies and the European Commission are increasingly demanding more open access to research data as part of a drive towards open science. Quality-assured research data is a cornerstone of scientific knowledge and can form the basis for further research, regardless of why it is collected. Yet scientific quality assurance and research integrity are also key reasons behind these recommendations: Empirical research should be replicable – and the published findings, methods used and calculation methods should be comprehensible and verifiable. The ZBW – Leibniz Information Centre for Economics helps economics journals make this a reality with their new Journal Data Archive.

A lesson for journals: Guidelines on handling research data

The case of US top economists Carmen Reinhart and Kenneth Rogoff highlights the significance of open science and the public provision of research data for quality assurance and research integrity. Their research findings on the relationship between state debt and economic growth caused a stir in the world of politics in 2010 but were cast into doubt (Link in German) after a student carefully checked the Excel file containing the two economists’ figures and calculations.

The debate surrounding Rogoff and Reinhart’s paper on the relationship between state debt and economic growth was not the first instance of incorrect calculations sparking discussions in the scientific community and public sphere – and it certainly won’t be the last.

Nevertheless, it is particularly unpleasant for journals when a published article has to be withdrawn at a later stage as a result of incorrect calculations. Even if such calculation errors are not an everyday occurrence in the scientific and research fields, they nevertheless demonstrate how crucial it is to make calculation codes and data available to the scientific debate wherever legally possible. Journals play a key role in this process and can take simple measures to enforce “good scientific practice”. Another side benefit: Authors check published research findings more carefully when the data and calculation methods are published alongside the article.

Data Availability Policy
It is the policy of the American Economic Association to publish papers only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication.

Prior to publication of an accepted article, journals should therefore first of all develop and implement guidelines which require the authors of empirical studies, simulations and experimental scientific research to submit (where legally possible) the calculation and/or programming code for their calculations along with additional information and descriptions (such as a codebook or at least a meaningful readme file). A good example of this type of guideline may be found in the journals of the American Economic Association.

ZBW data archive makes data provision and replication easier

If a guideline for handling data and programming code is initially agreed by a journal, the next question which arises concerns where data can be stored and made available. With the Journal Data Archive, the ZBW offers a free solution specifically for economic journals. The ZBW Journal Data Archive is the main digital outcome of the EDaWaX (“European Data Watch Extended”) project, supported by the German Research Foundation (DFG).

The workflow how data, programming code and descriptions are included in a journal’s digital archive is made quick and easy for journals: First of all, the editorial team registers the authors’ email addresses in the system. The authors are then automatically informed that a user account has been set up for them. They may then save the replication data for their articles in the Journal Data Archive and include their own descriptions.

Once all files are saved, the system shall inform the editorial team that the replication data has been stored. The editorial team or a reviewer then assesses the plausibility of the stored data and metadata and determines whether the research data guideline requirements have been met. If this is the case, all that remains for the editorial team to do is to add the article’s page numbers, edition number and URL and/or DOI (Link in German) to the metadata. The data adjusted by the authors shall then be allocated their own DOI and are thereby able to be cited and easy to find.

No need for editorial teams to run their own data servers

Using the ZBW Data Archive has a range of benefits for journals: Editorial teams do not have to set up and run their own server for the data archive or purchase corresponding IT services. Instead, editorial teams can look after their core responsibilities and leave the technical aspect to ZBW’s information experts. All journals receive detailed information prior to using the Data Archive and may arrange intensive training on request. Handbooks and documentation on software use are available in German and English for authors and editorial teams.

Journals supporting open science and research integrity

Introducing guidelines on handling research data and establishing a data archive is another way for journals to show that they are taking the current scientific policy debate on research integrity and open science seriously: That is why the European Commission was already encouraging member states to take measures to “be able to identify data sets easily and link them with other data sets and publications” back in 2012. The ZBW Journal Data Archive can also make this happen.

Articles with associated replication files also reflect a journal’s high scientific standards. It’s not an accident that many of the journals with corresponding guidelines are among the very best.

Studies from other scientific disciplines also indicate that articles with replication data are cited more frequently. Furthermore, the journal and published articles also benefit from increased visibility, as more sources reference the article. The replication data stored in the ZBW Journal Data Archive can also be referenced and directly linked with the articles in discipline-specific portals.

Created in autumn 2016, the ZBW Journal Data Archive has so far been put to productive use by two renowned journals, the German Economic Review and the Journal of Economics and Statistics. Other journals have also declared their interest in using the service.