Review: How the Economic Impacts of Open Science are Being Measured

by Birgit Fingerle

Among the reasons for fostering open science are its potential economic benefits. But does open science really lead to economic benefits? An article examining “The Economic Impacts of Open Science: A Rapid Evidence Assessment” was published on 1 July 2019. It researches the economic impacts of open science and their contextual factors by systematically reviewing existing studies. The result: evidence is patchy and diverse. One of the reasons is that using the outputs of open science does not often leave an obvious trace, especially outside of the academic context. So, it is often necessary to conduct interviews or surveys to obtain data. This often leads to a low significance due to methodological reasons, like small, non-random samples of participants or the difficulties of attributing specific economic impacts to the usage of open science outputs.

The result: evidence is patchy and diverse. One of the reasons is that using the outputs of open science does not often leave an obvious trace, especially outside of the academic context. So, it is often necessary to conduct interviews or surveys to obtain data. This often leads to a low significance due to methodological reasons, like small, non-random samples of participants or the difficulties of attributing specific economic impacts to the usage of open science outputs.

Furthermore, it should be noted, that all the economic evaluations of public/private open research collaborations found are focused on the life sciences. All in all, the data situation seems to be insufficient which results in limited evidence for economic benefits. Nevertheless, the rapid evidence assessment gives a broad overview of existing studies and interesting insights into the economic success of open science projects.

How can economic impacts of Open Science be measured?

A diverse range of research approaches was identified in the rapid evidence assessment. Among them are:

• Using surveys to ask open data users how much time they spend searching for data and how this amount of time is reduced by opening data. In the next step salary estimates are used to assess the savings in labour cost.
• Asking for the willingness to pay for access to currently free services to estimate the value of free resources. When subtracting willingness to pay from use value, this gives insight into the consumer surplus.
• Asking users to estimate for instance the time spent preparing data for upload or the time spent working with data they had downloaded, and to which extent they would be able to perform their work without having access to the open data. Combining this data with estimates of salary costs, gives insight into the investment in preparing or using open data.

Approaches like these have a significant potential for measurement errors and errors in assuming costs. Moreover often several approaches are combined in one study. This heterogeneity of methods also makes it harder to compare the findings of different studies. As empirical evidence on economic impacts of open science, more broadly is often collected in the form of case studies, the results are difficult to generalize but nevertheless include interesting insights.

What economic benefits does Open Science have?

The review focuses on the direct economic impacts of open science in the economy in general, like changes in productivity, competitiveness, employment, income, and value. Thus, it is not restricted to the ecosystem of scholarly communication. Indirect benefits are out of the review’s scope also they are seen as potentially very important, like open science contributing to new drug development and thereby to a rise in economic productivity through health improvements.

The review identified different ways open science leads to economic impacts which can be broadly subdivided into efficiency (getting a set output from research or innovation by using less input) and enablement (open science approaches leading to activities with economic impact which would probably not have occurred in a closed environment).

Efficiency benefits of open science

Economic benefits classified as “efficiency” can be:

• Savings in access costs: cost-savings in the university or publishing ecosystem are expected as a result of open access, despite significant costs caused for instance by the transition to some models. In addition, open access could lead to reduced costs for firms which access research results or increase the availability of research results. According to one study found in the review which summed the costs and savings associated with accessing research findings under open or paid access models, for the sector of higher education in the UK open access would have saved £813–1,180 per journal article in comparison to toll access. This would have been a saving of £80–116m per year (in 2007). But these figures are disputed, for example by questioning how the publishing was estimated. Furthermore, the costs of transitioning to open access might be influenced by higher article processing charges as recent research suggests. This might result in a negative impact on the net benefits.
• Savings in labour costs (or productivity improvements): The time it takes for researchers or firms to access research outputs is reflected in labour costs. It is assumed that accessing closed research outputs can take more time than accessing open outputs. One survey with a non-representative sample in Denmark researched how much could theoretically be saved by removing all access barriers. The actual time saving due to open access was not tested though. Open access can also foster text and data mining. In comparison to manual work this can reduce the time taken to extract useful information from sources, but it is difficult to estimate the possible amount of time. One study cited suggests a saving of over 4.7 million working hours per year.
• Savings in transaction costs: Open access can reduce transaction costs, especially the costs and time required to reach agreements necessary to access data or publications. These savings in transaction costs also apply to text and data mining. Collaborative research projects avoid direct costs and labour costs for negotiating agreements. One study has estimated that the costs of agreements can run into the hundreds of thousands of dollars for one single collaboration. So, collaborations between multiple institutions have considerable potential for transaction cost savings.

Other efficiency benefits of open science include: duplication of work is avoided which is causing high costs and is estimated to often occur in closed research, because teams work separately in different companies not knowing each other working on the same thing. The closed nature of the research makes it hard to estimate the extent of this. Redundancy in research is also a result of the general under-publication of ‘null’ results. Open approaches such as the pre-registration of trials could help to ameliorate the situation, because they would reduce duplication of ineffective approaches.

Enablement as a benefit of Open Science

Enablement benefits arise in the form of new outputs and work that could not otherwise have been undertaken:

• New Products/Services/Companies: No economy- or industry-wide studies measuring to what extent new products, services or companies were formed were found, but a number of case studies revealing the existence of such mechanisms.
• Collaborations: Open science can bring collaborative research to life that otherwise might not have been.
• Permitting work: open science outputs can permit further research that would not otherwise have been possible. For instance, a survey which was done as part of the evaluation of the European Bioinformatics Institute revealed that 45% of the respondents could not have found the data they access through the institute’s repository themselves, nor have created it themselves. So, work which is based on this data can be viewed as additional, because it is only permitted by the institute’s existence. It can also be assumed that open science permits enhanced text- and data-mining capabilities which can result in discovering previously hidden connections between different areas of research.

Barriers and costs of Open Science

The review also examined whether there was evidence of negative economic impacts caused by open science approaches. There were many examples identified where open science caused specific additional costs or different costs, for instance extra costs of preparing datasets for publication or paying article processing charges. But there were no negative overall value estimates found.

Beyond those direct costs there are a lot of barriers hampering the realization of the full benefits of open science or leading to even negative effects. For example, a lack of skills capacity in search, interpretation and text mining within institutions required to make use of open science outputs leads to little exploiting of the potential benefits. Another barrier seems to be a lack of clarity about the benefits of open science outputs. But none of the studies included in the review tried to evaluate these effects.

Another topic is possible indirect benefits or disadvantages of open science practices to researchers. One potential conflict arises from institutions expecting publishing in certain journals, which do not offer open access. In addition, researchers may fear a lack of credit for following open science approaches. The economic welfare of the research community could also be diminished due to disincentives connected to the sharing of data and publishing their papers as soon as possible. Data sharing requirements could even act as a disincentive to its collection.

Recommendations

The review also captured recommendations for maximising the positive economic impacts of
open science. They include:

• Promoting and supporting new open collaborations
• Simplifying text and data mining
• Developing an open access findings/data portal targeted at businesses
• Producing aligned positions and guidance on open science and commercialization
• Continuing support for open research data repositories
• Continuing research into new metrics and incentives
• Collecting more data on open science usage, users and costs

Further information:

The Economic Impacts of Open Science: A Rapid Evidence Assessment