Science Checker: Open Access and Artificial Intelligence Help Verify Claims

An Interview with Sylvain Massip

What is the Science Checker?

In July 2021, the Science Checker went online in a beta version. In this version, it only deals with health topics. As a first step, it is intended to help science journalists and other scientific fact checkers to test the likelihood of a claim. 3 million Open Access articles from PubMed (out of 36 million) serve as the data basis for the Open Source tool. It uses artificial intelligence to check whether a claim is supported, discussed or rejected by the scientific literature. As a result, it shows how many and which documents it has found on the topic, when they were published and to what extent they make the claim probable. The guiding question is always: “What does the research literature say about this? In the practical operation of the Science Checker, three fields must first be filled in: agent, effect (increase, cause, prevent, cure) and disease.

To make the Science Checker more imaginable, here are a few examples: “Does caffeine lead to more intelligence?” (unfortunately unlikely). For this question, the tool finds three sources in the database.

Science Checker

There are 5933 sources on the question whether smoking causes cancer. Of these, 80% are confirmatory and 20% negative. For the question “Does sport prevent heart attacks?” the Science Checker finds 420 sources, of which only the first 20 relevant ones are included in the first probability calculation. Click on “Add” to add the next 20 articles or on “All” to calculate the total. Since the latter takes some time, a notification is sent out by e-mail as soon as the result is available.

We have already introduced the idea behind the Science Checker in the article “Opscidia: Fighting Fake News via Open Access”. To get a practical impression of the tool’s possibilities, we recommend simply trying it out yourself: to the Science Checker.

In this interview, we talk to Sylvain Massip, one of five members of the Science Checker team, about his experiences during the first five months that the tool has been online in a beta version. He explains who the Science Checker is aimed at, how it is financed and what contribution libraries can make.

What happened since you introduced your idea to use Open Access (OA) to fight fake news a year ago here at ZBW MediaTalk?

We have now developed a beta version of the Science Checker, which is available and usable by everyone online. It is a tool for journalists and fact-checkers which works with an Artificial Intelligence (AI) pipeline that retrieves the articles of interest for the request of the user, and classifies them as supporting and contradicting the original claim entered by the user, or neutral in some cases. The data used for the Science Checker comes from a dump of Open Access articles from Europe PMC.

For whom did you create the Science Checker?

Our first targeted audience are the scientific journalists and scientific fact-checkers.
But the Science Checker carries the will of Opscidia to make scientific literature more and more accessible beyond academic circles only. That’s why we aim for a larger use of it, open to all curious people.

The Science Checker has been launched in July 2021. So, it is online for about five months now. What are your first experiences and feedback? What were your biggest challenges?

Since the release of the Science Checker, it has been tried by more than 400 people. It is indeed a relatively slow uptake, but that was to be expected with a beta version. Our main challenge now is to find the right partners to help us in two aspects: increasing the accuracy of the tool and its growth potential.

What role does Artificial Intelligence play?

In simple words, AI is trained by our developers to be able to read articles, to understand it and to get the essential information out of it. Thanks to this upstream process, the AI used in the Science Checker will analyse millions of articles in a very short time in order to give you an answer based on many different sources of information.

Why is it so important that there are practical application examples for the use of OA?

Open Access is an important issue of our era. The free diffusion of academic knowledge is of paramount importance for many topics, from sanitary crisis to sustainable development. The OA community has to show the real value of it, that its activity is useful even outside of academia and related to global challenges. Open Access should not stay a topic for academic activists, it should spread for the common good.

You told us that there are now five people working on the Science Checker. How is it financed? Who pays the bill?

Yes, there are indeed five people who took part in the project, but in different ways. One main developer has shaped the Science Checker, Loic Rakotoson, who worked for more than four months full time on it. But he is not the only developer who has worked on it actually. Frejus Laleye and Timothée Babinet have developed part of the code used by the Science Checker. Charles Letaillieur, Opscidia’s CTO, has managed the project technically and Sylvain Massip, Opscidia’s CEO, has done most of the scientific design. In addition, together with Enzo Rodrigues, I also did a lot of work for the promotion of the Science Checker in conferences and on social media.

Financially, this beta version of the Science Checker was developed as a project, which is now over. This project was funded by the Vietsch Foundation, that we would like to thank warmly for their support.

We see it as a first step, and now that we have done a successful first draft, we are looking for the funding of our next iteration to keep the process going and build a second version of our Science Checker.

How can you guarantee its sustainability?

For the time being, we try to ensure its sustainability by focusing our maintenance on the very most important things, as we are doing it with Opscidia’s own funds. But in the future, our goal is to have a major partner, such as a large media company, in order to fund its development, communication and maintenance.

How can libraries and information infrastructures support you? Which role do they play in the project?

First of all, libraries can fund Open Science, Opscidia’s and others, to ensure that initiatives such as the Science Checker have the data they need. Indeed, we are directly dependent on information sources, their quantity and their quality.

They can also help us by spreading the word about the Science Checker and Opscidia’s other activities, and of course, we are happy to partner with any interested party for the continuation of the project.

Are you still looking for partners/support for the Science Checker? Who? How can you be supported?

We want to continue to develop the Science Checker to improve the results and optimise its performance. It is also possible that we will have to develop additional features to the tool, if we identify other needs for the user. Thus, we continue to seek funding to help us in this direction. Moreover, we are quite open to potential technological partnerships if they are relevant for the evolution of the Science Checker. Furthermore, anyone can support us by providing feedback on its use. This is an essential source of information for us and has very often allowed us to match our tools to the needs of the users.

Is your system open and can be (re-)used by others?

Yes, absolutely. Our system is totally open, but we do not own the data. It comes from the Europe PMC database. The system is Open Source, the source code is accessible for anybody in our Github and can be reused freely as long as it is for non-commercial applications.

What is your vision for the Science Checker? Where do you see it in, say, five years?

In terms of software development, the next objectives are to increase the size of the dataset that we use, make it more precise and more general. By that, we mean that we aim for a tool capable of doing the same work for all scientific fields and not just medical sciences.

In terms of usage, we want to partner with major media so that our Science Checker could be used on a daily basis for fact checking purposes.

This might also interest you:

• Opscidia: Fighting Fake News via Open Access
• Interview: Open Access Preprints boost Article Citations and Mentions
• Open Access: AfricArXiv facilitates knowledge exchange between Africa and Europe
• Open Access goes Barcamp, Part 1: A new networking opportunity for the Open Access community