For a transparent and responsible community of scientists and publishers
Just a couple of months ago nobody had heard of Covid-19. Today, Covid-19 has significantly changed our lives. The lethality of the disease – which has caused high excess death rates in many countries, and its exponential spread have had governments make extraordinary decisions. Severe containment measures curtailing civil liberties have been put in place to protect the population, such as school closures, social distancing, extended lockdowns, curfews, quarantine and surveillance. As a result, stock markets have plummeted and economies are expected to shrink at levels not seen in over 100 years. Those drastic measures were often implemented in the name of scientific advice.
The scientific community reacted quickly and forcefully to the challenge posed by Covid-19: within weeks, the genetic make-up of the virus was revealed, and asymptomatic carriers were found to spread the disease. Scientific discoveries continue to be made at unprecedented speed, and populate the front pages of major scientific outlets, including Science, Nature, Lancet and Nature Medicine, among others. Collaborative science is also on the rise, with laboratories across the globe teaming up to beat the virus. However, in light of immense productivity under high time pressure not all science has lived up to the highest standards. For example, an early high-profile paper on asymptomatic transmission had to be corrected shortly after publication as the critical anamnesis of the patient was incomplete. Clinical trials have also been criticized as many are poorly designed, e.g., lack control groups, are not randomized, or are too small to draw meaningful conclusions.
Amongst the fields that have seen both flourishing productivity and immense scrutiny is modelling. In a world of uncertainties and unknowns, but where rapid and decisive actions are needed, being able to model data to predict future developments and outcomes is key. Hence, a multitude of models forecast which control measures (or combinations thereof) could most effectively reduce the spread of the disease and ‘flatten the curve’; when such measures should be implemented; whether or not schools should be closed or reopened; whether herd immunity is a viable option, etc. Unsurprisingly, models have fueled political decisions and are used to justify political actions that affect millions of people. The impact of scientific models on society is unquestionable.
Yet, as famously stated by Paul Box, all models are wrong and furthermore, only as good as the data upon which they are built. The latter vary grossly in reliability as countries adopt different testing policies and definitions, even for basic parameters such as the numbers of active cases or reported deaths. Critical aspects of the pathogen are yet to be discovered, e.g., different modes of transmission and carriers. Hence, models have to rely on assumptions, e.g., extrapolate from other diseases or epidemiological models to enable inferences. Accordingly, strong criticism of various models and their underlying assumptions have been voiced.
How to trust scientific studies: the role of peer review
How can scientific results and models be trusted to inform the public at large and political decision makers? A gold standard for scientific quality control is peer review. Preprint servers, flooded with Covid-19 studies, prominently warn that studies are to be taken as preliminary, and should not guide clinical practice/health-related behavior as they have not been peer-reviewed. Although the review process is not without caveats, once a study has undergone thorough vetting by expert, unbiased scientists, it is deemed trustworthy. However, the urgency to battle the Covid-19 pandemic has also impacted the peer review process. Pressure on editors and reviewers is high. A quick turnaround of papers is expected because of the fast-evolving situation and the lives at stake. An additional source of pressure is competition among journals to attract high-profile papers, readership, and impact. Research von Covid-19 is being prioritized, and the average turnaround for submitted manuscripts on Covid-19 has fallen from 117 to 60 days (see [1] ) for 14 major journals including ZZ, UU, YY. Furthermore, Covid-19 is a newly emerging disease for which scientific expertise cannot yet exist, and the few experts which do exist are likely overloaded with review requests, apart from their own research. Hence, there is reason to believe that the peer review process currently does not provide the same quality assurance it usually does.
In a situation where scientific output quickly enters political decision making with far reaching consequences, additional scrutiny seems more than warranted. A simple solution that does not slow scientific progress is open review. Providing the reviews along the scientific articles would allow scientists, but also non-experts like political decision makers, to more fully evaluate the validity of the published studies. Did a proper peer review take place? [1] Were all reviewer concerns resolved? Did reviewers raise concerns that explain why some results hold under very restrictive conditions, or why a model fails while others fair better?
The public has a right and the need to have a public record from the scientific insight to the policy decision. As much as papers about Covid-19 should be available to everyone, their review process should also be opened. Open access to peer reviews can also help policy making in future pandemics, when a second wave of Covid-19 arrives or if a yet unknown disease emerges. We will appreciate that knowledge in all its forms, including the review process, to make better decision than we have now. The current pandemic and the availability of digital records provide us with another unprecedented opportunity: to act as responsible librarians, preserving knowledge and informed opinions for future generations.
When so much is at stake, full transparency would suit science well. If we stand behind the science and the peer review process as our primary means of quality control, we should not be afraid to open this process to the public. When the time comes and political decisions taken during the crisis are scrutinized, we risk losing the trust in science that is currently increasing, and that could do damage to all future policy decisions. As a community of scientist and publishers, we have a moral, and ethical responsibility. We need to raise to the challenge, and act as the guardians of knowledge for future generations. Is there something to lose? Only gains come to mind: transparency, openness and knowledge. It is time to challenge the status quo, and demand openness, and to unseal the review process.
