Doctor: Anthony Chauvin
Title: Development and evaluation of interventions to improve the peer-review process in biomedical research
Supervisors: Isabelle Boutron, David Moher
Doctoral school: ED 393 Epidemiology and Biomedical Information Sciences, Université Paris Cité
Date of thesis defense: 20/09/2018
Jury: Bruno Giraudeau, Yonathan Freund, Jean-Damien Ricard, Frédéric Adnet, Isabelle Boutron
Summary:
The peer-review process is a cornerstone of biomedical research publication. However, studies have shown that peer-reviewers’ evaluations are often discordant, that they fail to detect fraud, errors and that their evaluation is influenced by the results.
The objective of this thesis work was to explore and rethink the process of peer-review editorial. As a first step, we identified the tasks expected of peer-reviewers when evaluating a randomized controlled trial report and compared the importance of these tasks from the points of view of peer-reviewers and editors. Our study showed that the most important tasks for peer reviewers were not congruent with the tasks most often requested by journal editors in theirs guidelines to reviewers.
In a second step, we conducted a systematic review and a meta-analysis of randomized controlled trials evaluating an intervention aimed at improving the quality of peer-review in biomedical journals. We identified 22 randomized controlled trials that evaluated 5 interventions. These trials included a small number of participants, were mostly monocentric and had biases that limited the interpretation of their results.
Finally, we have developed and evaluated a tool to help achieve the peer-review of randomized trials associated with online training. This tool is specifically aimed at evaluating the quality of the reporting that means the compliance with the recommendations of the CONSORT statement (i.e., international recommendations required by publishers to report randomized trials in a complete and transparent manner). We compared the performance of young trained peer- reviewers using the tool with the performance of reviewers evaluating the manuscript as part of the usual peer-review process. We included 120 randomized controlled trials. Our results showed that the tool improved the number of items correctly classified per manuscript by young peer-reviewers. This finding suggests that the use of a CONSORT checklist focusing on specific items could improve the quality of the peer-review process, and thus the quality of the published manuscript
At the end of this work, we propose to rethink the peer-review process that could be carried out in 2 stages with junior peer-reviewers assessing compliance with the recommendations of the CONSORT statement and experienced reviewers evaluating the methodology and relevance of the question.
