Doctor: Ketevan Glonti
Title: Peer-review content and communication process in biomedical journals
Supervisors: Darko Hren, Erik Cobo, Isabelle Boutron
Doctoral school: ED 393 Epidemiology and Biomedical Information Sciences, Université Paris Cité
Date of thesis defense: 29/05/2020
Jury: Flaminio Squazzoni, Ozren Polašek, Lotty Hooft, Kalpana Shankar, Thed Van Leuwen, Corinne Alberti, Darko Hren, Isabelle Boutron
Summary:
Peer review of scientific articles has been continually debated and criticized since its emergence, but to date it remains a key mechanism for ensuring the scientific quality of publications and is likely to remain so for the foreseeable future. Therefore, it is important to continue exploring ways to improve the peer review process.
The consequences of failing peer review can be dramatic: the dishonest study by Wakefield et al. which escaped the peer review process of The Lancet – a world-renowned and influential biomedical journal – has increased vaccine hesitancy worldwide and has had a profound and widespread impact on the public confidence in science. The peer review process goes beyond simply controlling the quality of scientific manuscripts. In a broad sense, it is supposed to be a mechanism for self-regulation and quality assurance in the scientific field. Peer review gives published studies a seal of approval and legitimizes the underlying science as a reliable point of reference for society at large. It is therefore essential to maintain high standards of peer review, both to preserve public confidence in science and to continue to promote evidence-based (clinical) practice. This is particularly important at a time when the production of knowledge and information is rapid and extensive, and where there is a wave of “fake news” and denial of scientific evidence. At the same time, peer review is not perfect. It is essentially based on human interaction between key stakeholders, which introduces “human behavior” into the equation. It is therefore important to study it from this angle, in order to highlight the determining factors of the interaction between authors, peers and publishers.
Despite growing recognition and awareness, very little research has been conducted on content and communication in biomedical journals that may contribute to peer review problems. When I began this research in October 2016, to my knowledge, only one published study had explored the social and subjective dimensions of manuscript review in scientific biomedical publishing. The study published by Galipeau et al. presented a first attempt to systematically identify the core competencies of scientific editors of biomedical journals. I found it both intriguing and worrying that, decades after the establishment of editor-run journals, it was only in 2016 that studies recognized the need to establish the position of journal editor as a ” profession” by describing their roles and responsibilities, and reaching consensus on them, have been published. However, although progress has been made with respect to journal editors, there has been no agreement on what constitutes quality peer review or the roles and tasks that journal editors examiners must accomplish. This represents a major challenge. Lack of agreement on the roles and tasks of peer reviewers and the expected content of peer reviewer reports is an obstacle to the development of peer review quality criteria. It can also result in misunderstandings which, in turn, can hinder collaboration between stakeholders in the peer review process, impacting the quality of reviewers’ reports.
The peer review process in biomedical journals involves collaboration between authors, journal editors, and peer reviewers aimed at ensuring the dissemination of high-quality research. The most fundamental underlying principle of any successful collaboration is that stakeholders are aware of their own and others’ roles and tasks, as well as the skills needed to execute them effectively. Science is now an international enterprise; Researchers from all over the world submit their work to journals, so everyone needs to be on the same page for peer review to work effectively. Additionally, effective communication practices are essential to ensure the process runs smoothly. Although good communication practices between these actors are essential to achieve this goal, the evidence suggests that there are many flaws in the peer review process, with communication failures being at the heart of the problem. For example, existing research suggests that a critical aspect of collaboration – the mutual understanding of stakeholders’ professional roles and tasks within the process – is not communicated appropriately. This is manifested in part by the lack of consistency in peer reviewer guidelines across biomedical journals. Ineffective communication practices are also evident in the lack of transparency and considerable variation observed in the content of peer reviewer rating forms (used to evaluate original manuscripts). A study to identify the tasks that journal editors expect of peer reviewers who evaluate a manuscript that has undergone a randomized controlled trial found a significant gap between the expectations of journal editors and those of peer reviewers. This can negatively impact the quality of peer reviewer reports because the expectations of both parties are not met. These situations can be seen as a waste of resources, putting strain on an already overloaded system. Another study highlights the importance of effective communication among key stakeholders before and during peer review to avoid delays and resulting frustration. There will likely be communication issues that go beyond simple miscommunication and misunderstanding of stakeholders’ roles and job duties. For example, peer reviewers would like to receive feedback from journal editors on their reports and see comments from other peer reviewers. However, this feedback is rarely provided. Such unmet expectations caused by a lack of communication can influence the willingness and motivation of peer reviewers to participate in the process.
Despite the crucial need for in-depth research and evidence on communication between actors involved in peer review, until now the problem has not been empirically analyzed and the underlying factors have not been not been sufficiently evaluated. Through this thesis, I aim to produce relevant data that helps clarify the content and communication practices within the peer review process in scientific biomedical journals.
