Doctor: Lorenzo Bertizzolo
Title: Improving risk of bias assessment and quality of reporting in clinical research
Supervisors: Agnès Dechartres, Patrick Bossuyt
Doctoral school: ED 393 Epidemiology and Biomedical Information Sciences, Université Paris Cité
Date of thesis defense: 09/06/2021
Jury: Jörg Meerpohl, Emilie Sbidian, Matthieu Resche-Rigon, Marcel Dijkgraaf, Marc Besselink, Agnès Dechartres, Patrick Bossuyt
Summary:
Research waste is a common problem in biomedical research. High risk of bias and poor quality of reporting are two main causes of waste, which can partly explain the lack of reproducibility of research findings.
The research reported in this thesis aimed to explore different questions related to risk of bias (RoB) assessment and inadequate reporting in clinical research. It consists of five different projects; two of them focused on sources of disagreements when assessing RoB for randomized controlled trials (RCTs) that were included in two different systematic reviews (SRs). A third project evaluated associations between RCT characteristics, including RoB items as well as other features, and reported treatment effects. The other projects explored the quality of reporting in observational studies, through an evaluation of the knowledge and adherence to reporting guidelines and an analysis of the use of causal language in abstracts.
In the first two projects, our results confirm the suboptimal agreement when assessing risk of bias using the RoB tool, with researchers agreeing 70% of the times. The most common reason of disagreements was a different interpretation of the same information and this was most frequently related to incomplete or unclear reporting in the study report. We also evaluated whether external factors related to the review, the authors, or the study itself may influence the risk of bias judgement in case of incomplete or unclear reporting in the study report. Review groups and authors that had completed a lower number of SRs assigned significantly more often a low risk of bias where others reported “unclear”. The study year of publication, compared to the publication date of other studies included in the review, was also associated with a different RoB judgment.
In the third project, evaluating 132 RCTs on testosterone therapy in men, we found no RCT characteristic significantly associated with reported treatment effects. Although trials rated at high overall RoB reported treatment effects that were 21% larger compared with trials rated at low RoB, the difference was not statistically significant.
Regarding observational research, we conducted a survey on the knowledge and use of STROBE, a reporting guideline dedicated to observational studies. Researchers who conducted an observational study were recruited and contacted through various channels (contact with editors, corresponding authors of relevant studies using an observational design, social media). Among 1015 participants, 62.6% had used STROBE in the past while 19% had never heard it.
The last study examined the use of causal language in 60 abstracts of observational studies published in the BMJ and found use of causal language in 48% of them, which outlines the need for more guidance.
The projects reported in this thesis explored different questions related to risk of bias and quality of reporting, across different methodologies and topics. The results suggest possible ways of improvement. Better guidance could improve agreement in the RoB assessment in case of incomplete information. Initiatives could be taken to improve adherence to reporting guidelines, as shown in the study on STROBE. Our work on use of causal language in observational research confirms that clarity and consistency on methodology of research is needed.
