Doctor: Mauricia Davidson
Title: Integration of various types of information in living systematic reviews
Supervisors: Isabelle Boutron, Anna Chaimani
Doctoral school: ED 393 Epidemiology and Biomedical Information Sciences, Université Paris Cité
Date of thesis defense: : 30/10/2024
Jury : Florian NAUDET, PU-PH, Université de Rennes, rapporteur, Anna-Bettina HAIDICH, Professeur, Aristotle University of Thessaloniki, rapporteuse, Ana MARUŠIĆ, Professeur, University of Split, examinatrice, Jérôme LAMBERT, PU-PH, Université Paris Cité, examinateur, Anna CHAIMANI, CR-HDR, Université Paris Cité, co-directrice de thèse, Isabelle BOUTRON, PU-PH, Université Paris Cité, directrice de thèse
Abstract
The landscape of scientific research is complex, offering a wealth of information from various sources including journal articles and grey literature. Living systematic reviews provide comprehensive and continuously updated summaries of the literature as new evidence emerges. Questions remain about the reliability of data from informal sources. My research aims to investigate the benefits and risks of integrating these types of information into traditional processes.
To achieve this aim, I first considered the influence of publication type on treatment effect. I conducted a meta-epidemiological study to evaluate whether summary treatment effect estimates differ between preprint and journal article trials. From this study, I did not find an important difference between summary treatment effects of preprints and summary treatment effects of journal articles.
Second, building on the findings of the first study, I evaluated the consistency in effect estimates between preprint and subsequent journal article trials, for a one-to-one comparison of the different publication types of the same trial. I found effect estimates to be generally consistent between preprints and subsequent journal articles. Also, the main results and interpretation did not change in any trial. Nevertheless, few trials had a minor discrepancy in effect estimate, and some trial outcomes were added and deleted in the journal article.
Overall, based on the results of these first two studies, I considered that in the context of a fast-moving pandemic, incorporating preprint results may be reasonable, once caution is taken to assess risk of bias and completeness of reporting.
Third, given the limitations of peer review, I conducted a qualitative study to assess the role of systematic reviews and post-publication peer reviews in the identification of methodological and reporting issues of trials. Through risk of bias and outcome reporting bias assessments, systematic reviewers identified issues in the majority of trials that could be easily resolved by trial authors. Post-publication peer review poorly identified key issues in research quality. From this study, I proposed a feedback loop between systematic reviewers and trial authors to supplement peer review, as well as a method for incorporating post- preprint peer review into the formal workflow.
Finally, I investigated the consistency of outcome reporting between RCTs with results available in clinical trial registries and the final published report. Preliminary analysis showed that the majority of the data is inconsistent.
All data in this thesis concern COVID-19 trials from the COVID-NMA living systematic review.
In conclusion, this thesis showcases the importance and utility of different types of information, and emphasized the need to streamline all data sources to improve the reliability and robustness of evidence synthesis. It also suggests a framework for creating an evidence ecosystem with strong links between research enterprises.
Keywords: preprint, post-publication peer review, clinical trial registry, risk of bias, living systematic review
