A recent study published in JAMA Internal Medicine (March 2025), conducted by the Methods team in collaboration with Guillaume Cabanac, examined the presence and impact of retracted studies in systematic reviews and meta-analyses. These studies, withdrawn from the literature due to fraud or major errors, can affect the results of meta-analyses and potentially influence clinical conclusions.
The researchers identified 50 systematic reviews published in the 25 leading general medical journals, which included at least one retracted in the analysis. Most of the reviews (74%) were published before the retraction of the included studies. Overall, there were 62 retracted studies included in at least one meta-analysis. A total of 166 meta-analyses were recalculated, this time excluding the retracted studies:
- 96% of the recalculated effect estimates remained within the original confidence intervals.
- 11% had a change in the statistical significance of the results.
- 42% showed an effect size change of ≥10%, and 19% of ≥50%.
Impact on the abstracts:
- 68% of the systematic reviews reported incorrect results in their abstract due to the inclusion of retracted studies
- In 14% of the systematic reviews, the findings reported in the abstract would have undergone clinically meaningful changes in interpretation if the retracted studies had been excluded.
The results of this study demonstrate that the inclusion of retracted studies in meta-analyses can change the interpretation of results in certain situations. They also underline the importance of systematically identify systematic reviews referencing retracted studies status and determine if corrections or retractions are needed. Tools such as the “Retraction Watch” database and the automated “Feet of Clay” software can help identify retracted studies, reinforce the rigor of literature syntheses and prevent the spread of unreliable data in the scientific literature.
🔗 Article: Graña Possamai C, Cabanac G, Perrodeau E, Ghosn L, Ravaud P, Boutron I. Inclusion of Retracted Studies in Systematic Reviews and Meta-Analyses of Interventions: A Systematic Review and Meta-Analysis. JAMA Intern Med. Published online March 31, 2025. doi:10.1001/jamainternmed.2025.0256
🔗 Editorial: Gross CP, Flanagin A, Perencevich EN, Inouye SK. Mitigating the Impact of Retracted Studies in the Medical Literature. JAMA Intern Med. Published online March 31, 2025. doi:10.1001/jamainternmed.2025.0251
🔗 Feet of Clay Detector available on “Problematic Paper Screener” platform
