Comparison of the individualized treatment rules developed from different methods in two randomized controlled trials
Identifying patients who will benefit from specific treatments is an essential component of personalized medicine, which allows the construction of individual treatment rules (ITRs). ITRs, in essence, are decision rules that recommend treatment based on patients’ characteristics. Various methods for constructing ITRs have been proposed over the last decade. However, it remains unclear how these methods compare in performance and whether the derived ITRs lead to the same treatment recommendations for the same individuals.
The CRESS’s (Methods team) Florie Bouvier recently published an original research article in Statistics in Medicine that reports the results of a study exploring this issue, completed as part of Florie’s PhD. The study compared 22 methods for constructing ITRs using data from two randomized control trials (ie, the International Stroke Trial and the CRASH-3 trial), assessed the performance of ITRs derived via different computation methods, and measured the pairwise agreement between these different ITRs.
The results highlighted substantial discrepancies among the ITRs generated by the different computation methods, often leading to different treatment recommendations for the same patients. The choice of method can thus significantly impact treatment recommendations, raising some concerns about their practical use. Additionally, the study found that the overall performance of ITRs was relatively limited. It suggested that simpler approaches or the careful incorporation of specific interactions between predefined treatment-effect modifiers and the treatment might be more effective than the complex ITR algorithms currently in use, which risk overfitting the treatment effect heterogeneity.
This study calls for caution in the use of machine learning in developing personalized treatment strategies. It also underscores the importance of ongoing innovation and validation in the field to ensure that personalized medicine can fulfill its potential to tailor treatments to individual patient needs effectively.
By Florie Bouvier
https://doi.org/10.1002/sim.10059
Florie Bouvier is a 4th year PhD candidate in Biostatistics with the CRESS’ METHODS team, currently finishing her thesis under the supervision of Prof Raphaël Porcher and Dr Anna Chaimani. Florie’s primary research focus lies in deriving individualized treatment effects from RCT data to create personalized treatment rules. The recently published article represents Florie’s second thesis project.
