The article “Do machine learning methods lead to similar individualized treatment rules? A comparison study on real data” by Florie Bouvier (then a PhD student in the METHODS team), Etienne Peyrot (PhD student in the METHODS team), Alan Balendran (PhD student in the METHODS team), Corentin Ségalas, Ian Roberts, François Petit, and Raphaël Porcher is among the 10 most cited articles published by the journal Statistics in Medicine in 2024.
In this work, 21 algorithms for identifying individualized treatment rules (or variants of algorithms) were compared using data from two large randomized controlled trials to study the agreement between the rules identified. We show that there is relatively good agreement between algorithms belonging to the same family, but very low, or even no, agreement between very different algorithms. These algorithms are therefore not interchangeable, and particular attention must be paid to validating treatment rules derived from the analysis of randomized controlled trial data, even when those trials are large. This work also questions some naïve applications of these algorithms (for example, those that do not assess model performance in terms of calibration or discrimination).
By Raphaël Porcher
