Conceptualizing and assessing the robustness of healthcare algorithms

The use of artificial intelligence (AI) and machine learning (ML) algorithms in healthcare for diagnostic and decision-support tools requires special attention. The concept of ethical, trustworthy, or responsible AI emphasizes the development of solutions that consider various intrinsic aspects of AI, such as generalizability, interpretability, fairness, reproducibility, and robustness. The latter aspect pertains to evaluating the performance of a model under perturbations.

Studies have shown that models are generally vulnerable to small perturbations, sometimes imperceptible to humans. One of the key challenges in implementing AI in clinical practice is to develop algorithms that are resilient to the most likely perturbations encountered in healthcare and to define appropriate evaluation methods.
To understand the influence of perturbations on model behavior, it is necessary to establish tests to assess the robustness of a model. However, there are many ways to perturb a model (input errors, different domains, adversarial attacks, etc.), and this can occur at various stages in the life of an algorithm (data collection, training or validation of the algorithm, deployment).
This PhD project on the robustness of ML algorithms in healthcare is divided into three successive stages. The first stage aims to identify various existing concepts and aspects grouped under the term “robustness” of an ML algorithm, as well as the metrics and methods used to evaluate or quantify them. The second stage aims to group and prioritize the identified concepts of robustness based on categories of use cases (frequency of occurrence, severity, ease to remediate). Finally, the last stage involves constructing a pipeline to assess the robustness of ML algorithms in healthcare.