Towards an accurate cancer diagnosis modelization:Comparison of Random Forest strategies

Biomarker signature discovery remains the main path to develop clinical diagnostic tools when the biological knowledge for pathology is missing. Identifying optimal and clinically useful short signatures goes hand in hand with signature model robustness. The Machine Learning (ML) community has made substantial contributions to the field of biomarker signature discovery, enabling the mining of thousands of potential candidates in small to moderate numbers of subjects. One such contribution is Random Forest (RF), for which multiple implementations exist. Here, we describe two AUC hyper-stability scores as performance metrics complementary to the average AUC, namely HRS and the HSS. Such scores measure the ability of RF implementations to reproduce the same performance in the presence of data perturbation. We investigate their utility when comparing 15 Random Forest implementations. Together with a notion of run-time, these measures aid in selecting the most optimal strategy. Our findings show that the performance of a Random Forest implementation depends on the nature of the input data. No Random Forest implementation can be used universally for any binary classification and on any datasets. We recommend using particular real-life data of interest as a template for an in-depth study with a few Random Forest implementations, to be able to select the most optimal one for the data at hand, using the performance measures that were outlined in this study.