Prediction of Hematopoietic Stem Cell Transplantation Related Mortality- Lessons Learned from the In-Silico Approach: A European Society for Blood and Marrow Transplantation Acute Leukemia Working Party Data Mining Study
[en] Models for prediction of allogeneic hematopoietic stem transplantation (HSCT) related mortality
partially account for transplant risk. Improving predictive accuracy requires understating of prediction limiting factors, such as the statistical methodology used, number and quality of features collected, or simply the population size. Using an in-silico approach (i.e., iterative computerized simulations), based on machine learning (ML) algorithms, we set out to analyze these factors. A cohort of 25,923 adult acute leukemia patients from the European Society for Blood and Marrow Transplantation (EBMT) registry was analyzed. Predictive objective was non-relapse mortality (NRM) 100 days following HSCT. Thousands of prediction models were developed under varying conditions: increasing sample size, specific subpopulations and an increasing number of variables, which were selected and ranked by separate feature selection algorithms. Depending on the algorithm, predictive performance plateaued on a population size of 6,611–8,814 patients, reaching a maximal area under the receiver operator characteristic curve (AUC) of 0.67. AUCs’ of models developed on specific subpopulation ranged from 0.59 to 0.67 for patients in second complete remission and receiving reduced intensity conditioning, respectively. Only 3–5 variables were necessary to achieve near maximal AUCs. The top 3 ranking variables, shared by all algorithms were disease stage, donor type, and conditioning regimen. Our findings empirically demonstrate that with regards to NRM prediction, few variables “carry the weight” and that traditional HSCT data has been “worn out”. “Breaking through” the predictive boundaries will likely require additional types of inputs.
Prediction of Hematopoietic Stem Cell Transplantation Related Mortality- Lessons Learned from the In-Silico Approach: A European Society for Blood and Marrow Transplantation Acute Leukemia Working Party Data Mining Study
Publication date :
2016
Journal title :
PLoS ONE
eISSN :
1932-6203
Publisher :
Public Library of Science, United States - California
Copelan EA. Hematopoietic stem-cell transplantation. The New England journal of medicine. 2006; 354 (17):1813-26. PMID: 16641398.
Gratwohl A, Hermans J, Goldman JM, Arcese W, Carreras E, Devergie A, et al. Risk assessment for patients with chronic myeloid leukaemia before allogeneic blood or marrow transplantation. Chronic Leukemia Working Party of the European Group for Blood and Marrow Transplantation. Lancet. 1998; 352(9134):1087-92. PMID: 9798583.
Gratwohl A, Stern M, Brand R, Apperley J, Baldomero H, De Witte T, et al. Risk Score for Outcome After Allogeneic Hematopoietic Stem Cell Transplantation A Retrospective Analysis. Cancer. 2009; 115(20):4715-26. doi: 10.1002/cncr.24531 WOS:000270740900011. PMID: 19642176
Barba P, Martino R, Perez-Simon JA, Fernandez-Aviles F, Castillo N, Pinana JL, et al. Combination of the Hematopoietic Cell Transplantation Comorbidity Index and the European Group for Blood and Marrow Transplantation score allows a better stratification of high-risk patients undergoing reduced-toxicity allogeneic hematopoietic cell transplantation. Biol Blood Marrow Transplant. 2014; 20(1):66-72. doi: 10.1016/j.bbmt.2013.10.011 PMID: 24141006.
Nakaya A, Mori T, Tanaka M, Tomita N, Nakaseko C, Yano S, et al. Does the Hematopoietic Cell Transplantation Specific Comorbidity Index (HCT-CI) Predict Transplantation Outcomes? A Prospective Multicenter Validation Study of the Kanto Study Group for Cell Therapy. Biol Blood Marrow Transplant. 2014; 20(10):1553-9. doi: 10.1016/j.bbmt.2014.06.005 PMID: 25034961.
Parimon T, Au DH, Martin PJ, Chien JW. A risk score for mortality after allogeneic hematopoietic cell transplantation. Annals of internal medicine. 2006; 144(6):407-14. PMID: 16549853.
Raimondi R, Tosetto A, Oneto R, Cavazzina R, Rodeghiero F, Bacigalupo A, et al. Validation of the Hematopoietic Cell Transplantation-Specific Comorbidity Index: A prospective, multicenter GITMO study. Blood. 2012; 120(6):1327-33. PMID: 22740454. doi: 10.1182/blood-2012-03-414573
Versluis J, Labopin M, Niederwieser D, Socie G, Schlenk RF, Milpied N, et al. Prediction of non-relapse mortality in recipients of reduced intensity conditioning allogeneic stem cell transplantation with AML in first complete remission. Leukemia. 2015; 29(1):51-7. doi: 10.1038/leu.2014.164 WOS:000347673700006. PMID: 24913728
Shouval R, Labopin M, Bondi O, Mishan-Shamay H, Shimoni A, Ciceri F, et al. Prediction of Allogeneic Hematopoietic Stem-Cell Transplantation Mortality 100 Days After Transplantation Using a Machine Learning Algorithm: A European Group for Blood and Marrow Transplantation Acute Leukemia Working Party Retrospective Data Mining Study. Journal of Clinical Oncology. 2015:JCO. 2014.59. 1339.
Breiman L. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science. 2001; 16(3):199-231.
Shouval R, Bondi O, Mishan H, Shimoni A, Unger R, Nagler A. Application of machine learning algorithms for clinical predictive modeling: A data-mining approach in SCT. Bone marrow transplantation. 2014; 49(3):332-7. doi: 10.1038/bmt.2013.146 PMID: 24096823
Shouval R, Nagler A, Labopin M, Unger R. Interpretable Boosted Decision Trees for Prediction of Mortality Following Allogeneic Hematopoietic Stem Cell Transplantation. J Data Mining Genomics Proteomics. 2015; 6(4):2. 13./MED-AB Forms Manual: Http://www.ebmt.org/Contents/Data-Management/Registrystructure/MEDABdatacollectionforms/Documents/MED-ABFormsManual.pdf.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD Statement. BMC medicine. 2015; 13(1):1.
Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and elaboration. Annals of internal medicine. 2015; 162(1):W1-W73. doi: 10.7326/M14-0698 PMID: 25560730
Breiman L. Random Forests. Machine Learning. 2001; 45(1):5-32.
Freund Y, Mason L, editors. The alternating decision tree learning algorithm. ICML; 1999.
Freund Y, Schapire R, Abe N. A short introduction to boosting. Journal-Japanese Society For Artificial Intelligence. 1999; 14(771-780):1612.
Han J, Kamber M, Pei J. Data Mining: Concepts and Techniques. 3rd ed: Morgan Kaufmann; 2012.
Hastie T, Tibshirani R, Friedman J, Hastie T, Friedman J, Tibshirani R. The elements of statistical learning: Springer; 2009.
Krogh A. What are artificial neural networks? Nature biotechnology. 2008; 26(2):195-7. PMID: 18259176. doi: 10.1038/nbt1386
Shouval R, Bondi O, Mishan H, Shimoni A, Unger R, Nagler A. Application of machine learning algorithms for clinical predictive modeling: A data-mining approach in SCT. Bone marrow transplantation. 2014; 49(3):332-7. doi: 10.1038/bmt.2013.146 PMID: 24096823.
Hall MA, Holmes G. Benchmarking attribute selection techniques for discrete class data mining. Knowledge and Data Engineering, IEEE Transactions on. 2003; 15(6):1437-47.
Cornelissen JJ, Gratwohl A, Schlenk RF, Sierra J, Bornhauser M, Juliusson G, et al. The European LeukemiaNet AML Working Party consensus statement on allogeneic HSCT for patients with AML in remission: An integrated-risk adapted approach. Nature reviews Clinical oncology. 2012; 9(10):579-90. doi: 10.1038/nrclinonc.2012.150 PMID: 22949046.
Cornelissen JJ, Van Putten WL, Verdonck LF, Theobald M, Jacky E, Daenen SM, et al. Results of a HOVON/SAKK donor versus no-donor analysis of myeloablative HLA-identical sibling stem cell transplantation in first remission acute myeloid leukemia in young and middle-aged adults: Benefits for whom? Blood. 2007; 109(9):3658-66. doi: 10.1182/blood-2006-06-025627 PMID: 17213292.
Koreth J, Schlenk R, Kopecky KJ, Honda S, Sierra J, Djulbegovic BJ, et al. Allogeneic stem cell transplantation for acute myeloid leukemia in first complete remission: Systematic review and meta-analysis of prospective clinical trials. Jama. 2009; 301(22):2349-61. PMID: 19509382. doi: 10.1001/jama.2009. 813
Yanada M, Matsuo K, Emi N, Naoe T. Efficacy of allogeneic hematopoietic stem cell transplantation depends on cytogenetic risk for acute myeloid leukemia in first disease remission: A metaanalysis. Cancer. 2005; 103(8):1652-8. doi: 10.1002/cncr.20945 PMID: 15742336.
Socié G, Stone JV, Wingard JR, Weisdorf D, Henslee-Downey PJ, Bredeson C, et al. Long-term survival and late deaths after allogeneic bone marrow transplantation. New England Journal of Medicine. 1999; 341(1):14-21. PMID: 10387937
Gratwohl A, Stern M, Brand R, Apperley J, Baldomero H, De Witte T, et al. Risk score for outcome after allogeneic hematopoietic stem cell transplantation: A retrospective analysis. Cancer. 2009; 115 (20):4715-26. PMID: 19642176. doi: 10.1002/cncr.24531
Sorror ML, Giralt S, Sandmaier BM, De Lima M, Shahjahan M, Maloney DG, et al. Hematopoietic cell transplantation specific comorbidity index as an outcome predictor for patients with acute myeloid leukemia in first remission: Combined FHCRC and MDACC experiences. Blood. 2007; 110(13):4606-13. doi: 10.1182/blood-2007-06-096966 PMID: 17873123; PubMed Central PMCID: PMCPMC2234788.
Teixeira GM, Bittencourt H, De Macedo AV, Martinho GH, Colosimo EA, Rezende SM. Assessing the Influence of Different Comorbidities Indexes on the Outcomes of Allogeneic Hematopoietic Stem Cell Transplantation in a Developing Country. PloS one. 2015; 10(9):e0137390. doi: 10.1371/journal.pone. 0137390 PMID: 26394228.
Sorror ML, Storb RF, Sandmaier BM, Maziarz RT, Pulsipher MA, Maris MB, et al. Comorbidity-age index: A clinical measure of biologic age before allogeneic hematopoietic cell transplantation. J Clin Oncol. 2014; 32(29):3249-56. doi: 10.1200/JCO.2013.53.8157 PMID: 25154831; PubMed Central PMCID: PMCPMC4178523.
Dohner H, Weisdorf DJ, Bloomfield CD. Acute Myeloid Leukemia. The New England journal of medicine. 2015; 373(12):1136-52. doi: 10.1056/NEJMra1406184 PMID: 26376137.
Lauer MS, D'Agostino RB Sr., The randomized registry trial-the next disruptive technology in clinical research? The New England journal of medicine. 2013; 369(17):1579-81. PMID: 23991657. doi: 10. 1056/NEJMp1310102
Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random survival forests. The Annals of Applied Statistics. 2008:841-60.
Hsich E, Gorodeski EZ, Blackstone EH, Ishwaran H, Lauer MS. Identifying important risk factors for survival in patient with systolic heart failure using random survival forests. Circulation: Cardiovascular Quality and Outcomes. 2011; 4(1):39-45.