[en] Purpose: To develop handcrafted radiomics (HCR) and deep learning (DL) based automated diagnostic tools that can differentiate between idiopathic pulmonary fibrosis (IPF) and non-IPF interstitial lung diseases (ILDs) in patients using high-resolution computed tomography (HRCT) scans. Material and Methods: In this retrospective study, 474 HRCT scans were included (mean age, 64.10 years ± 9.57 [SD]). Five-fold cross-validation was performed on 365 HRCT scans. Furthermore, an external dataset comprising 109 patients was used as a test set. An HCR model, a DL model, and an ensemble of HCR and DL model were developed. A virtual in-silico trial was conducted with two radiologists and one pulmonologist on the same external test set for performance comparison. The performance was compared using DeLong method and McNemar test. Shapley Additive exPlanations (SHAP) plots and Grad-CAM heatmaps were used for the post-hoc interpretability of HCR and DL models, respectively. Results: In five-fold cross-validation, the HCR model, DL model, and the ensemble of HCR and DL models achieved accuracies of 76.2 ± 6.8, 77.9 ± 4.6, and 85.2 ± 2.7%, respectively. For the diagnosis of IPF and non-IPF ILDs on the external test set, the HCR, DL, and the ensemble of HCR and DL models achieved accuracies of 76.1, 77.9, and 85.3%, respectively. The ensemble model outperformed the diagnostic performance of clinicians who achieved a mean accuracy of 66.3 ± 6.7% (p < 0.05) during the in-silico trial. The area under the receiver operating characteristic curve (AUC) for the ensemble model on the test set was 0.917 which was significantly higher than the HCR model (0.817, p = 0.02) and the DL model (0.823, p = 0.005). The agreement between HCR and DL models was 61.4%, and the accuracy and specificity for the predictions when both the models agree were 93 and 97%, respectively. SHAP analysis showed the texture features as the most important features for IPF diagnosis and Grad-CAM showed that the model focused on the clinically relevant part of the image. Conclusion: Deep learning and HCR models can complement each other and serve as useful clinical aids for the diagnosis of IPF and non-IPF ILDs.
Disciplines :
Cardiovascular & respiratory systems Radiology, nuclear medicine & imaging
Author, co-author :
Refaee, Turkey
Salahuddin, Zohaib
Frix, Anne-Noëlle ; Centre Hospitalier Universitaire de Liège - CHU > > Service de pneumologie - allergologie
Yan, Chenggong
Wu, Guangyao
Woodruff, Henry C.
Gietema, Hester
MEUNIER, Paul ; Centre Hospitalier Universitaire de Liège - CHU > > Service médical de radiodiagnostic
LOUIS, Renaud ; Centre Hospitalier Universitaire de Liège - CHU > > Service de pneumologie - allergologie
GUIOT, Julien ; Centre Hospitalier Universitaire de Liège - CHU > > Service de pneumologie - allergologie
Lambin, Philippe
Language :
English
Title :
Diagnosis of Idiopathic Pulmonary Fibrosis in High-Resolution Computed Tomography Scans Using a Combination of Handcrafted Radiomics and Deep Learning
Coultas DB Zumwalt RE Black WC Sobonya RE. The epidemiology of interstitial lung diseases. Am J Respir Crit Care Med. (1994) 150:967–72. 10.1164/ajrccm.150.4.7921471 7921471
Cottin V. Pulmonary fibrosis: “idiopathic” is not “cryptogenic”. Eur Respir J. (2019) 53:1802314. 10.1183/13993003.02314-2018 30886022
Travis WD Costabel U Hansell DM King TE Jr. Lynch DA Nicholson AG et al. An official American Thoracic Society/European Respiratory Society statement: update of the international multidisciplinary classification of the idiopathic interstitial pneumonias. Am J Respir Crit Care Med. (2013) 188:733–48. 10.1164/rccm.201308-1483ST 24032382
Raghu G Remy-Jardin M Myers JL Richeldi L Ryerson CJ Lederer DJ et al. Diagnosis of Idiopathic Pulmonary Fibrosis. An Official ATS/ERS/JRS/ALAT Clinical Practice Guideline. Am J Respir Crit Care Med. (2018) 198:e44–68. 10.1164/rccm.201807-1255ST 30168753
Fischer A du Bois R. Interstitial lung disease in connective tissue disorders. Lancet. (2012) 380:689–98. 10.1016/S0140-6736(12)61079-4
Fernández Pérez ER Swigris JJ Forssén AV Tourin O Solomon JJ Huie TJ et al. Identifying an inciting antigen is associated with improved survival in patients with chronic hypersensitivity pneumonitis. Chest. (2013) 144:1644–51. 10.1378/chest.12-2685 23828161
Raghu G Collard HR Egan JJ Martinez FJ Behr J Brown KK et al. An official ATS/ERS/JRS/ALAT statement: idiopathic pulmonary fibrosis: evidence-based guidelines for diagnosis and management. Am J Respir Crit Care Med. (2011) 183:788–824. 10.1164/rccm.2009-040GL 21471066
Richeldi L Collard HR Jones MG. Idiopathic pulmonary fibrosis. Lancet. (2017) 389:1941–52. 10.1016/S0140-6736(17)30866-8
Ley B Collard HR King TE Jr. Clinical course and prediction of survival in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. (2011) 183:431–40. 10.1164/rccm.201006-0894CI 20935110
Guiot J Vaidyanathan A Deprez L Zerka F Danthine D Frix A-N et al. A review in radiomics: making personalized medicine a reality via routine imaging. Med Res Rev. (2022) 42:426–40. 10.1002/med.21846 34309893
Cho YH Seo JB Lee SM Lee SM Choe J Lee D et al. Quantitative CT imaging in chronic obstructive pulmonary disease: review of current status and future challenges. Korean J Radiol. (2018) 78:1. 10.3348/jksr.2018.78.1.1
Hochhegger B Marchiori E Zanon M Rubin AS Fragomeni R Altmayer S et al. Imaging in idiopathic pulmonary fibrosis: diagnosis and mimics. Clinics. (2019) 74:e225. 10.6061/clinics/2019/e225 30726312
Tominaga J Sakai F Johkoh T Noma S Akira M Fujimoto K et al. Diagnostic certainty of idiopathic pulmonary fibrosis/usual interstitial pneumonia: the effect of the integrated clinico-radiological assessment. Eur J Radiol. (2015) 84:2640–5. 10.1016/j.ejrad.2015.08.016 26391232
Walsh S de Jong EEC van Timmeren JE Ibrahim A Compter I Peerlings J et al. Decision support systems in oncology. JCO Clin Cancer Inform. (2019) 3:1–9. 10.1200/CCI.18.00001 30730766
Lambin P Rios-Velazquez E Leijenaar R Carvalho S van Stiphout RGPM Granton P et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. (2012) 48:441–6. 10.1016/j.ejca.2011.11.036 22257792
Gillies RJ Kinahan PE Hricak H. Radiomics: images Are More than Pictures, They Are Data. Radiology. (2016) 278:563–77. 10.1148/radiol.2015151169 26579733
LeCun Y Bengio Y Hinton G. Deep learning. Nature. (2015) 521:436–44. 10.1038/nature14539 26017442
Walsh SLF Calandriello L Silva M Sverzellati N. Deep learning for classifying fibrotic lung disease on high-resolution computed tomography: a case-cohort study. Lancet Respir Med. (2018) 6:837–45. 10.1016/S2213-2600(18)30286-8
Salahuddin Z Woodruff HC Chatterjee A Lambin P. Transparency of deep neural networks for medical image analysis: a review of interpretability methods. Comput Biol Med. (2021) 140:105111. 10.1016/j.compbiomed.2021.105111 34891095
Lundberg SM Lee S-I. A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, CA: Curran Associates Inc (2017). p. 4768–77.
Selvaraju RR Cogswell M Das A Vedantam R Parikh D Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision. Cambridge, MA: IEEE (2017). p. 618–26.
Zwanenburg A Vallières M Abdalah MA Aerts HJWL Andrearczyk V Apte A et al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. (2020) 295:328–38. 10.1148/radiol.2020191145 32154773
Jia T-Y Xiong J-F Li X-Y Yu W Xu Z-Y Cai X-W et al. Identifying EGFR mutations in lung adenocarcinoma by noninvasive imaging using radiomics features and random forest modeling. Eur Radiol. (2019) 29:4742–50. 10.1007/s00330-019-06024-y 30778717
Bashir U Kawa B Siddique M Mak SM Nair A Mclean E et al. Non-invasive classification of non-small cell lung cancer: a comparison between random forest models utilising radiomic and semantic features. Br J Radiol. (2019) 92:20190159. 10.1259/bjr.20190159 31166787
Jiang C Luo Y Yuan J You S Chen Z Wu M et al. CT-based radiomics and machine learning to predict spread through air space in lung adenocarcinoma. Eur Radiol. (2020) 30:4050–7. 10.1007/s00330-020-06694-z 32112116
Huang G Liu Z Van Der Maaten L Weinberger KQ. Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE (2017). p. 4700–8.
Monai Consortium. MONAI: Medical Open Network for AI. Zenodo (2020). 10.5281/zenodo.5728262
Bossuyt PM Reitsma JB Bruns DE Gatsonis CA Glasziou PP Irwig L et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. (2015) 351:h5527. 10.1136/bmj.h5527 26511519
Lambin P Leijenaar RTH Deist TM Peerlings J de Jong EEC van Timmeren J et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. (2017) 14:749–62. 10.1038/nrclinonc.2017.141 28975929
Christe A Peters AA Drakopoulos D Heverhagen JT Geiser T Stathopoulou T et al. Computer-aided diagnosis of pulmonary fibrosis using deep learning and CT images. Invest Radiol. (2019) 54:627–32. 10.1097/RLI.0000000000000574 31483764
Park JE Kim D Kim HS Park SY Kim JY Cho SJ et al. Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. Eur Radiol. (2020) 30:523–36. 10.1007/s00330-019-06360-z 31350588
Lee S Han K Suh YJ. Quality assessment of radiomics research in cardiac CT: a systematic review. Eur Radiol. (2022) 32:3458–68. 10.1007/s00330-021-08429-0 34981135
Stanzione A Gambardella M Cuocolo R Ponsiglione A Romeo V Imbriaco M. Prostate MRI radiomics: a systematic review and radiomic quality score assessment. Eur J Radiol. (2020) 129:109095.
Spadarella G Calareso G Garanzini E Ugga L Cuocolo A Cuocolo R. MRI based radiomics in nasopharyngeal cancer: systematic review and perspectives using radiomic quality score (RQS) assessment. Eur J Radiol. (2021) 140:109744. 10.1016/j.ejrad.2021.109744 33962253
Won SY Park YW Ahn SS Moon JH Kim EH Kang S-G et al. Quality assessment of meningioma radiomics studies: bridging the gap between exploratory research and clinical applications. Eur J Radiol. (2021) 138:109673. 10.1016/j.ejrad.2021.109673 33774441
Park JE Kim HS Kim D Park SY Kim JY Cho SJ et al. A systematic review reporting quality of radiomics research in neuro-oncology: toward clinical utility and quality improvement using high-dimensional imaging features. BMC Cancer. (2020) 20:29. 10.1186/s12885-019-6504-5 31924170
Ibrahim A Primakov S Beuque M Woodruff HC Halilaj I Wu G et al. Radiomics for precision medicine: current challenges, future prospects, and the proposal of a new framework. Methods. (2021) 188:20–9. 10.1016/j.ymeth.2020.05.022 32504782
Mali SA Ibrahim A Woodruff HC Andrearczyk V Müller H Primakov S et al. Making radiomics more reproducible across scanner and imaging protocol variations: a review of harmonization methods. J Pers Med. (2021) 11:842. 10.3390/jpm11090842 34575619
Graziani M Andrearczyk V Müller H. Regression concept vectors for bidirectional explanations in histopathology. In: Stoyanov D Taylor Z Mostafa Kia S Oguz I Reyes M et al. editors. Understanding and Interpreting Machine Learning in Medical Image Computing Applications. Berlin: Springer International Publishing (2018). p. 124–32. 10.1007/978-3-030-02628-8_14