[en] [en] BACKGROUND: Recently, artificial intelligence (AI)-based applications for chest imaging have emerged as potential tools to assist clinicians in the diagnosis and management of patients with coronavirus disease 2019 (COVID-19).
OBJECTIVES: To develop a deep learning-based clinical decision support system for automatic diagnosis of COVID-19 on chest CT scans. Secondarily, to develop a complementary segmentation tool to assess the extent of lung involvement and measure disease severity.
METHODS: The Imaging COVID-19 AI initiative was formed to conduct a retrospective multicentre cohort study including 20 institutions from seven different European countries. Patients with suspected or known COVID-19 who underwent a chest CT were included. The dataset was split on the institution-level to allow external evaluation. Data annotation was performed by 34 radiologists/radiology residents and included quality control measures. A multi-class classification model was created using a custom 3D convolutional neural network. For the segmentation task, a UNET-like architecture with a backbone Residual Network (ResNet-34) was selected.
RESULTS: A total of 2,802 CT scans were included (2,667 unique patients, mean [standard deviation] age = 64.6 [16.2] years, male/female ratio 1.3:1). The distribution of classes (COVID-19/Other type of pulmonary infection/No imaging signs of infection) was 1,490 (53.2%), 402 (14.3%), and 910 (32.5%), respectively. On the external test dataset, the diagnostic multiclassification model yielded high micro-average and macro-average AUC values (0.93 and 0.91, respectively). The model provided the likelihood of COVID-19 vs other cases with a sensitivity of 87% and a specificity of 94%. The segmentation performance was moderate with Dice similarity coefficient (DSC) of 0.59. An imaging analysis pipeline was developed that returned a quantitative report to the user.
CONCLUSION: We developed a deep learning-based clinical decision support system that could become an efficient concurrent reading tool to assist clinicians, utilising a newly created European dataset including more than 2,800 CT scans.
Disciplines :
Cardiovascular & respiratory systems Radiology, nuclear medicine & imaging
Author, co-author :
Topff, Laurens ; Department of Radiology, Netherlands Cancer Institute, Amsterdam, The Netherlands ; GROW School for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands
Sánchez-García, José; Quantitative Imaging Biomarkers in Medicine (Quibim), Valencia, Spain
López-González, Rafael; Quantitative Imaging Biomarkers in Medicine (Quibim), Valencia, Spain
Pastor, Ana Jiménez; Quantitative Imaging Biomarkers in Medicine (Quibim), Valencia, Spain
Visser, Jacob J ; Department of Radiology and Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
Huisman, Merel; Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, The Netherlands
Guiot, Julien ; Centre Hospitalier Universitaire de Liège - CHU > > Service de pneumologie - allergologie
Beets-Tan, Regina G H; Department of Radiology, Netherlands Cancer Institute, Amsterdam, The Netherlands ; GROW School for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands
Alberich-Bayarri, Angel; Quantitative Imaging Biomarkers in Medicine (Quibim), Valencia, Spain
Fuster-Matanzo, Almudena; Quantitative Imaging Biomarkers in Medicine (Quibim), Valencia, Spain
Ranschaert, Erik R; Department of Radiology, St. Nikolaus Hospital, Eupen, Belgium ; Ghent University, Ghent, Belgium
Imaging COVID-19 AI initiative
Language :
English
Title :
A deep learning-based application for COVID-19 diagnosis on CT: The Imaging COVID-19 AI initiative.
Mallah SI, Ghorab OK, Al-Salmi S, Abdellatif OS, Tharmaratnam T, Iskandar MA, et al. COVID-19: breaking down a global health crisis. Ann Clin Microbiol Antimicrob. 2021; 20(1):35. https://doi.org/10.1186/s12941-021-00438-7 PMID: 34006330
Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet. 2020; 395(10223):497-506.
Mercer TR, Salit M. Testing at scale during the COVID-19 pandemic. Nat Rev Genet. 2021; 22(7):415-426. https://doi.org/10.1038/s41576-021-00360-w PMID: 33948037
Filchakova O, Dossym D, Ilyas A, Kuanysheva T, Abdizhamil A, Bukasov R. Review of COVID-19 testing and diagnostic methods. Talanta. 2022; 244:123409. https://doi.org/10.1016/j.talanta.2022.123409 PMID: 35390680
Sharma S, Shrivastava S, Kausley SB, Rai B, Pandit AB. Coronavirus: a comparative analysis of detection technologies in the wake of emerging variants. Infection. 2022:1-19. https://doi.org/10.1007/s15010-022-01819-6 PMID: 35471631
Rubin GD, Ryerson CJ, Haramati LB, Sverzellati N, Kanne JP, Raoof S, et al. The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology. 2020; 296(1):172-180. https://doi.org/10.1148/radiol. 2020201365 PMID: 32255413
Inui S, Gonoi W, Kurokawa R, Nakai Y, Watanabe Y, Sakurai K, et al. The role of chest imaging in the diagnosis, management, and monitoring of coronavirus disease 2019 (COVID-19). Insights Imaging. 2021; 12(1):155. https://doi.org/10.1186/s13244-021-01096-1 PMID: 34727257
Li J, Long X, Wang X, Fang F, Lv X, Zhang D, et al. Radiology indispensable for tracking COVID-19. Diagn Interv Imaging. 2021; 102(2):69-75. https://doi.org/10.1016/j.diii.2020.11.008 PMID: 33281082
Ai T, Yang Z, Hou H, Zhan C, Chen C, Lv W, et al. Correlation of Chest CT and RT-PCR Testing for Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology. 2020; 296(2): E32-e40. https://doi.org/10.1148/radiol.2020200642 PMID: 32101510
Xie X, Zhong Z, Zhao W, Zheng C, Wang F, Liu J. Chest CT for Typical Coronavirus Disease 2019 (COVID-19) Pneumonia: Relationship to Negative RT-PCR Testing. Radiology. 2020; 296(2):E41-e45. https://doi.org/10.1148/radiol.2020200343 PMID: 32049601
Asghari A, Adeli S-H, Parham M, Bagherzade M, Ahmadpour S, Shajari R, et al. Comparing the Sensitivity and Specificity of Lung CT-scan with RT-PCR for Diagnosis of COVID-19. Current Medical Imaging. 2022; 18:1-6.
Mair MD, Hussain M, Siddiqui S, Das S, Baker A, Conboy P, et al. A systematic review and meta-analysis comparing the diagnostic accuracy of initial RT-PCR and CT scan in suspected COVID-19 patients. Br J Radiol. 2021; 94(1119):20201039. https://doi.org/10.1259/bjr.20201039 PMID: 33353381
Wang L, Gao YH, Lou LL, Zhang GJ. The clinical dynamics of 18 cases of COVID-19 outside of Wuhan, China. Europ Resp J. 2020; 55(4). https://doi.org/10.1183/13993003.00398-2020 PMID: 32139464
Zhang S, Li H, Huang S, You W, Sun H. High-resolution computed tomography features of 17 cases of coronavirus disease 2019 in Sichuan province, China. Europ Resp J. 2020; 55(4). https://doi.org/10.1183/13993003.00334-2020 PMID: 32139463
Lee EYP, Ng MY, Khong PL. COVID-19 pneumonia: what has CT taught us? Lancet Infect Dis. 2020; 20(4):384-385. https://doi.org/10.1016/S1473-3099(20)30134-1 PMID: 32105641
Shi H, Han X, Jiang N, Cao Y, Alwalid O, Gu J, et al. Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study. Lancet Infect Dis. 2020; 20(4):425-434. https://doi.org/10.1016/S1473-3099(20)30086-4 PMID: 32105637
Alsharif W, Qurashi A. Effectiveness of COVID-19 diagnosis and management tools: A review. Radiography (Lond). 2021; 27(2):682-687. https://doi.org/10.1016/j.radi.2020.09.010 PMID: 33008761
Long C, Xu H, Shen Q, Zhang X, Fan B, Wang C, et al. Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT? Eur J Radiol. 2020; 126:108961. https://doi.org/10.1016/j.ejrad.2020.108961 PMID: 32229322
Chassagnon G, Vakalopoulou M, Paragios N, Revel MP. Artificial intelligence applications for thoracic imaging. Europ Resp J. 2020; 123:108774. https://doi.org/10.1016/j.ejrad.2019.108774 PMID: 31841881
Topff L. Annotator Training-Imaging COVID-19 AI; 2023. [cited 2023, Jan 23]. Database: FigShare [Internet]. Available from: https://figshare.com/articles/media/Annotator_Training_-_Imaging_COVID19_AI/20660907/1.
Tsai EB, Simpson S, Lungren MP, Hershman M, Roshkovan L, Colak E, et al. The RSNA International COVID-19 Open Radiology Database (RICORD). Radiology. 2021; 299(1):E204-e213. https://doi.org/10.1148/radiol.2021203957 PMID: 33399506
Berta L, Rizzetto F, De Mattia C, Lizio D, Felisi M, Colombo PE, et al. Automatic lung segmentation in COVID-19 patients: Impact on quantitative computed tomography analysis. Phys Med. 2021; 87:115-122. https://doi.org/10.1016/j.ejmp.2021.06.001 PMID: 34139383
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. PyTorch: an imperative style, high-performance deep learning library. Proceedings of the 33rd International Conference on Neural Information Processing Systems: Curran Associates Inc.; 2019. p. Article 721.
Pan F, Ye T, Sun P, Gui S, Liang B, Li L, et al. Time Course of Lung Changes at Chest CT during Recovery from Coronavirus Disease 2019 (COVID-19). Radiology. 2020; 295(3):715-721. https://doi.org/10.1148/radiol.2020200370 PMID: 32053470
Boettiger C. An introduction to Docker for reproducible research. SIGOPS Oper Syst Rev. 2015; 49 (1):71-79.
Wang L, Zhang Y, Wang D, Tong X, Liu T, Zhang S, et al. Artificial Intelligence for COVID-19: A Systematic Review. Front Med (Lausanne). 2021; 8:704256. https://doi.org/10.3389/fmed.2021.704256 PMID: 34660623
Bai HX, Wang R, Xiong Z, Hsieh B, Chang K, Halsey K, et al. Artificial Intelligence Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Origin at Chest CT. Radiology. 2020; 296(3):E156-E165. https://doi.org/10.1148/radiol.2020201491 PMID: 32339081
Zhang K, Liu X, Shen J, Li Z, Sang Y, Wu X, et al. Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements, and Prognosis of COVID-19 Pneumonia Using Computed Tomography. Cell. 2020; 181(6):1423-1433.e1411. https://doi.org/10.1016/j.cell.2020.04.045 PMID: 32416069
Chen J, Wu L, Zhang J, Zhang L, Gong D, Zhao Y, et al. Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography. Scientific Reports. 2020; 10 (1):19196. https://doi.org/10.1038/s41598-020-76282-0 PMID: 33154542
Wang S, Kang B, Ma J, Zeng X, Xiao M, Guo J, et al. A deep learning algorithm using CT images to screen for Corona virus disease (COVID-19). Eur Radiol. 2021; 31(8):6096-6104. https://doi.org/10.1007/s00330-021-07715-1 PMID: 33629156
Wang S, Zha Y, Li W, Wu Q, Li X, Niu M, et al. A fully automatic deep learning system for COVID-19 diagnostic and prognostic analysis. Europ Resp J. 2020; 56(2):2000775. https://doi.org/10.1183/13993003.00775-2020 PMID: 32444412
Francone M, Iafrate F, Masci GM, Coco S, Cilia F, Manganaro L, et al. Chest CT score in COVID-19 patients: correlation with disease severity and short-term prognosis. Eur Radiol. 2020; 30(12):6808-6817. https://doi.org/10.1007/s00330-020-07033-y PMID: 32623505
Grodecki K, Lin A, Cadet S, McElhinney PA, Razipour A, Chan C, et al. Quantitative Burden of COVID-19 Pneumonia on Chest CT Predicts Adverse Outcomes: A Post-Hoc Analysis of a Prospective International Registry. Radiol Cardiothorac Imaging. 2020; 2(5):e200389. https://doi.org/10.1148/ryct. 2020200389 PMID: 33778629
Lanza E, Muglia R, Bolengo I, Santonocito OG, Lisi C, Angelotti G, et al. Quantitative chest CT analysis in COVID-19 to predict the need for oxygenation support and intubation. Eur Radiol. 2020; 30 (12):6770-6778. https://doi.org/10.1007/s00330-020-07013-2 PMID: 32591888
Yang R, Li X, Liu H, Zhen Y, Zhang X, Xiong Q, et al. Chest CT Severity Score: An Imaging Tool for Assessing Severe COVID-19. Radiol Cardiothorac Imaging. 2020; 2(2):e200047. https://doi.org/10.1148/ryct.2020200047 PMID: 33778560
Guiot J, Maes N, Winandy M, Henket M, Ernst B, Thys M, et al. Automatized lung disease quantification in patients with COVID-19 as a predictive tool to assess hospitalization severity. Front Med (Lausanne). 2022; 9:930055. https://doi.org/10.3389/fmed.2022.930055 PMID: 36106317
Diniz JOB, Quintanilha DBP, Santos Neto AC, da Silva GLF, Ferreira JL, Netto SMB, et al. Segmentation and quantification of COVID-19 infections in CT using pulmonary vessels extraction and deep learning. Multimed Tools Appl. 2021; 80(19):29367-29399. https://doi.org/10.1007/s11042-021-11153-y PMID: 34188605
Enshaei N, Oikonomou A, Rafiee MJ, Afshar P, Heidarian S, Mohammadi A, et al. COVID-rate: an automated framework for segmentation of COVID-19 lesions from chest CT images. Sci Rep. 2022; 12 (1):3212. https://doi.org/10.1038/s41598-022-06854-9 PMID: 35217712
Mergen V, Kobe A, Blüthgen C, Euler A, Flohr T, Frauenfelder T, et al. Deep learning for automatic quantification of lung abnormalities in COVID-19 patients: First experience and correlation with clinical parameters. Eur J Radiol Open. 2020;7. https://doi.org/10.1016/j.ejro.2020.100272 PMID: 33043101
Roberts M, Driggs D, Thorpe M, Gilbey J, Yeung M, Ursprung S, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat Mach Intell. 2021; 3(3):199-217.
