Application of artificial intelligence in nuclear medicine and molecular imaging: a review of current status and future perspectives for clinical translation.
Artificial intelligence; Molecular imaging; Nuclear medicine; Radiology, Nuclear Medicine and Imaging; General Medicine
Abstract :
[en] Artificial intelligence (AI) will change the face of nuclear medicine and molecular imaging as it will in everyday life. In this review, we focus on the potential applications of AI in the field, both from a physical (radiomics, underlying statistics, image reconstruction and data analysis) and a clinical (neurology, cardiology, oncology) perspective. Challenges for transferability from research to clinical practice are being discussed as is the concept of explainable AI. Finally, we focus on the fields where challenges should be set out to introduce AI in the field of nuclear medicine and molecular imaging in a reliable manner.
Disciplines :
Radiology, nuclear medicine & imaging
Author, co-author :
Visvikis, Dimitris; LaTIM, INSERM, UMR 1101, University of Brest, Brest, France
Lambin, Philippe; The D-Lab, Department of Precision Medicine, GROW - School for Oncology, Maastricht University Medical Center (MUMC +), Maastricht, The Netherlands ; Department of Radiology and Nuclear Medicine, GROW - School for Oncology, Maastricht University Medical Center (MUMC +), Maastricht, The Netherlands
Beuschau Mauridsen, Kim; Center of Functionally Integrative Neuroscience and MindLab, Department of Clinical Medicine, Aarhus University, Aarhus, Denmark ; Department of Nuclear Medicine, University of Bern, Bern, Switzerland
Hustinx, Roland ; Centre Hospitalier Universitaire de Liège - CHU > > Service médical de médecine nucléaire et imagerie onco
Lassmann, Michael; Klinik Und Poliklinik Für Nuklearmedizin, Universitätsklinikum Würzburg, Würzburg, Germany
Rischpler, Christoph; Department of Nuclear Medicine, University Hospital Essen, University of Duisburg-Essen, Essen, Germany
Shi, Kuangyu; Department of Nuclear Medicine, University of Bern, Bern, Switzerland ; Department of Informatics, Technical University of Munich, Munich, Germany
Pruim, Jan ; Medical Imaging Center, Dept. of Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands. j.pruim@umcg.nl
Language :
English
Title :
Application of artificial intelligence in nuclear medicine and molecular imaging: a review of current status and future perspectives for clinical translation.
Publication date :
09 July 2022
Journal title :
European Journal of Nuclear Medicine and Molecular Imaging
ISSN :
1619-7070
eISSN :
1619-7089
Publisher :
Springer Science and Business Media Deutschland GmbH, Germany
Visvikis D, Rest CCL, Jaouen V, Hatt M. Artificial intelligence, machine (deep) learning and radio(geno)mics: definitions and nuclear medicine imaging applications. Eur J Nucl Med Mol Imaging. 2019;46:2630–7. DOI: 10.1007/s00259-019-04373-w
Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE Inst Electr Electron Eng. 2021;86:2278–324. DOI: 10.1109/5.726791
Hinton GE, Osindero S, Teh Y-W. A fast learning algorithm for deep belief nets. Neural Comput. 2006;18:1527–54. DOI: 10.1162/neco.2006.18.7.1527
Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun Acm. 2017;60:84–90. DOI: 10.1145/3065386
Torres-Velazquez M, Chen W-J, Li X, McMillan AB. Application and construction of deep learning networks in medical imaging. IEEE Trans Radiat Plasma Med Sci. 2021;5:137–59.
McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, et al. International evaluation of an AI system for breast cancer screening. Nature. 2020;577:89–94. DOI: 10.1038/s41586-019-1799-6
Haibe-Kains B, Adam GA, Hosny A, Khodakarami F, Shraddha T, Kusko R, et al. Transparency and reproducibility in artificial intelligence. Nature. 2020;586:E14–6. DOI: 10.1038/s41586-020-2766-y
Curtis C, Liu C, Bollerman TJ, Pianykh OS. Machine learning for predicting patient wait times and appointment delays. J Am Coll Radiol. 2018;15:1310–6. DOI: 10.1016/j.jacr.2017.08.021
Hatt M, Rest CCL, Tixier F, Badic B, Schick U, Visvikis D. Radiomics: data are also images. J Nucl Med. 2019;60:38S-44S. DOI: 10.2967/jnumed.118.220582
Uribe CF, Mathotaarachchi S, Gaudet V, Smith KC, Rosa-Neto P, Bénard F, et al. Machine learning in nuclear medicine: part 1-introduction. J Nucl Med. 2019;60:451–8. DOI: 10.2967/jnumed.118.223495
Zukotynski K, Gaudet V, Uribe CF, Mathotaarachchi S, Smith KC, Rosa-Neto P, et al. Machine learning in nuclear medicine: part 2-neural networks and clinical aspects. J Nucl Med. 2021;1:22–9. DOI: 10.2967/jnumed.119.231837
Nensa F, Demircioglu A, Rischpler C. Artificial intelligence in nuclear medicine. J Nucl Med. 2019;60:29S-37S. DOI: 10.2967/jnumed.118.220590
Reader AJ, Corda G, Mehranian A, da Costa-Luis C, Ellis S, Schnabel JA. Deep learning for PET image reconstruction. IEEE Trans Radiat Plasma Med Sci. 2021;5:1–25. DOI: 10.1109/TRPMS.2020.3014786
Lee JS. A review of deep learning-based approaches for attenuation correction in positron emission tomography. IEEE Trans Radiat Plasma Med Sci. 2021;5:160–84.
Sollini M, Antunovic L, Chiti A, Kirienko M. Towards clinical application of image mining: a systematic review on artificial intelligence and radiomics. Eur J Nucl Med Mol. 2019;I(46):2656–72. DOI: 10.1007/s00259-019-04372-x
Castiglioni I, Rundo L, Codari M, Leo GD, Salvatore C, Interlenghi M, et al. AI applications to medical images: From machine learning to deep learning. Phys Medica. 2021;83:9–24. DOI: 10.1016/j.ejmp.2021.02.006
Currie G, Rohren E. Intelligent imaging in nuclear medicine: the principles of artificial intelligence, machine learning and deep learning. Semin Nucl Med. 2020;51:102–11. DOI: 10.1053/j.semnuclmed.2020.08.002
European Commission. White paper: On artificial intelligence – A European approach to excellence and trust. Brussels, 19-2-2020. COM(2020) 65 final.
Goodman SN. A comment on replication, p-values and evidence. Stat Med. 1992;11:875–9. DOI: 10.1002/sim.4780110705
Simmons JP, Nelson LD, Simonsohn U. False-positive psychology. Psychol Sci. 2011;22:1359–66. DOI: 10.1177/0956797611417632
Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016;533:452–4. DOI: 10.1038/533452a
Wei L, Osman S, Hatt M, ElNaqa I. Machine learning for radiomics-based multi-modality and multi-parametric modeling. Q J Nucl Med Mol Imaging. 2019;63:323–38. DOI: 10.23736/S1824-4785.19.03213-8
Hatt M, Parmar C, Qi J, Naqa IE. Machine (Deep) Learning methods for image processing and radiomics. IEEE Trans Radiat Plasma Med Sci. 2019;3:104–8. DOI: 10.1109/TRPMS.2019.2899538
Häggström I, Schmidtlein CR, Campanella G, Fuchs TJ. DeepPET: a deep encoder–decoder network for directly solving the PET image reconstruction inverse problem. Med Image Anal. 2019;54:253–62. DOI: 10.1016/j.media.2019.03.013
Kandarpa VSS, Bousse A, Benoit D, Visvikis D. DUG-RECON: a framework for direct image reconstruction using convolutional generative networks. IEEE Trans Radiat Plasma Med Sci. 2021;5:44–53. DOI: 10.1109/TRPMS.2020.3033172
Gong K, Guan J, Liu C-C, Qi J. PET image denoising using a deep neural network through fine tuninG. IEEE Trans Radiat Plasma Med Sci. 2019;3:153–61. DOI: 10.1109/TRPMS.2018.2877644
Mehranian A, Reader AJ. Model-based deep learning PET image reconstruction using forward-backward splitting expectation maximisation. IEEE Trans Radiat Plasma Med Sci. 2021;5:54–64. DOI: 10.1109/TRPMS.2020.3004408
Shao W, Pomper MG, Du Y. A learned reconstruction network for SPECT imaging. IEEE Trans Radiat Plasma Med Sci. 2021;5:26–34. DOI: 10.1109/TRPMS.2020.2994041
Jang H, Liu F, Zhao G, Bradshaw T, McMillan AB. Technical note: deep learning based MRAC using rapid ultrashort echo time imaging. Med Phys. 2018;45:3697–704. DOI: 10.1002/mp.12964
Hwang D, Kang SK, Kim KY, Seo S, Paeng JC, Lee DS, et al. Generation of PET attenuation map for whole-body time-of-flight 18F-FDG PET/MRI using a deep neural network trained with simultaneously reconstructed activity and attenuation maps. J Nucl Med. 2019;60:1183–9. DOI: 10.2967/jnumed.118.219493
Dong X, Lei Y, Wang T, Higgins K, Liu T, Curran WJ, et al. Deep learning-based attenuation correction in the absence of structural information for whole-body positron emission tomography imaging. Phys Med Biol. 2020;65:055011. DOI: 10.1088/1361-6560/ab652c
Visvikis D, Merlin T, Bousse A, Benoit D, Laurent B. Deep learning based scatter correction for PET imaging. Eur J Nucl Med Mol Imaging. 2020;47(Suppl 1):S484.
Xiang H, Lim H, Fessler JA, Dewaraja YK. A deep neural network for fast and accurate scatter estimation in quantitative SPECT/CT under challenging scatter conditions. Eur J Nucl Med Mol Imaging. 2020;47:2956–67. DOI: 10.1007/s00259-020-04840-9
Yang J, Park D, Sohn JH, Wang ZJ, Gullberg GT, Seo Y. Joint correction of attenuation and scatter in image space using deep convolutional neural networks for dedicated brain 18 F-FDG PET. Phys Med Biol. 2019;64:075019. DOI: 10.1088/1361-6560/ab0606
Wang Y, Zhou L, Wang L, Yu B, Zu C, Lalush DS, et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018, 21st International Conference, Granada, Spain, September 16–20, 2018, Proceedings, Part I. Lect Notes Comput Sc. 2018;11070:329–37. DOI: 10.1007/978-3-030-00928-1_38
Song T-A, Chowdhury SR, Yang F, Dutta J. PET image super-resolution using generative adversarial networks. Neural Netw. 2020;125:83–91. DOI: 10.1016/j.neunet.2020.01.029
Hatt M, Laurent B, Ouahabi A, Fayad H, Tan S, Li L, et al. The first MICCAI challenge on PET tumor segmentation. Med Image Anal. 2018;44:177–95. DOI: 10.1016/j.media.2017.12.007
Pinochet P, Eude F, Becker S, Shah V, Sibille L, Toledano MN, et al. Evaluation of an automatic classification algorithm using convolutional neural networks in oncological positron emission tomography. Front Med. 2021;8:628179. DOI: 10.3389/fmed.2021.628179
Guo Z, Li X, Huang H, Guo N, Li Q. Deep learning-based image segmentation on multimodal medical imaging. IEEE Trans Radiat Plasma Med Sci. 2019;3:162–9. DOI: 10.1109/TRPMS.2018.2890359
Wang G, Rahmim A, Gunn RN. PET parametric imaging: past, present, and future. IEEE Trans Radiat Plasma Medical Sci. 2020;4:663–75. DOI: 10.1109/TRPMS.2020.3025086
Boellaard R, Delgado-Bolton R, Oyen W, Giammarile F, Tatsch K, Eschner W, et al. FDG PET/CT: EANM procedure guidelines for tumour imaging: version 2.0. Eur J Nucl Med Mol Imaging. 2015;42:328–54. DOI: 10.1007/s00259-014-2961-x
Liang G, Fouladvand S, Zhang J, Brooks MA, Jacobs N, Chen J. GANai: Standardizing CT images using generative adversarial network with alternative improvement. BioRxiv. 2018. https://doi.org/10.1101/460188.
Kim DH, Wit H, Thurston M. Artificial intelligence in the diagnosis of Parkinson’s disease from ioflupane-123 single-photon emission computed tomography dopamine transporter scans using transfer learning. Nuc Med Commun. 2018;10:887–93. DOI: 10.1097/MNM.0000000000000890
Choi H, Kim YK, Yoon EJ, Lee J-Y, Lee DS, Initiative ADN. Cognitive signature of brain FDG PET based on deep learning: domain transfer from Alzheimer’s disease to Parkinson’s disease. Eur J Nucl Med Mol Imaging. 2020;47:403–12. DOI: 10.1007/s00259-019-04538-7
Son HJ, Oh JS, Oh M, Kim SJ, Lee J-H, Roh JH, et al. The clinical feasibility of deep learning-based classification of amyloid PET images in visually equivocal cases. Eur J Nucl Med Mol Imaging. 2020;47:332–41. DOI: 10.1007/s00259-019-04595-y
Weehaeghe DV, Devrome M, Schramm G, Vocht JD, Deckers W, Baete K, et al. Combined brain and spinal FDG PET allows differentiation between ALS and ALS mimics. Eur J Nucl Med Mol. 2020;I(47):2681–90. DOI: 10.1007/s00259-020-04786-y
Betancur J, Rubeaux M, Fuchs TA, Otaki Y, Arnson Y, Slipczuk L, et al. Automatic valve plane localization in myocardial perfusioN SPECT/CT by machine learning: anatomic and clinical validation. J Nucl Med. 2016;58:961–7. DOI: 10.2967/jnumed.116.179911
Arsanjani R, Xu Y, Hayes SW, Fish M, Lemley M, Gerlach J, et al. Comparison of fully automated computer analysis and visual scoring for detection of coronary artery disease from myocardial perfusion SPECT in a large population. J Nucl Med. 2013;54:221–8. DOI: 10.2967/jnumed.112.108969
Arsanjani R, Dey D, Khachatryan T, Shalev A, Hayes SW, Fish M, et al. Prediction of revascularization after myocardial perfusion SPECT by machine learning in a large population. J Nucl Cardiol. 2015;22:877–84. DOI: 10.1007/s12350-014-0027-x
Hu L-H, Betancur J, Sharir T, Einstein AJ, Bokhari S, Fish MB, et al. Machine learning predicts per-vessel early coronary revascularization after fast myocardial perfusion SPECT: results from multicentre REFINE SPECT registry. Eur Heart J Cardiovasc Imaging. 2019;21:549–59. DOI: 10.1093/ehjci/jez177
Betancur J, Hu L-H, Commandeur F, Sharir T, Einstein AJ, Fish MB, et al. Deep learning analysis of upright-supine high-efficiency SPECT myocardial perfusion imaging for prediction of obstructive coronary artery disease: a multicenter study. J Nucl Med: Off Publ Soc Nucl Med. 2019;60:664–70. DOI: 10.2967/jnumed.118.213538
Betancur J, Otaki Y, Motwani M, Fish MB, Lemley M, Dey D, et al. Prognostic value of combined clinical and myocardial perfusion imaging data using machine learning. JACC Cardiovasc Imaging. 2018;11:1000–9. DOI: 10.1016/j.jcmg.2017.07.024
Slart RHJA, Williams MC, Juarez-Orozco LE, Rischpler C, Dweck MR, Glaudemans AWJM, et al. Position paper of the EACVI and EANM on artificial intelligence applications in multimodality cardiovascular imaging using SPECT/CT, PET/CT, and cardiac CT. Eur J Nucl Med Mol. 2021;I(48):1399–413. DOI: 10.1007/s00259-021-05341-z
Zhao Z, Pi Y, Jiang L, Xiang Y, Wei J, Yang P, et al. Deep neural network based artificial intelligence assisted diagnosis of bone scintigraphy for cancer bone metastasis. Sci Rep-uk. 2020;10:17046. DOI: 10.1038/s41598-020-74135-4
Ntakolia C, Diamantis DE, Papandrianos N, Moustakidis S, Papageorgiou EI. A lightweight convolutional neural network architecture applied for bone metastasis classification in nuclear medicine: a case study on prostate cancer patients. Healthc. 2020;8:493. DOI: 10.3390/healthcare8040493
Wei L, ElNaqa I. AI for response evaluation with PET/CT. Semin Nucl Med. 2021;51:157–69. DOI: 10.1053/j.semnuclmed.2020.10.003
Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749–62. DOI: 10.1038/nrclinonc.2017.141
Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RGPM, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:441–6. DOI: 10.1016/j.ejca.2011.11.036
O’Connor JPB, Aboagye EO, Adams JE, Aerts HJWL, Barrington SF, Beer AJ, et al. Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol. 2017;14:169–86. DOI: 10.1038/nrclinonc.2016.162
Deist TM, Dankers FJWM, Valdes G, Wijsman R, Hsu I, Oberije C, et al. Machine learning algorithms for outcome prediction in (chemo)radiotherapy: an empirical comparison of classifiers. Med Phys. 2018;45:3449–59. DOI: 10.1002/mp.12967
Amyar A, Ruan S, Gardin I, Chatelain C, Decazes P, Modzelewski R. 3-D RPET-NET: development of a 3-D PET imaging convolutional neural network for radiomics analysis and outcome prediction. IEEE Trans Radiat Plasma Med Sci. 2019;3:225–31. DOI: 10.1109/TRPMS.2019.2896399
Hansen L, Salamon P. Neural network ensembles. IEEE Trans Pattern Anal Mach Intell. 1990;12:993–1001.
Hatt M, Rest CCL, Antonorsi N, Tixier F, Tankyevych O, Jaouen V, et al. Radiomics in PET/CT: current status and future AI-based evolutions. Semin Nucl Med. 2020;51:126–33. DOI: 10.1053/j.semnuclmed.2020.09.002
Afshar P, Mohammadi A, Plataniotis KN, Oikonomou A, Benali H. From handcrafted to deep-learning-based cancer radiomics. IEEE Signal Proc Mag. 2019;36:132–60. DOI: 10.1109/MSP.2019.2900993
Tixier F, Jaouen V, Hognon C, Gallinato O, Colin T, Visvikis D. Evaluation of conventional and deep learning based image harmonization methods in radiomics studies. Phys Med Biol. 2021;66:245009. DOI: 10.1088/1361-6560/ac39e5
Wang Y-R (Joyce), Baratto L, Hawk KE, Theruvath AJ, Pribnow A 1, Thakor ASArtificial intelligence enables whole-body positron emission tomography scans with minimal radiation exposure. Eur J Nucl Med Mol I. 2021;48:2771–81.
Ljungberg M, Gleisner KS. 3-D Image-based dosimetry in radionuclide therapy. IEEE Trans Radiat Plasma Med Sci. 2018;2:527–40. DOI: 10.1109/TRPMS.2018.2860563
Flux GD, Gleisner KS, Chiesa C, Lassmann M, Chouin N, Gear J, et al. From fixed activities to personalized treatments in radionuclide therapy: lost in translation? Eur J Nucl Med Mol Imaging. 2018;45:152–4. DOI: 10.1007/s00259-017-3859-1
Zhao Y, Gafita A, Vollnberg B, Tetteh G, Haupt F, Afshar-Oromieh A, et al. Deep neural network for automatic characterization of lesions on 68Ga-PSMA-11 PET/CT. Eur J Nucl Med Mol. 2020;I(47):603–13. DOI: 10.1007/s00259-019-04606-y
Xue S, Gafita A, Afshar-Oromieh A, Eiber M, Rominger A, Shi K. Voxel-wise prediction of post-therapy dosimetry for 177Lu-PSMA I&T therapy using deep learning. J Nucl Med. 2020;61:1424.
Lee MS, Hwang D, Kim JH, Lee JS. Deep-dose: a voxel dose estimation method using deep convolutional neural network for personalized internal dosimetry. Sci Rep-uk. 2019;9:10308. DOI: 10.1038/s41598-019-46620-y
Ataeinia B, Heidari P. Artificial intelligence and the future of diagnostic and therapeutic radiopharmaceutical development: in silico smart molecular design. Pet Clin. 2021;16:513–23. DOI: 10.1016/j.cpet.2021.06.008
Kletting P, Thieme A, Eberhardt N, Rinscheid A, D’Alessandria C, Allmann J, et al. Modeling and predicting tumor response in radioligand therapy. J Nucl Med. 2019;60:65–70. DOI: 10.2967/jnumed.118.210377
Guo R, Hu X, Song H, Xu P, Xu H, Rominger A, et al. Weakly supervised deep learning for determining the prognostic value of 18F-FDG PET/CT in extranodal natural killer/T cell lymphoma, nasal type. Eur J Nucl Med Mol. 2021;I:1–11.
Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data. 2019;6:60. DOI: 10.1186/s40537-019-0197-0
Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, et al. The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging. 2013;26:1045–57. DOI: 10.1007/s10278-013-9622-7
Quellec G, Charrière K, Boudi Y, Cochener B, Lamard M. Deep image mining for diabetic retinopathy screening. Med Image Anal. 2017;39:178–93. DOI: 10.1016/j.media.2017.04.012
Brocki L, Chung NC. Concept saliency maps to visualize relevant features in deep generative models. Arxiv [Internet]. 2019;1910.13140. https://doi.org/10.48550/arXiv/1910.13140.
O’Neil C. Weapons of math destruction. How big data increases inequality and threatens democracy. New York: Crown Publishing Group; 2016.
Bradshaw TJ, Boellaard R, Dutta J, Jha AK, Jacobs P, Li Q, et al. Nuclear medicine and artificial intelligence: best practices for algorithm development. J Nucl Med. 2022;63:500–10. DOI: 10.2967/jnumed.121.262567
