ComBat harmonization; image harmonization; radiomics reproducibility; reconstruction kernel; Medicine (miscellaneous)
Abstract :
[en] Handcrafted radiomics features (HRFs) are quantitative features extracted from medical images to decode biological information to improve clinical decision making. Despite the potential of the field, limitations have been identified. The most important identified limitation, currently, is the sensitivity of HRF to variations in image acquisition and reconstruction parameters. In this study, we investigated the use of Reconstruction Kernel Normalization (RKN) and ComBat harmonization to improve the reproducibility of HRFs across scans acquired with different reconstruction kernels. A set of phantom scans (n = 28) acquired on five different scanner models was analyzed. HRFs were extracted from the original scans, and scans were harmonized using the RKN method. ComBat harmonization was applied on both sets of HRFs. The reproducibility of HRFs was assessed using the concordance correlation coefficient. The difference in the number of reproducible HRFs in each scenario was assessed using McNemar's test. The majority of HRFs were found to be sensitive to variations in the reconstruction kernels, and only six HRFs were found to be robust with respect to variations in reconstruction kernels. The use of RKN resulted in a significant increment in the number of reproducible HRFs in 19 out of the 67 investigated scenarios (28.4%), while the ComBat technique resulted in a significant increment in 36 (53.7%) scenarios. The combination of methods resulted in a significant increment in 53 (79.1%) scenarios compared to the HRFs extracted from original images. Since the benefit of applying the harmonization methods depended on the data being harmonized, reproducibility analysis is recommended before performing radiomics analysis. For future radiomics studies incorporating images acquired with similar image acquisition and reconstruction parameters, except for the reconstruction kernels, we recommend the systematic use of the pre- and post-processing approaches (respectively, RKN and ComBat).
Disciplines :
Radiology, nuclear medicine & imaging
Author, co-author :
Refaee, Turkey ; The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands ; Department of Diagnostic Radiology, Faculty of Applied Medical Sciences, Jazan University, Jazan 45142, Saudi Arabia
Salahuddin, Zohaib; The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands
Widaatalla, Yousif; The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands
Primakov, Sergey; The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands ; Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, 6200 Maastricht, The Netherlands
Woodruff, Henry C ; The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands ; Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, 6200 Maastricht, The Netherlands
Hustinx, Roland ; Centre Hospitalier Universitaire de Liège - CHU > > Service médical de médecine nucléaire et imagerie onco
Mottaghy, Felix M; Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, 6200 Maastricht, The Netherlands ; Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
Ibrahim, Abdalla ; The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands ; Department of Radiology, Columbia University Irving Medical Center, New York, NY 10032, USA
Lambin, Philippe ; The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands ; Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, 6200 Maastricht, The Netherlands
Language :
English
Title :
CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features.
ERC - European Research Council [BE] DCS - Dutch Cancer Society [NL]
Funding text :
The authors acknowledge financial support from ERC advanced grant (ERC-ADG-2015 n° 694812—Hypoximmuno), ERC-2020-PoC: 957565-AUTO.DISTINCT. The authors also acknowledge financial support from the European Union’s Horizon 2020 research and innovation program under grant agreement: ImmunoSABR n° 733008, MSCA-ITN-PREDICT n° 766276, CHAIMELE-ON n° 952172, EuCanImage n° 952103, JTI-IMI2-2020-23-two-stage IMI-OPTIMA n° 101034347 and TRANSCAN Joint Transnational Call 2016 (JTC2016 CLEARLY n° UM 2017-8295).Funding: The authors acknowledge financial support from ERC advanced grant (ERC-ADG-2015 n◦ 694812—Hypoximmuno), ERC-2020-PoC: 957565-AUTO.DISTINCT. The authors also acknowledge financial support from the European Union’s Horizon 2020 research and innovation program under grant agreement: ImmunoSABR n◦ 733008, MSCA-ITN-PREDICT n◦ 766276, CHAIMELE-ON n◦ 952172, EuCanImage n◦ 952103, JTI-IMI2-2020-23-two-stage IMI-OPTIMA n◦ 101034347 and TRANSCAN Joint Transnational Call 2016 (JTC2016 CLEARLY n◦ UM 2017-8295).
Walsh, S.; de Jong, E.E.C.; van Timmeren, J.E.; Ibrahim, A.; Compter, I.; Peerlings, J.; Sanduleanu, S.; Refaee, T.; Keek, S.; Larue, R.T.H.M.; et al. Decision support systems in oncology. JCO Clin. Cancer Inform. 2019, 3, 1–9. [CrossRef] [PubMed]
Gillies, R.J.; Kinahan, P.E.; Hricak, H. Radiomics: Images are more than pictures, they are data. Radiology 2016, 278, 563–577. [CrossRef]
Lambin, P.; Rios-Velazquez, E.; Leijenaar, R.; Carvalho, S.; van Stiphout, R.G.P.M.; Granton, P.; Zegers, C.M.L.; Gillies, R.; Boellard, R.; Dekker, A.; et al. Radiomics: Extracting more information from medical images using advanced feature analysis. Eur. J. Cancer 2012, 48, 441–446. [CrossRef]
Yip, S.S.F.; Aerts, H.J.W.L. Applications and limitations of radiomics. Phys. Med. Biol. 2016, 61, R150–R166. [CrossRef] [PubMed]
Hosny, A.; Aerts, H.J.; Mak, R.H. Handcrafted versus deep learning radiomics for prediction of cancer therapy response. Lancet Digit. Health 2019, 1, e106–e107. [CrossRef]
Aerts, H.J.W.L.; Velazquez, E.R.; Leijenaar, R.T.H.; Parmar, C.; Grossmann, P.; Carvalho, S.; Bussink, J.; Monshouwer, R.; Haibe-Kains, B.; Rietveld, D.; et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat. Commun. 2014, 5, 4006. [CrossRef]
Bae, S.; Choi, Y.S.; Ahn, S.S.; Chang, J.H.; Kang, S.-G.; Kim, E.H.; Kim, S.H.; Lee, S.-K. Radiomic MRI phenotyping of glioblastoma: Improving survival prediction. Radiology 2018, 289, 797–806. [CrossRef] [PubMed]
Oikonomou, A.; Khalvati, F.; Tyrrell, P.N.; Haider, M.A.; Tarique, U.; Jimenez-Juan, L.; Tjong, M.C.; Poon, I.; Eilaghi, A.; Ehrlich, L.; et al. Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy. Sci. Rep. 2018, 8, 4003. [CrossRef]
Wu, W.; Parmar, C.; Grossmann, P.; Quackenbush, J.; Lambin, P.; Bussink, J.; Mak, R.; Aerts, H.J.W.L. Exploratory study to identify radiomics classifiers for lung cancer histology. Front. Oncol. 2016, 6, 71. [CrossRef] [PubMed]
Blüthgen, C.; Patella, M.; Euler, A.; Baessler, B.; Martini, K.; von Spiczak, J.; Schneiter, D.; Opitz, I.; Frauenfelder, T. Computed tomography radiomics for the prediction of thymic epithelial tumor histology, TNM stage and myasthenia gravis. PLoS ONE 2021, 16, e0261401. [CrossRef]
Linning, E.; Lu, L.; Li, L.; Yang, H.; Schwartz, L.H.; Zhao, B. Radiomics for Classification of Lung Cancer Histological Subtypes Based on Nonenhanced Computed Tomography. Acad. Radiol. 2019, 26, 1245–1252. [CrossRef]
Stefan, P.-A.; Puscas, M.E.; Csuak, C.; Lebovici, A.; Petresc, B.; Lupean, R.; Mihu, C.M. The utility of texture-based classification of different types of ascites on magnetic resonance. J. BUON 2020, 25, 1237–1244. [PubMed]
Csutak, C.; Ștefan, P.-A.; Lupean, R.-A.; Lenghel, L.M.; Mihu, C.M.; Lebovici, A. Computed tomography in the diagnosis of intraperitoneal effusions: The role of texture analysis. Bosn. J. Basic Med. Sci. 2021, 21, 488–494. [CrossRef] [PubMed]
Horvat, N.; Veeraraghavan, H.; Khan, M.; Blazic, I.; Zheng, J.; Capanu, M.; Sala, E.; Garcia-Aguilar, J.; Gollub, M.J.; Petkovska, I. MR imaging of rectal cancer: Radiomics analysis to assess treatment response after neoadjuvant therapy. Radiology 2018, 287, 833–843. [CrossRef] [PubMed]
Tharmalingam, H.; Tsang, Y.M.; Alonzi, R.; Beasley, W.; Taylor, N.J.; McWilliam, A.; Padhani, A.; Choudhury, A.; Hoskin, P.J. Changes in magnetic resonance imaging radiomic features in response to androgen deprivation therapy in patients with intermediate-and high-risk prostate cancer. Clin. Oncol. 2022. [CrossRef] [PubMed]
Ștefan, P.-A.; Lupean, R.-A.; Mihu, C.M.; Lebovici, A.; Oancea, M.D.; Hîțu, L.; Duma, D.; Csutak, C. Ultrasonography in the diagnosis of adnexal lesions: The role of texture analysis. Diagnostics 2021, 11, 812. [CrossRef] [PubMed]
Ștefan, R.-A.; Ștefan, P.-A.; Mihu, C.M.; Csutak, C.; Melincovici, C.S.; Crivii, C.B.; Maluțan, A.M.; Hîțu, L.; Lebovici, A. Ultrasonography in the differentiation of endometriomas from hemorrhagic ovarian cysts: The role of texture analysis. J. Pers. Med. 2021, 11, 611. [CrossRef]
Lohmann, P.; Bousabarah, K.; Hoevels, M.; Treuer, H. Radiomics in radiation oncology—Basics, methods, and limitations. Strahlenther. Onkol. 2020, 196, 848–855. [CrossRef] [PubMed]
Ibrahim, A.; Primakov, S.; Beuque, M.; Woodruff, H.C.; Halilaj, I.; Wu, G.; Refaee, T.; Granzier, R.; Widaatalla, Y.; Hustinx, R.; et al. Radiomics for precision medicine: Current challenges, future prospects, and the proposal of a new framework. Methods 2020, 188, 20–29. [CrossRef]
Mali, S.A.; Ibrahim, A.; Woodruff, H.C.; Andrearczyk, V.; Müller, H.; Primakov, S.; Salahuddin, Z.; Chatterjee, A.; Lambin, P. Making radiomics more reproducible across scanner and imaging protocol variations: A review of harmonization methods. J. Pers. Med. 2021, 11, 842. [CrossRef] [PubMed]
Midya, A.; Chakraborty, J.; Gönen, M.; Do, R.K.G.; Simpson, A.L. Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility. J. Med. Imaging 2018, 5, 011020. [CrossRef] [PubMed]
Reiazi, R.; Abbas, E.; Famiyeh, P.; Rezaie, A.; Kwan, J.Y.Y.; Patel, T.; Bratman, S.V.; Tadic, T.; Liu, F.-F.; Haibe-Kains, B. The impact of the variation of imaging parameters on the robustness of computed tomography radiomic features: A review. Comput. Biol. Med. 2021, 133, 104400. [CrossRef] [PubMed]
Espinasse, M.; Pitre-Champagnat, S.; Charmettant, B.; Bidault, F.; Volk, A.; Balleyguier, C.; Lassau, N.; Caramella, C. CT texture analysis challenges: Influence of acquisition and reconstruction parameters: A comprehensive review. Diagnostics 2020, 10, 258. [CrossRef] [PubMed]
Zhao, B. Understanding sources of variation to improve the reproducibility of radiomics. Front. Oncol. 2021, 11, 633176. [CrossRef] [PubMed]
Granzier, R.W.Y.; Ibrahim, A.; Primakov, S.; Keek, S.A.; Halilaj, I.; Zwanenburg, A.; Engelen, S.M.E.; Lobbes, M.B.I.; Lambin, P.; Woodruff, H.C.; et al. Test-retest data for the assessment of breast MRI radiomic feature repeatability. J. Magn. Reson. Imaging 2021. [CrossRef]
Shiri, I.; Abdollahi, H.; Shaysteh, S.; Mahdavi, S.R. Test-retest reproducibility and robustness analysis of recurrent glioblastoma MRI radiomics texture features. Iran. J. Radiol. 2017, 5, e48035. [CrossRef]
Peerlings, J.; Woodruff, H.C.; Winfield, J.M.; Ibrahim, A.; van Beers, B.E.; Heerschap, A.; Jackson, A.; Wildberger, J.E.; Mottaghy, F.M.; DeSouza, N.M.; et al. Stability of radiomics features in apparent diffusion coefficient maps from a multi-centre test-retest trial. Sci. Rep. 2019, 9, 4800. [CrossRef] [PubMed]
Pfaehler, E.; Beukinga, R.J.; de Jong, J.R.; Slart, R.H.J.A.; Slump, C.H.; Dierckx, R.A.J.O.; Boellaard, R. Repeatability of 18 F-FDG PET radiomic features: A phantom study to explore sensitivity to image reconstruction settings, noise, and delineation method. Med. Phys. 2019, 46, 665–678. [CrossRef]
Prayer, F.; Hofmanninger, J.; Weber, M.; Kifjak, D.; Willenpart, A.; Pan, J.; Röhrich, S.; Langs, G.; Prosch, H. Variability of computed tomography radiomics features of fibrosing interstitial lung disease: A test-retest study. Methods 2021, 188, 98–104. [CrossRef] [PubMed]
Zhao, B.; Tan, Y.; Tsai, W.-Y.; Qi, J.; Xie, C.; Lu, L.; Schwartz, L.H. Reproducibility of radiomics for deciphering tumor phenotype with imaging. Sci. Rep. 2016, 6, 23428. [CrossRef] [PubMed]
Ibrahim, A.; Refaee, T.; Primakov, S.; Barufaldi, B.; Acciavatti, R.J.; Granzier, R.W.Y.; Hustinx, R.; Mottaghy, F.M.; Woodruff, H.C.; Wildberger, J.E.; et al. The effects of in-plane spatial resolution on CT-based radiomic features’ stability with and without ComBat harmonization. Cancers 2021, 13, 1848. [CrossRef] [PubMed]
Zhovannik, I.; Bussink, J.; Traverso, A.; Shi, Z.; Kalendralis, P.; Wee, L.; Dekker, A.; Fijten, R.; Monshouwer, R. Learning from scanners: Bias reduction and feature correction in radiomics. Clin. Transl. Radiat. Oncol. 2019, 19, 33–38. [CrossRef] [PubMed]
Ibrahim, A.; Primakov, S.; Barufaldi, B.; Acciavatti, R.J.; Granzier, R.W.Y.; Hustinx, R.; Mottaghy, F.M.; Woodruff, H.C.; Wildberger, J.E.; Lambin, P.; et al. Reply to Orlhac, F.; Buvat, I. Comment on “Ibrahim et al. The Effects of In-Plane Spatial Resolution on CT-Based Radiomic Features’ Stability with and without ComBat Harmonization. Cancers 2021, 13, 1848”. Cancers 2021, 13, 3080. [CrossRef] [PubMed]
Ibrahim, A.; Refaee, T.; Leijenaar, R.T.H.; Primakov, S.; Hustinx, R.; Mottaghy, F.M.; Woodruff, H.C.; Maidment, A.D.A.; Lambin, P. The application of a workflow integrating the variable reproducibility and harmonizability of radiomic features on a phantom dataset. PLoS ONE 2021, 16, e0251147. [CrossRef]
Ibrahim, A.; Widaatalla, Y.; Refaee, T.; Primakov, S.; Miclea, R.L.; Öcal, O.; Fabritius, M.P.; Ingrisch, M.; Ricke, J.; Hustinx, R.; et al. Reproducibility of CT-based hepatocellular carcinoma radiomic features across different contrast imaging phases: A proof of concept on SORAMIC trial data. Cancers 2021, 13, 4638. [CrossRef]
Park, J.E.; Park, S.Y.; Kim, H.J.; Kim, H.S. Reproducibility and generalizability in radiomics modeling: Possible strategies in radiologic and statistical perspectives. Korean J. Radiol. 2019, 20, 1124–1137. [CrossRef] [PubMed]
Meyer, M.; Ronald, J.; Vernuccio, F.; Nelson, R.C.; Ramirez-Giraldo, J.C.; Solomon, J.; Patel, B.N.; Samei, E.; Marin, D. Repro-ducibility of CT radiomic features within the same patient: Influence of radiation dose and CT reconstruction settings. Radiology 2019, 293, 583–591. [CrossRef] [PubMed]
Lu, L.; Ehmke, R.C.; Schwartz, L.H.; Zhao, B. Assessing agreement between radiomic features computed for multiple CT imaging settings. PLoS ONE 2016, 11, e0166550. [CrossRef]
Johnson, W.E.; Li, C.; Rabinovic, A. Adjusting batch effects in microarray expression data using empirical bayes methods. Biostatistics 2007, 8, 118–127. [CrossRef]
Ligero, M.; Jordi-Ollero, O.; Bernatowicz, K.; Garcia-Ruiz, A.; Delgado-Muñoz, E.; Leiva, D.; Mast, R.; Suarez, C.; Sala-Llonch, R.; Calvo, N.; et al. Minimizing acquisition-related radiomics variability by image resampling and batch effect correction to allow for large-scale data analysis. Eur. Radiol. 2021, 31, 1460–1470. [CrossRef]
Foy, J.J.; Al-Hallaq, H.A.; Grekoski, V.; Tran, T.; Guruvadoo, K.; Armato, S.G., III; Sensakovic, W.F. Harmonization of radiomic feature variability resulting from differences in CT image acquisition and reconstruction: Assessment in a cadaveric liver. Phys. Med. Biol. 2020, 65, 205008. [CrossRef] [PubMed]
Arendt, C.T.; Leithner, D.; Mayerhoefer, M.E.; Gibbs, P.; Czerny, C.; Arnoldner, C.; Burck, I.; Leinung, M.; Tanyildizi, Y.; Lenga, L.; et al. Radiomics of high-resolution computed tomography for the differentiation between cholesteatoma and middle ear inflammation: Effects of post-reconstruction methods in a dual-center study. Eur. Radiol. 2021, 31, 4071–4078. [CrossRef]
Fortin, J.-P.; Cullen, N.; Sheline, Y.I.; Taylor, W.D.; Aselcioglu, I.; Cook, P.A.; Adams, P.; Cooper, C.; Fava, M.; McGrath, P.J.; et al. Harmonization of cortical thickness measurements across scanners and sites. Neuroimage 2018, 167, 104–120. [CrossRef] [PubMed]
Crombé, A.; Kind, M.; Fadli, D.; le Loarer, F.; Italiano, A.; Buy, X.; Saut, O. Intensity harmonization techniques influence radiomics features and radiomics-based predictions in sarcoma patients. Sci. Rep. 2020, 10, 15496. [CrossRef]
Lucia, F.; Visvikis, D.; Vallières, M.; Desseroit, M.-C.; Miranda, O.; Robin, P.; Bonaffini, P.A.; Alfieri, J.; Masson, I.; Mervoyer, A.; et al. External validation of a combined PET and MRI radiomics model for prediction of recurrence in cervical cancer patients treated with chemoradiotherapy. Eur. J. Nucl. Med. Mol. Imaging 2019, 46, 864–877. [CrossRef] [PubMed]
Shiri, I.; Amini, M.; Nazari, M.; Hajianfar, G.; Haddadi Avval, A.; Abdollahi, H.; Oveisi, M.; Arabi, H.; Rahmim, A.; Zaidi, H. Impact of feature harmonization on radiogenomics analysis: Prediction of EGFR and KRAS mutations from non-small cell lung cancer PET/CT images. Comput. Biol. Med. 2022, 142, 105230. [CrossRef] [PubMed]
Masson, I.; Da-Ano, R.; Lucia, F.; Doré, M.; Castelli, J.; Goislard de Monsabert, C.; Ramée, J.-F.; Sellami, S.; Visvikis, D.; Hatt, M.; et al. Statistical harmonization can improve the development of a multicenter CT-based radiomic model predictive of nonresponse to induction chemotherapy in laryngeal cancers. Med. Phys. 2021, 48, 4099–4109. [CrossRef] [PubMed]
Gallardo-Estrella, L.; Lynch, D.A.; Prokop, M.; Stinson, D.; Zach, J.; Judy, P.F.; van Ginneken, B.; van Rikxoort, E.M. Normalizing computed tomography data reconstructed with different filter kernels: Effect on emphysema quantification. Eur. Radiol. 2016, 26, 478–486. [CrossRef] [PubMed]
Mackin, D.; Fave, X.; Zhang, L.; Fried, D.; Yang, J.; Taylor, B.; Rodriguez-Rivera, E.; Dodge, C.; Jones, A.K.; Court, L. Credence cartridge radiomics phantom CT scans—The cancer imaging archive (TCIA) public access—Cancer imaging archive wiki. Cancer Imaging Arch. 2017. [CrossRef]
Clark, K.; Vendt, B.; Smith, K.; Freymann, J.; Kirby, J.; Koppel, P.; Moore, S.; Phillips, S.; Maffitt, D.; Pringle, M.; et al. The cancer imaging archive (TCIA): Maintaining and operating a public information repository. J. Digit. Imaging 2013, 26, 1045–1057. [CrossRef] [PubMed]
Van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.-C.; Pieper, S.; Aerts, H.J.W.L. Computational radiomics system to decode the radiographic phenotype. Cancer Res. 2017, 77, e104–e107. [CrossRef] [PubMed]
Team, R.C. R Language Definition; R Foundation for Statistical Computing: Vienna, Austria, 2000.
Gandrud, C. Reproducible Research with R and R Studio; CRC Press: Boca Raton, FL, USA, 2013; ISBN 9781466572843.
Lin, L.I. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989, 45, 255–268. [CrossRef] [PubMed]
Stevenson, M.; Stevenson, M.M.; BiasedUrn, I. Package “epiR”. Available online: https://vps.fmvz.usp.br/CRAN/web/packages/epiR/epiR.pdf (accessed on 15 January 2022).
McNemar, Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 1947, 12, 153–157. [CrossRef]
Denzler, S.; Vuong, D.; Bogowicz, M.; Pavic, M.; Frauenfelder, T.; Thierstein, S.; Eboulet, E.I.; Maurer, B.; Schniering, J.; Gabryś, H.S.; et al. Impact of CT convolution kernel on robustness of radiomic features for different lung diseases and tissue types. Br. J. Radiol. 2021, 94, 20200947. [CrossRef]
He, L.; Huang, Y.; Ma, Z.; Liang, C.; Liang, C.; Liu, Z. Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci. Rep. 2016, 6, 34921. [CrossRef]
Ibrahim, A.; Barufaldi, B.; Refaee, T.; Silva Filho, T.M.; Acciavatti, R.J.; Salahuddin, Z.; Hustinx, R.; Mottaghy, F.M.; Maidment, A.D.A.; Lambin, P. MaasPenn radiomics reproducibility score: A novel quantitative measure for evaluating the reproducibility of CT-based handcrafted radiomic features. Cancers 2022, 14, 1599. [CrossRef]
Li, Y.; Ammari, S.; Balleyguier, C.; Lassau, N.; Chouzenoux, E. Impact of preprocessing and harmonization methods on the removal of scanner effects in brain MRI radiomic features. Cancers 2021, 13, 3000. [CrossRef]
Lupean, R.-A.; Ștefan, P.-A.; Csutak, C.; Lebovici, A.; Măluțan, A.M.; Buiga, R.; Melincovici, C.S.; Mihu, C.M. Differentiation of endometriomas from ovarian hemorrhagic cysts at magnetic resonance: The role of texture analysis. Medicina 2020, 56, 487. [CrossRef]
