data harmonisation; data standardisation; domain adaptation; Information fusion; reproducibility; Computational data; Data harmonization; Data standardization; Domain adaptation; Evaluation metrics; Future research directions; Meta-analysis; Reproducibilities; State of the art; Systematic Review; Software; Signal Processing; Information Systems; Hardware and Architecture
Abstract :
[en] Removing the bias and variance of multicentre data has always been a challenge in large scale digital healthcare studies, which requires the ability to integrate clinical features extracted from data acquired by different scanners and protocols to improve stability and robustness. Previous studies have described various computational approaches to fuse single modality multicentre datasets. However, these surveys rarely focused on evaluation metrics and lacked a checklist for computational data harmonisation studies. In this systematic review, we summarise the computational data harmonisation approaches for multi-modality data in the digital healthcare field, including harmonisation strategies and evaluation metrics based on different theories. In addition, a comprehensive checklist that summarises common practices for data harmonisation studies is proposed to guide researchers to report their research findings more effectively. Last but not least, flowcharts presenting possible ways for methodology and metric selection are proposed and the limitations of different methods have been surveyed for future research.
Disciplines :
Computer science
Author, co-author :
Nan, Yang; National Heart and Lung Institute, Imperial College London, London, Ireland
Ser, Javier Del; Department of Communications Engineering, University of the Basque Country UPV/EHU, Bilbao, Spain ; TECNALIA, Basque Research and Technology Alliance (BRTA), Derio, Spain
Walsh, Simon; National Heart and Lung Institute, Imperial College London, London, Ireland
Schönlieb, Carola; Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, Ireland
Roberts, Michael; Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, Ireland ; Oncology R&D, AstraZeneca, Cambridge, Ireland
Selby, Ian; Department of Radiology, University of Cambridge, Cambridge, Ireland
Howard, Kit; Clinical Data Interchange Standards Consortium, Austin, United States
Owen, John; Clinical Data Interchange Standards Consortium, Austin, United States
Neville, Jon; Clinical Data Interchange Standards Consortium, Austin, United States
GUIOT, Julien ; Centre Hospitalier Universitaire de Liège - CHU > > Service de pneumologie - allergologie
Ernst, Benoit ; Université de Liège - ULiège > Département des sciences cliniques > Pneumologie - Allergologie
Pastor, Ana; QUIBIM, Valencia, Spain
Alberich-Bayarri, Angel; QUIBIM, Valencia, Spain
Menzel, Marion I.; Technische Hochschule Ingolstadt, Ingolstadt, Germany ; GE Healthcare GmbH, Germany
Chatterjee, Avishek; Department of Precision Medicine, Maastricht University, Maastricht, Netherlands
Woodruff, Henry; Department of Precision Medicine, Maastricht University, Maastricht, Netherlands
Lambin, Philippe; Department of Precision Medicine, Maastricht University, Maastricht, Netherlands
Cerdá-Alberich, Leonor; Medical Imaging Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain
Martí-Bonmatí, Luis; Medical Imaging Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain
Herrera, Francisco; Department of Computer Sciences and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada, Granada, Spain ; Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia
Yang, Guang; National Heart and Lung Institute, Imperial College London, London, Ireland ; Cardiovascular Research Centre, Royal Brompton Hospital, London, Ireland ; School of Biomedical Engineering & Imaging Sciences, King's College London, London, Ireland
This study was supported in part by the European Research Council Innovative Medicines Initiative (DRAGON # , H2020-JTI-IMI2 101005122 ), the AI for Health Imaging Award (CHAIMELEON ## , H2020-SC1-FA-DTS-2019–1 952172), the UK Research and Innovation Future Leaders Fellowship ( MR/V023799/1 ), the British Heart Foundation (Project Number: TG/18/5/34111, PG/16/78/32402), the SABRE project supported by Boehringer Ingelheim Ltd, the European Union's Horizon 2020 research and innovation programme (ICOVID, 101016131), the Euskampus Foundation (COVID19 Resilience, Ref. COnfVID19), and the Basque Government (consolidated research group MATHMODE, Ref. IT1294–19, and 3KIA project from the ELKARTEK funding program, Ref. KK-2020/00049).This study was supported in part by the European Research Council Innovative Medicines Initiative (DRAGON#, H2020-JTI-IMI2 101005122), the AI for Health Imaging Award (CHAIMELEON##, H2020-SC1-FA-DTS-2019?1 952172), the UK Research and Innovation Future Leaders Fellowship (MR/V023799/1), the British Heart Foundation (Project Number: TG/18/5/34111, PG/16/78/32402), the SABRE project supported by Boehringer Ingelheim Ltd, the European Union's Horizon 2020 research and innovation programme (ICOVID, 101016131), the Euskampus Foundation (COVID19 Resilience, Ref. COnfVID19), and the Basque Government (consolidated research group MATHMODE, Ref. IT1294?19, and 3KIA project from the ELKARTEK funding program, Ref. KK-2020/00049). # DRAGON Consortium:, Xiaodan Xinga, Ming Lia, Scott Wagersb, Rebecca Bakerc, Cosimo Nardid, Brice van Eeckhoute, Paul Skippf, Pippa Powellg, Miles Carrollh, Alessandro Ruggieroi, Muhunthan Thillaii, Judith Babari, Evis Salai, William Murchj, Julian Hiscoxk, Diana Barallel, Nicola Sverzellatim, ## CHAIMELEON Consortium:, Ana Miguel Blancon, Fuensanta Bellv?s Batallero, Mario Aznarp, Amelia Suarezp, Sergio Figueirasq, Katharina Krischakr, Monika Hierathr, Yisroel Mirskys, Yuval Elovicis, Jean Paul Beregit, Laure Fourniert, Francesco Sardanelliu, Tobias Penzkoferv, Karine Seymourw, Nacho Blanquerx, Emanuele Neriy, Andrea Laghiz, Manuela Fran?aaa, Ricard Martinezab, a National Heart and Lung Institute, Imperial College London, London, UK, b BioSci Consulting, Maasmechelen, Belgium, c Clinical Data Interchange Standards Consortium, Austin, Texas, United States, d University of Florence, Firenze, Italy, e Medical Cloud Company, Li?ge, Belgium, f TopMD, Southampton, UK, g European Lung Foundation, Sheffield, UK, h Department of Health, Public Health England, London, UK, i Department of Radiology, University of Cambridge, Cambridge, UK, j Owlstone Medical, Cambridge, UK, k University of Liverpool, Liverpool, UK, l University of Southampton, Southampton, UK, m University of Parma, Parma, Italy, n Medical Imaging Department, Hospital Universitari i Polit?cnic La Fe, Valencia, Spain, o QUIBIM, Valencia, Spain, p Matical Innovation, Madrid, Spain, q Bah?a Software, A Coru?a, Spain, r European Institute for Biomedical Imaging Research, Vienna, Austria, s Ben Gurion University of the Negev, Be'er Sheva, Israel, t Le Coll?ge des Enseignants en Radiologie de France, France, u Research Hospital Policlinico San Donato, Milan, Italy, v Charit? ? Universit?tsmedizin Berlin, Berlin, Germany, w Medexprim, Lab?ge, France, x Valencia Polytechnic University, Valencia, Spain, y University of Pisa, Pisa, Italy, z Sapienza University of Rome, Rome, Italy, aa The Centro Hospitalar Universit?rio do Porto, Portugal, ab University of Valencia, Valencia, Spain
Delbeke, D., Coleman, R.E., Guiberteau, M.J., Brown, M.L., Royal, H.D., Siegel, B.A., Townsend, D.W., Berland, L.L., Parker, J.A., Hubner, K., Procedure guideline for tumor imaging with 18F-FDG PET/CT 1.0. J. Nucl. Med. 47 (2006), 885–895.
Simon, J., Li, D., Traboulsee, A., Coyle, P., Arnold, D., Barkhof, F., Frank, J., Grossman, R., Paty, D., Radue, E., Standardized MR imaging protocol for multiple sclerosis: consortium of MS Centers consensus guidelines. Am. J. Neuroradiol. 27 (2006), 455–461.
Schmidt, B.-.M., Colvin, C.J., Hohlfeld, A., Leon, N., Defining and conceptualising data harmonisation: a scoping review protocol. Syst. Rev. 7 (2018), 1–6.
Schmidt, B.-.M., Colvin, C.J., Hohlfeld, A., Leon, N., Definitions, components and processes of data harmonisation in healthcare: a scoping review. BMC Med. Inform. Decis. Mak. 20 (2020), 1–19.
Da-Ano, R., Visvikis, D., Hatt, M., Harmonization strategies for multicenter radiomics investigations. Phy. Med. Biol., 65, 2020, 24TR02.
Pinto, M.S., Paolella, R., Billiet, T., Van Dyck, P., Guns, P.-.J., Jeurissen, B., Ribbens, A., den Dekker, A.J., Sijbers, J., Harmonization of brain diffusion MRI: concepts and methods. Front. Neurosci., 14, 2020, 396.
Gitto, S., Cuocolo, R., Albano, D., Morelli, F., Pescatori, L.C., Messina, C., Imbriaco, M., Sconfienza, L.M., CT and MRI radiomics of bone and soft-tissue sarcomas: a systematic review of reproducibility and validation strategies. Insights Imaging 12 (2021), 1–14.
Mali, S.A., Ibrahim, A., Woodruff, H.C., Andrearczyk, V., Müller, H., Primakov, S., Salahuddin, Z., Chatterjee, A., Lambin, P., Making radiomics more reproducible across scanner and imaging protocol variations: a review of harmonization methods. J. Pers. Med., 11, 2021, 842.
Chen, T., Philip, M., Cao, K.-A.Lê, Tyagi, S., A multi-modal data harmonisation approach for discovery of COVID-19 drug targets. Brief. Bioinform., 2021.
Tax, C.M., Grussu, F., Kaden, E., Ning, L., Rudrapatna, U., Evans, C.J., St-Jean, S., Leemans, A., Koppers, S., Merhof, D., Cross-scanner and cross-protocol diffusion MRI data harmonisation: a benchmark database and evaluation of algorithms. Neuroimage 195 (2019), 285–299.
Hutchinson, D.M., Silins, E., Mattick, R.P., Patton, G.C., Fergusson, D.M., Hayatbakhsh, R., Toumbourou, J.W., Olsson, C.A., Najman, J.M., Spry, E., How can data harmonisation benefit mental health research? An example of the Cannabis cohorts research consortium. Australian New Zealand J. Psychiat. 49 (2015), 317–323.
Zhao, B., Tan, Y., Tsai, W.-.Y., Qi, J., Xie, C., Lu, L., Schwartz, L.H., Reproducibility of radiomics for deciphering tumor phenotype with imaging. Sci. Rep. 6 (2016), 1–7.
Kumar, N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A., Sethi, A., A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36 (2017), 1550–1560.
Hotta, M., Minamimoto, R., Gohda, Y., Miwa, K., Otani, K., Kiyomatsu, T., Yano, H., Prognostic value of 18 F-FDG PET/CT with texture analysis in patients with rectal cancer treated by surgery. Ann. Nucl. Med., 2021, 1–10.
Mattoli, M.V., Calcagni, M.L., Taralli, S., Indovina, L., Spottiswoode, B.S., Giordano, A., How often do we fail to classify the treatment response with [18 F] FDG PET/CT acquired on different scanners? Data from clinical oncological practice using an automatic tool for SUV harmonization. Mol. Imaging Biol. 21 (2019), 1210–1219.
Zhu, T., Hu, R., Qiu, X., Taylor, M., Tso, Y., Yiannoutsos, C., Navia, B., Mori, S., Ekholm, S., Schifitto, G., Quantification of accuracy and precision of multi-center DTI measurements: a diffusion phantom and human brain study. Neuroimage 56 (2011), 1398–1411.
Jovicich, J., Marizzoni, M., Bosch, B., Bartrés-Faz, D., Arnold, J., Benninghoff, J., Wiltfang, J., Roccatagliata, L., Picco, A., Nobili, F., Multisite longitudinal reliability of tract-based spatial statistics in diffusion tensor imaging of healthy elderly subjects. Neuroimage 101 (2014), 390–403.
Leo, P., Lee, G., Shih, N.N., Elliott, R., Feldman, M.D., Madabhushi, A., Evaluating stability of histomorphometric features across scanner and staining variations: prostate cancer diagnosis from whole slide images. J. Med. Imaging, 3, 2016, 047502.
Berenguer, R., Pastor-Juan, M.D.R., Canales-Vázquez, J., Castro-García, M., Villas, M.V., Mansilla Legorburo, F., Sabater, S., Radiomics of CT features may be nonreproducible and redundant: influence of CT acquisition parameters. Radiology 288 (2018), 407–415.
Sunderland, J.J., Christian, P.E., Quantitative PET/CT scanner performance characterization based upon the society of nuclear medicine and molecular imaging clinical trials network oncology clinical simulator phantom. J. Nucl. Med. 56 (2015), 145–152.
Jha, A., Mithun, S., Jaiswar, V., Sherkhane, U., Purandare, N., Prabhash, K., Rangarajan, V., Dekker, A., Wee, L., Traverso, A., Repeatability and reproducibility study of radiomic features on a phantom and human cohort. Sci. Rep. 11 (2021), 1–12.
N. Emaminejad, M.W. Wahi-Anwar, G.H.J. Kim, W. Hsu, M. Brown, M. McNitt-Gray, Reproducibility of lung nodule radiomic features: multivariable and univariable investigations that account for interactions between CT acquisition and reconstruction parameters, Med. Phys., (2021).
Kim, M., Jung, S.C., Park, J.E., Park, S.Y., Lee, H., Choi, K.M., Reproducibility of radiomic features in SENSE and compressed SENSE: impact of acceleration factors. Eur. Radiol., 2021, 1–14.
Yamashita, R., Perrin, T., Chakraborty, J., Chou, J.F., Horvat, N., Koszalka, M.A., Midya, A., Gonen, M., Allen, P., Jarnagin, W.R., Radiomic feature reproducibility in contrast-enhanced CT of the pancreas is affected by variabilities in scan parameters and manual segmentation. Eur. Radiol. 30 (2020), 195–205.
Fiset, S., Welch, M.L., Weiss, J., Pintilie, M., Conway, J.L., Milosevic, M., Fyles, A., Traverso, A., Jaffray, D., Metser, U., Repeatability and reproducibility of MRI-based radiomic features in cervical cancer. Radiother. Oncol. 135 (2019), 107–114.
Saeedi, E., Dezhkam, A., Beigi, J., Rastegar, S., Yousefi, Z., Mehdipour, L.A., Abdollahi, H., Tanha, K., Radiomic feature robustness and reproducibility in quantitative bone radiography: a study on radiologic parameter changes. J. Clin. Densitom. 22 (2019), 203–213.
Meyer, M., Ronald, J., Vernuccio, F., Nelson, R.C., Ramirez-Giraldo, J.C., Solomon, J., Patel, B.N., Samei, E., Marin, D., Reproducibility of CT radiomic features within the same patient: influence of radiation dose and CT reconstruction settings. Radiology 293 (2019), 583–591.
Perrin, T., Midya, A., Yamashita, R., Chakraborty, J., Saidon, T., Jarnagin, W.R., Gonen, M., Simpson, A.L., Do, R.K., Short-term reproducibility of radiomic features in liver parenchyma and liver malignancies on contrast-enhanced CT imaging. Abdominal Radiol. 43 (2018), 3271–3278.
Midya, A., Chakraborty, J., Gönen, M., Do, R.K., Simpson, A.L., Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility. J.Med. Imaging, 5, 2018, 011020.
Altazi, B.A., Zhang, G.G., Fernandez, D.C., Montejo, M.E., Hunt, D., Werner, J., Biagioli, M.C., Moros, E.G., Reproducibility of F18-FDG PET radiomic features for different cervical tumor segmentation methods, gray-level discretization, and reconstruction algorithms. J. Appl. Clin. Med. Phy. 18 (2017), 32–48.
Hu, P., Wang, J., Zhong, H., Zhou, Z., Shen, L., Hu, W., Zhang, Z., Reproducibility with repeat CT in radiomics study for rectal cancer. Oncotarget, 7, 2016, 71440.
Choe, J., Lee, S.M., Do, K.-.H., Lee, G., Lee, J.-.G., Lee, S.M., Seo, J.B., Deep learning–based image conversion of CT reconstruction kernels improves radiomics reproducibility for pulmonary nodules or masses. Radiology 292 (2019), 365–373.
Primak, A.N., McCollough, C.H., Bruesewitz, M.R., Zhang, J., Fletcher, J.G., Relationship between noise, dose, and pitch in cardiac multi–detector row CT. Radiographics 26 (2006), 1785–1794.
Gierada, D.S., Bierhals, A.J., Choong, C.K., Bartel, S.T., Ritter, J.H., Das, N.A., Hong, C., Pilgram, T.K., Bae, K.T., Whiting, B.R., Effects of CT section thickness and reconstruction kernel on emphysema quantification: relationship to the magnitude of the CT emphysema index. Acad. Radiol. 17 (2010), 146–156.
Tung, P.-.Y., Blischak, J.D., Hsiao, C.J., Knowles, D.A., Burnett, J.E., Pritchard, J.K., Gilad, Y., Batch effects and the effective design of single-cell gene expression studies. Sci. Rep. 7 (2017), 1–15.
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W.M. III, Hao, Y., Stoeckius, M., Smibert, P., Satija, R., Comprehensive integration of single-cell data. Cell 177 (2019), 1888–1902 e1821.
Vijh, S., Saraswat, M., Kumar, S., A new complete color normalization method for H&E stained histopatholgical images. Appl. Intell., 2021, 1–14.
D.E. Chandler, R.W. Roberson, Bioimaging: current concepts in light and electron microscopy, (2009).
Sun, D., Wang, J., Han, Y., Dong, X., Ge, J., Zheng, R., Shi, X., Wang, B., Li, Z., Ren, P., TISCH: a comprehensive web resource enabling interactive single-cell transcriptome visualization of tumor microenvironment. Nucleic Acids Res. 49 (2021), D1420–D1430.
Shafiq-ul-Hassan, M., Zhang, G.G., Latifi, K., Ullah, G., Hunt, D.C., Balagurunathan, Y., Abdalah, M.A., Schabath, M.B., Goldgof, D.G., Mackin, D., Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. Med. Phys. 44 (2017), 1050–1062.
Mackin, D., Fave, X., Zhang, L., Fried, D., Yang, J., Taylor, B., Rodriguez-Rivera, E., Dodge, C., Jones, A.K., Court, L., Measuring CT scanner variability of radiomics features. Invest. Radiol., 50, 2015, 757.
Mårtensson, G., Ferreira, D., Granberg, T., Cavallin, L., Oppedal, K., Padovani, A., Rektorova, I., Bonanni, L., Pardini, M., Kramberger, M.G., The reliability of a deep learning model in clinical out-of-distribution MRI data: a multicohort study. Med. Image Anal., 66, 2020, 101714.
Rathore, S., Bakas, S., Akbari, H., Shukla, G., Rozycki, M., Davatzikos, C., Deriving stable multi-parametric MRI radiomic signatures in the presence of inter-scanner variations: survival prediction of glioblastoma via imaging pattern analysis and machine learning techniques. Proceedings of the Medical Imaging 2018: Computer-Aided Diagnosis, International Society for Optics and Photonics, 2018, 1057509.
Johnson, W.E., Li, C., Rabinovic, A., Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics 8 (2007), 118–127.
Pandey, U., Saini, J., Kumar, M., Gupta, R., Ingalhalikar, M., Normative baseline for radiomics in Brain MRI: evaluating the robustness, regional variations, and reproducibility on FLAIR Images. J. Magn. Reson. Imaging, 2020.
Ingalhalikar, M., Shinde, S., Karmarkar, A., Rajan, A., Rangaprakash, D., Deshpande, G., Functional connectivity-based prediction of Autism on site harmonized ABIDE dataset. IEEE Trans. Biomed. Eng., 2021.
Wengler, K., Cassidy, C., van Der Pluijm, M., Weinstein, J.J., Abi-Dargham, A., van de Giessen, E., Horga, G., Cross-scanner harmonization of neuromelanin-sensitive MRI for multisite studies. J. Magn. Reson. Imaging, 2021.
Beaumont, H., Iannessi, A., Bertrand, A.-.S., Cucchi, J.M., Lucidarme, O., Harmonization of radiomic feature distributions: impact on classification of hepatic tissue in CT imaging. Eur. Radiol., 2021, 1–10.
Garcia-Dias, R., Scarpazza, C., Baecker, L., Vieira, S., Pinaya, W.H., Corvin, A., Redolfi, A., Nelson, B., Crespo-Facorro, B., McDonald, C., Neuroharmony: a new tool for harmonizing volumetric MRI data from unseen scanners. Neuroimage, 220, 2020.
Beer, J.C., Tustison, N.J., Cook, P.A., Davatzikos, C., Sheline, Y.I., Shinohara, R.T., Linn, K.A., A.s.D.N. Initiative, Longitudinal combat: a method for harmonizing longitudinal multi-scanner imaging data. Neuroimage, 220, 2020, 117129.
Radua, J., Vieta, E., Shinohara, R., Kochunov, P., Quidé, Y., Green, M.J., Weickert, C.S., Weickert, T., Bruggemann, J., Kircher, T., Increased power by harmonizing structural MRI site differences with the ComBat batch adjustment method in ENIGMA. Neuroimage, 218, 2020, 116956.
Whitney, H.M., Li, H., Ji, Y., Liu, P., Giger, M.L., Harmonization of radiomic features of breast lesions across international DCE-MRI datasets. J. Med. Imaging, 7, 2020, 012707.
Yu, M., Linn, K.A., Cook, P.A., Phillips, M.L., McInnis, M., Fava, M., Trivedi, M.H., Weissman, M.M., Shinohara, R.T., Sheline, Y.I., Statistical harmonization corrects site effects in functional connectivity measurements from multi-site fMRI data. Hum Brain Mapp. 39 (2018), 4213–4227.
Espín-Pérez, A., Portier, C., Chadeau-Hyam, M., van Veldhoven, K., Kleinjans, J.C., de Kok, T.M., Comparison of statistical methods and the use of quality control samples for batch effect correction in human transcriptome data. PLoS ONE, 13, 2018, e0202947.
Fortin, J.-.P., Cullen, N., Sheline, Y.I., Taylor, W.D., Aselcioglu, I., Cook, P.A., Adams, P., Cooper, C., Fava, M., McGrath, P.J., Harmonization of cortical thickness measurements across scanners and sites. Neuroimage 167 (2018), 104–120.
Kothari, S., Phan, J.H., Stokes, T.H., Osunkoya, A.O., Young, A.N., Wang, M.D., Removing batch effects from histopathological images for enhanced cancer diagnosis. IEEE J. Biomed. Health Inform. 18 (2013), 765–772.
Arendt, C.T., Leithner, D., Mayerhoefer, M.E., Gibbs, P., Czerny, C., Arnoldner, C., Burck, I., Leinung, M., Tanyildizi, Y., Lenga, L., Radiomics of high-resolution computed tomography for the differentiation between cholesteatoma and middle ear inflammation: effects of post-reconstruction methods in a dual-center study. Eur. Radiol. 31 (2021), 4071–4078.
Ibrahim, A., Refaee, T., Leijenaar, R.T., Primakov, S., Hustinx, R., Mottaghy, F.M., Woodruff, H.C., Maidment, A.D., Lambin, P., The application of a workflow integrating the variable reproducibility and harmonizability of radiomic features on a phantom dataset. PLoS ONE, 16, 2021, e0251147.
Lan, J., Cai, S., Xue, Y., Gao, Q., Du, M., Zhang, H., Wu, Z., Deng, Y., Huang, Y., Tong, T., Unpaired stain style transfer using invertible neural networks based on channel attention and long-range residual. IEEE Access 9 (2021), 11282–11295.
Wachinger, C., Rieckmann, A., Pölsterl, S., Initiative, A.s.D.N., Detect and correct bias in multi-site neuroimaging datasets. Med. Image Anal., 67, 2021, 101879.
Foy, J.J., Al-Hallaq, H.A., Grekoski, V., Tran, T., Guruvadoo, K., Armato Iii, S.G., Sensakovic, W.F., Harmonization of radiomic feature variability resulting from differences in CT image acquisition and reconstruction: assessment in a cadaveric liver. Phy. Med. Biol., 65, 2020, 205008.
Martin, M.-J.Saint, Orlhac, F., Akl, P., Khalid, F., Nioche, C., Buvat, I., Malhaire, C., Frouin, F., A radiomics pipeline dedicated to Breast MRI: validation on a multi-scanner phantom study, magnetic resonance materials in physics. Biol. Med., 2020, 1–12.
Karayumak, S.C., Bouix, S., Ning, L., James, A., Crow, T., Shenton, M., Kubicki, M., Rathi, Y., Retrospective harmonization of multi-site diffusion MRI data acquired with different acquisition parameters. Neuroimage 184 (2019), 180–200.
Stein, C.K., Qu, P., Epstein, J., Buros, A., Rosenthal, A., Crowley, J., Morgan, G., Barlogie, B., Removing batch effects from purified plasma cell gene expression microarrays with modified ComBat. BMC Bioinformatics 16 (2015), 1–9.
Da-Ano, R., Masson, I., Lucia, F., Doré, M., Robin, P., Alfieri, J., Rousseau, C., Mervoyer, A., Reinhold, C., Castelli, J., Performance comparison of modified ComBat for harmonization of radiomic features for multicenter studies. Sci. Rep. 10 (2020), 1–12.
Müller, C., Schillert, A., Röthemeier, C., Trégouët, D.-.A., Proust, C., Binder, H., Pfeiffer, N., Beutel, M., Lackner, K.J., Schnabel, R.B., Removing batch effects from longitudinal gene expression-quantile normalization plus ComBat as best approach for microarray transcriptome data. PLoS ONE, 11, 2016, e0156594.
Benito, M., Parker, J., Du, Q., Wu, J., Xiang, D., Perou, C.M., Marron, J.S., Adjustment of systematic microarray data biases. Bioinformatics 20 (2004), 105–114.
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-r., Raychaudhuri, S., Fast, sensitive and accurate integration of single-cell data with Harmony. Nat. Methods 16 (2019), 1289–1296.
Haghverdi, L., Lun, A.T., Morgan, M.D., Marioni, J.C., Batch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors. Nat. Biotechnol. 36 (2018), 421–427.
Hie, B., Bryson, B., Berger, B., Efficient integration of heterogeneous single-cell transcriptomes using Scanorama. Nat. Biotechnol. 37 (2019), 685–691.
Polański, K., Young, M.D., Miao, Z., Meyer, K.B., Teichmann, S.A., Park, J.-.E., BBKNN: fast batch alignment of single cell transcriptomes. Bioinformatics 36 (2020), 964–965.
L. McInnes, J. Healy, J. Melville, Umap: uniform manifold approximation and projection for dimension reduction, arXiv preprint arXiv:1802.03426, (2018).
Butler, A., Hoffman, P., Smibert, P., Papalexi, E., Satija, R., Integrating single-cell transcriptomic data across different conditions, technologies, and species. Nat. Biotechnol. 36 (2018), 411–420.
Gagnon-Bartsch, J.A., Speed, T.P., Using control genes to correct for unwanted variation in microarray data. Biostatistics 13 (2012), 539–552.
Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S., Vert, J.-.P., A general and flexible method for signal extraction from single-cell RNA-seq data. Nat. Commun. 9 (2018), 1–17.
Alter, O., Brown, P.O., Botstein, D., Singular value decomposition for genome-wide expression data processing and modeling. Proc. Natl. Acad. Sci. 97 (2000), 10101–10106.
Leek, J.T., Johnson, W.E., Parker, H.S., Jaffe, A.E., Storey, J.D., The sva package for removing batch effects and other unwanted variation in high-throughput experiments. Bioinformatics 28 (2012), 882–883.
Mirzaalian, H., Ning, L., Savadjiev, P., Pasternak, O., Bouix, S., Michailovich, O., Karmacharya, S., Grant, G., Marx, C.E., Morey, R.A., Multi-site harmonization of diffusion MRI data in a registration framework. Brain Imaging Behav. 12 (2018), 284–295.
Mirzaalian, H., de Pierrefeu, A., Savadjiev, P., Pasternak, O., Bouix, S., Kubicki, M., Westin, C.-.F., Shenton, M.E., Rathi, Y., Harmonizing diffusion MRI data across multiple sites and scanners. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, Springer, 12–19.
Zhang, Y., Jenkins, D.F., Manimaran, S., Johnson, W.E., Alternative empirical Bayes models for adjusting for batch effects in genomic studies. BMC Bioinformatics 19 (2018), 1–15.
Huynh, K.M., Chen, G., Wu, Y., Shen, D., Yap, P.-.T., Multi-site harmonization of diffusion MRI data via method of moments. IEEE Trans. Med. Imaging 38 (2019), 1599–1609.
Wrobel, J., Martin, M., Bakshi, R., Calabresi, P., Elliot, M., Roalf, D., Gur, R., Gur, R., Henry, R., Nair, G., Intensity warping for multisite MRI harmonization. Neuroimage, 223, 2020, 117242.
Llera, A., Huertas, I., Mir, P., Beckmann, C.F., Quantitative intensity harmonization of dopamine transporter SPECT images using gamma mixture models. Mol. Imaging Biol. 21 (2019), 339–347.
Lazar, C., Taminau, J., Meganck, S., Steenhoff, D., Coletta, A., Solís, D.Y.W., Molter, C., Duque, R., Bersini, H., Nowé, A., GENESHIFT: a nonparametric approach for integrating microarray gene expression data based on the inner product as a distance measure between the distributions of genes. IEEE/ACM Trans. Comput. Biol. Bioinform. 10 (2013), 383–392.
Mackin, D., Fave, X., Zhang, L., Yang, J., Jones, A.K., Ng, C.S., Court, L., Harmonizing the pixel size in retrospective computed tomography radiomics studies. PLoS ONE, 12, 2017, e0178524.
Moradmand, H., Aghamiri, S.M.R., Ghaderi, R., Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma. J. Appl. Clin. Med. Phy. 21 (2020), 179–190.
Pitas, I., Digital Image Processing Algorithms and Applications. 2000, John Wiley & Sons.
Zarella, M.D., Yeoh, C., Breen, D.E., Garcia, F.U., An alternative reference space for H&E color normalization. PLoS ONE, 12, 2017, e0174489.
Li, X., Plataniotis, K.N., A complete color normalization approach to histopathology images using color cues computed from saturation-weighted statistics. IEEE Trans. Biomed. Eng. 62 (2015), 1862–1873.
Swinehart, D.F., The beer-lambert law. J. Chem. Educ., 39, 1962, 333.
Tosta, T.A.A., de Faria, P.R., Neves, L.A., do Nascimento, M.Z., Color normalization of faded H&E-stained histological images using spectral matching. Comput. Biol. Med., 111, 2019, 103344.
Khan, A.M., Rajpoot, N., Treanor, D., Magee, D., A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans. Biomed. Eng. 61 (2014), 1729–1738.
Ruifrok, A.C., Johnston, D.A., Quantification of histochemical staining by color deconvolution. Anal. Quant. Cytol. Histol. 23 (2001), 291–299.
Hoque, M.Z., Keskinarkaus, A., Nyberg, P., Seppänen, T., Retinex model based stain normalization technique for whole slide image analysis. Comput. Med. Imaging Graph., 90, 2021, 101901.
Vahadane, A., Peng, T., Sethi, A., Albarqouni, S., Wang, L., Baust, M., Steiger, K., Schlitter, A.M., Esposito, I., Navab, N., Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans. Med. Imaging 35 (2016), 1962–1971.
Lei, G., Xia, Y., Zhai, D.-.H., Zhang, W., Chen, D., Wang, D., StainCNNs: an efficient stain feature learning method. Neurocomputing 406 (2020), 267–273.
Zheng, Y., Jiang, Z., Zhang, H., Xie, F., Shi, J., Xue, C., Adaptive color deconvolution for histological WSI normalization. Comput. Methods Programs Biomed. 170 (2019), 107–120.
Maji, P., Mahapatra, S., Rough-fuzzy circular clustering for color normalization of histological images. Fundam. Inform. 164 (2019), 103–117.
Maji, P., Mahapatra, S., Circular clustering in fuzzy approximation spaces for color normalization of histological images. IEEE Trans Med Imaging 39 (2019), 1735–1745.
Cheng, H.-.D., Cai, X., Min, R., A novel approach to color normalization using neural network. Neural Comput. Appl. 18 (2009), 237–247.
Golkov, V., Dosovitskiy, A., Sperl, J.I., Menzel, M.I., Czisch, M., Sämann, P., Brox, T., Cremers, D., Q-space deep learning: twelve-fold shorter and model-free diffusion MRI scans. IEEE Trans. Med. Imaging 35 (2016), 1344–1351.
Koppers, S., Bloy, L., Berman, J.I., Tax, C.M., Edgar, J.C., Merhof, D., Spherical harmonic residual network for diffusion signal harmonization. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019, Springer, 173–182.
Karayumak, S.C., Kubicki, M., Rathi, Y., Harmonizing diffusion MRI data across magnetic field strengths. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018, Springer, 116–124.
Dewey, B.E., Zhao, C., Reinhold, J.C., Carass, A., Fitzgerald, K.C., Sotirchos, E.S., Saidha, S., Oh, J., Pham, D.L., Calabresi, P.A., DeepHarmony: a deep learning approach to contrast harmonization across scanner changes. Magn. Reson. Imaging 64 (2019), 160–170.
Tong, Q., Gong, T., He, H., Wang, Z., Yu, W., Zhang, J., Zhai, L., Cui, H., Meng, X., Tax, C.W., A deep learning–based method for improving reliability of multicenter diffusion kurtosis imaging with varied acquisition protocols. Magn. Reson. Imaging 73 (2020), 31–44.
Park, S., Lee, S.M., Do, K.-.H., Lee, J.-.G., Bae, W., Park, H., Jung, K.-.H., Seo, J.B., Deep learning algorithm for reducing CT slice thickness: effect on reproducibility of radiomic features in lung cancer. Korean J. Radiol. 20 (2019), 1431–1440.
Shaham, U., Stanton, K.P., Zhao, J., Li, H., Raddassi, K., Montgomery, R., Kluger, Y., Removal of batch effects using distribution-matching residual networks. Bioinformatics 33 (2017), 2539–2546.
He, K., Zhang, X., Ren, S., Sun, J., Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, 770–778.
Ioffe, S., Szegedy, C., Batch normalization: accelerating deep network training by reducing internal covariate shift. Proceedings of the International conference on machine learning, 2015, PMLR, 448–456.
Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., Smola, A., A kernel method for the two-sample-problem. Adv. Neural Inf. Process. Syst. 19 (2006), 513–520.
Jog, A., Carass, A., Roy, S., Pham, D.L., Prince, J.L., MR image synthesis by contrast learning on neighborhood ensembles. Med. Image Anal. 24 (2015), 63–76.
Jog, A., Carass, A., Roy, S., Pham, D.L., Prince, J.L., Random forest regression for magnetic resonance image synthesis. Med. Image Anal. 35 (2017), 475–488.
Zhu, J.-.Y., Park, T., Isola, P., Efros, A.A., Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference on computer vision, 2017, 2223–2232.
Zhao, F., Wu, Z., Wang, L., Lin, W., Xia, S., Shen, D., Li, G., Harmonization of infant cortical thickness using surface-to-surface cycle-consistent adversarial networks. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019, Springer, 475–483.
Ren, M., Dey, N., Fishbaugh, J., Gerig, G., Segmentation-renormalized deep feature modulation for unpaired image harmonization. IEEE Trans. Med. Imaging 40 (2021), 1519–1530.
Zhong, J., Wang, Y., Li, J., Xue, X., Liu, S., Wang, M., Gao, X., Wang, Q., Yang, J., Li, X., Inter-site harmonization based on dual generative adversarial networks for diffusion tensor imaging: application to neonatal white matter development. Biomed. Eng. Online 19 (2020), 1–18.
Moyer, D., Ver Steeg, G., Tax, C.M., Thompson, P.M., Scanner invariant representations for diffusion MRI harmonization. Magn. Reson. Med. 84 (2020), 2174–2189.
Russkikh, N., Antonets, D., Shtokalo, D., Makarov, A., Vyatkin, Y., Zakharov, A., Terentyev, E., Style transfer with variational autoencoders is a promising approach to RNA-Seq data harmonization and analysis. Bioinformatics 36 (2020), 5076–5085.
Johansen, N., Quon, G., scAlign: a tool for alignment, integration, and rare cell identification from scRNA-seq data. Genome Biol. 20 (2019), 1–21.
Haeusser, P., Mordvintsev, A., Cremers, D., Learning by association–a versatile semi-supervised training method for neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, 89–98.
Wang, D., Hou, S., Zhang, L., Wang, X., Liu, B., Zhang, Z., iMAP: integration of multiple single-cell datasets by adversarial paired transfer networks. Genome Biol. 22 (2021), 1–24.
Mairal, J., Bach, F., Ponce, J., Sapiro, G., Online learning for matrix factorization and sparse coding. J. Machine Learn. Res., 11, 2010.
St-Jean, S., Coupé, P., Descoteaux, M., Non local spatial and angular matching: enabling higher spatial resolution diffusion MRI datasets through adaptive denoising. Med. Image Anal. 32 (2016), 115–130.
Tosta, T.A.A., de Faria, P.R., Servato, J.P.S., Neves, L.A., Roberto, G.F., Martins, A.S., do Nascimento, M.Z., Unsupervised method for normalization of hematoxylin-eosin stain in histological images. Comput. Med. Imaging Graph., 77, 2019, 101646.
Lu, C., Shi, J., Jia, J., Online robust dictionary learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, 415–422.
Li, X., Wang, K., Lyu, Y., Pan, H., Zhang, J., Stambolian, D., Susztak, K., Reilly, M.P., Hu, G., Li, M., Deep learning enables accurate clustering with batch effect removal in single-cell RNA-seq analysis. Nat. Commun. 11 (2020), 1–14.
Blondel, V.D., Guillaume, J.-.L., Lambiotte, R., Lefebvre, E., Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp., 2008, P10008 2008.
Wang, T., Johnson, T.S., Shao, W., Lu, Z., Helm, B.R., Zhang, J., Huang, K., BERMUDA: a novel deep transfer learning method for single-cell RNA sequencing batch correction reveals hidden high-resolution cellular subtypes. Genome Biol. 20 (2019), 1–15.
Guan, H., Liu, Y., Yang, E., Yap, P.-.T., Shen, D., Liu, M., Multi-site MRI harmonization via attention-guided deep domain adaptation for brain disorder identification. Med. Image Anal., 71, 2021, 102076.
Dinsdale, N.K., Jenkinson, M., Namburete, A.I., Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal. Neuroimage, 228, 2021, 117689.
Ge, S., Wang, H., Alavi, A., Xing, E., Bar-Joseph, Z., Supervised adversarial alignment of single-cell RNA-seq data. J. Comput. Biol. 28 (2021), 501–513.
Rong, Z., Tan, Q., Cao, L., Zhang, L., Deng, K., Huang, Y., Zhu, Z.-.J., Li, Z., Li, K., NormAE: deep adversarial learning model to remove batch effects in liquid chromatography mass spectrometry-based metabolomics data. Anal. Chem. 92 (2020), 5082–5090.
Büttner, M., Miao, Z., Wolf, F.A., Teichmann, S.A., Theis, F.J., A test metric for assessing single-cell RNA-seq batch correction. Nat. Methods 16 (2019), 43–49.
Kolaman, A., Yadid-Pecht, O., Quaternion structural similarity: a new quality index for color images. IEEE Trans. Image Process. 21 (2011), 1526–1536.
Zhang, L., Zhang, L., Mou, X., Zhang, D., FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20 (2011), 2378–2386.
Pambrun, J.-.F., Noumeir, R., Limitations of the SSIM quality metric in the context of diagnostic imaging. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), 2015, IEEE, 2960–2963.
Nyúl, L.G., Udupa, J.K., Zhang, X., New variants of a method of MRI scale standardization. IEEE Trans. Med. Imaging 19 (2000), 143–150.
Albert, A., Zhang, L., A novel definition of the multivariate coefficient of variation. Biomet. J. 52 (2010), 667–675.
Chirra, P., Leo, P., Yim, M., Bloch, B.N., Rastinehad, A.R., Purysko, A., Rosen, M., Madabhushi, A., Viswanath, S.E., Multisite evaluation of radiomic feature reproducibility and discriminability for identifying peripheral zone prostate tumors on MRI. J. Med. Imaging, 6, 2019, 024502.
Lawrence, I., Lin, K., A concordance correlation coefficient to evaluate reproducibility. Biometrics, 1989, 255–268.
Liljequist, D., Elfving, B., Skavberg Roaldsen, K., Intraclass correlation–a discussion and demonstration of basic features. PLoS ONE, 14, 2019, e0219854.
Orlhac, F., Lecler, A., Savatovski, J., Goya-Outi, J., Nioche, C., Charbonneau, F., Ayache, N., Frouin, F., Duron, L., Buvat, I., How can we combat multicenter variability in MR radiomics? Validation of a correction procedure. Eur. Radiol. 31 (2021), 2272–2280.
Mahon, R., Ghita, M., Hugo, G., Weiss, E., ComBat harmonization for radiomic features in independent phantom and lung cancer patient computed tomography datasets. Phy. Med. Biol., 65, 2020, 015010.
Ioannidis, G.S., Trivizakis, E., Metzakis, I., Papagiannakis, S., Lagoudaki, E., Marias, K., Pathomics and deep learning classification of a heterogeneous fluorescence histology image dataset. Appl. Sci., 11, 2021, 3796.
Jiang, X., Bian, G.-.B., Tian, Z., Removal of artifacts from EEG signals: a review. Sensors, 19, 2019, 987.
He, P., Kahle, M., Wilson, G., Russell, C., Removal of ocular artifacts from EEG: a comparison of adaptive filtering method and regression method using simulated data. Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, 2006, IEEE, 1110–1113.
Kumar, P.S., Arumuganathan, R., Sivakumar, K., Vimal, C., Removal of ocular artifacts in the EEG through wavelet transform without using an EOG reference channel. Int. J. Open Problems Compt. Math 1 (2008), 188–200.
Aerts, H.J., Velazquez, E.R., Leijenaar, R.T., Parmar, C., Grossmann, P., Carvalho, S., Bussink, J., Monshouwer, R., Haibe-Kains, B., Rietveld, D., Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat. Commun. 5 (2014), 1–9.
Arrieta, A.B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inform. Fusion 58 (2020), 82–115.
Yang, G., Ye, Q., Xia, J., Unbox the black-box for the medical explainable ai via multi-modal and multi-centre data fusion: a mini-review, two showcases and beyond. Inform. Fusion 77 (2022), 29–52.
Holzinger, A., Dehmer, M., Emmert-Streib, F., Cucchiara, R., Augenstein, I., Del Ser, J., Samek, W., Jurisica, I., Díaz-Rodríguez, N., Information fusion as an integrative cross-cutting enabler to achieve robust, explainable, and trustworthy medical artificial intelligence. Inform. Fusion 79 (2022), 263–278.
Mongan, John, Linda Moy, and Charles E. Kahn Jr. "Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers." Radiology: Artificial Intelligence 2.2 (2020): e200029.
St‐Jean, S., Viergever, M.A., Leemans, A., Harmonization of diffusion MRI data sets with adaptive dictionary learning. Human brain mapping 41:16 (2020), 4478–4499.
