Deep Learning, Multi-modal image analysis, landmark detection, artificial intelligence, bio images, fish
Résumé :
[en] In this paper we perform an empirical evaluation of variants of deep learning methods to automatically localize anatomical landmarks in bioimages of fishes acquired using different imaging modalities (microscopy and radiography). We compare two methodologies namely heatmap based regression and multivariate direct regression, and evaluate them in combination with several Convolutional Neural Network (CNN) architectures. Heatmap based regression approaches employ Gaussian or Exponential heatmap generation functions combined with CNNs to output the heatmaps corresponding to landmark locations whereas direct regression approaches output directly the (x, y) coordinates corresponding to landmark locations. In our experiments, we use two microscopy datasets of Zebrafish and Medaka fish and one radiography
dataset of gilthead Seabream. On our three datasets, the heatmap approach with Exponential function and U-Net architecture performs better. Datasets and open-source code for training and prediction are made available to ease future landmark detection research and bioimaging applications.
Disciplines :
Sciences informatiques Sciences de la santé humaine: Multidisciplinaire, généralités & autres
Auteur, co-auteur :
Kumar, Navdeep ; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Claudia Di Biagio; University of Rome Tor Vergata > Biology
Zachary Dellacqua; University of Rome Tor Vergata > Biology
Raman, Ratish ; Université de Liège - ULiège > GIGA ; Université de Liège - ULiège > Département des sciences de la vie ; Université de Liège - ULiège > GIGA > GIGA I3 - Laboratory for Organogenesis and Regeneration ; Université de Liège - ULiège > Département des sciences de la vie > GIGA-R : Biologie et génétique moléculaire
Arianna Martini; CREA - Council for Agricultural Research and Economics Research Centre for Animal Production and Aquaculture
Clara Boglione; University of Rome Tor Vergata > Biology
Muller, Marc ; Université de Liège - ULiège > GIGA > GIGA I3 - Laboratory for Organogenesis and Regeneration
Geurts, Pierre ; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Marée, Raphaël ; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Langue du document :
Anglais
Titre :
Empirical Evaluation of Deep Learning Approaches for Landmark Detection in Fish Bioimages
Date de publication/diffusion :
14 février 2023
Nom de la manifestation :
Bio Image Computing Workshop in European Conference on Computer Vision 2022
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, software available from tensorflow.org
Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2d human pose estimation: new benchmark and state of the art analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3686–3693 (2014)
Aubert, B., Vazquez, C., Cresson, T., Parent, S., De Guise, J.: Automatic spine and pelvis detection in frontal x-rays using deep neural networks for patch displacement learning. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1426–1429. IEEE (2016)
Bookstein, F.L.: Combining the tools of geometric morphometrics. In: Marcus, L.F., Corti, M., Loy, A., Naylor, G.J.P., Slice, D.E. (eds) Advances in Morphometrics. NATO ASI Series, vol 284, pp. 131–151. Springer, Boston (1996). https://doi. org/10.1007/978-1-4757-9083-2 12(1996)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. Ieee (2009)
Fragkoulis, S., Printzi, A., Geladakis, G., Katribouzas, N., Koumoundouros, G.: Recovery of haemal lordosis in gilthead seabream (sparus aurata l.). Sci. Rep. 9(1), 1–11 (2019)
Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9 6
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, W., Yang, C., Hou, T.: Spine landmark localization with combining of heatmap regression and direct coordinate regression. arXiv preprint arXiv:2007.05355 (2020)
Ibragimov, B., Vrtovec, T.: Landmark-based statistical shape representations. In: Statistical Shape and Deformation Analysis, pp. 89–113. Elsevier (2017)
Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., Schiele, B.: DeeperCut: a deeper, stronger, and faster multi-person pose estimation model. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 34–50. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4 3
Jarque, S., Rubio-Brotons, M., Ibarra, J., Ordoñez, V., Dyballa, S., Miñana, R., Terriente, J.: Morphometric analysis of developing zebrafish embryos allows predicting teratogenicity modes of action in higher vertebrates. Reprod. Toxicol. 96, 337–348 (2020)
Khabarlak, K., Koriashkina, L.: Fast facial landmark detection and applications: A survey. arXiv preprint arXiv:2101.10808 (2021)
Lee, H., Park, M., Kim, J.: Cephalometric landmark detection in dental x-ray images using convolutional neural networks. In: Medical Imaging 2017: Computer-Aided Diagnosis, vol. 10134, p. 101341W. International Society for Optics and Photonics (2017)
Lindner, C., Cootes, T.F.: Fully automatic cephalometric evaluation using random forest regression-voting. In: IEEE International Symposium on Biomedical Imaging (ISBI) 2015-Grand Challenges in Dental X-ray Image Analysis-Automated Detection and Analysis for Diagnosis in Cephalometric X-ray Image (2015)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Loy, B.A., Boglione, C., Cataudella, S.: Geometric morphometrics and morpho-anatomy: a combined tool in the study of sea bream (sparus aurata, sparidae) shape. J. Appl. Ichthyol. 15(3), 104–110 (1999)
Mahpod, S., Das, R., Maiorana, E., Keller, Y., Campisi, P.: Facial landmarks localization using cascaded neural networks. Comput. Vis. Image Underst. 205, 103171 (2021)
Marée, R., Rollus, L., Stévens, B., Hoyoux, R., Louppe, G., Vandaele, R., Begon, J.M., Kainz, P., Geurts, P., Wehenkel, L.: Collaborative analysis of multi-gigapixel imaging data using cytomine. Bioinformatics 32(9), 1395–1401 (2016)
Mathis, A., Mamidanna, P., Cury, K.M., Abe, T., Murthy, V.N., Mathis, M.W., Bethge, M.: Deeplabcut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21(9), 1281–1289 (2018)
Mohseni, H., Kasaei, S.: Automatic localization of cephalometric landmarks. In: 2007 IEEE International Symposium on Signal Processing and Information Technology, pp. 396–401. IEEE (2007)
Park, J.H., Hwang, H.W., Moon, J.H., Yu, Y., Kim, H., Her, S.B., Srinivasan, G., Aljanabi, M.N.A., Donatelli, R.E., Lee, S.J.: Automated identification of cephalometric landmarks: Part 1-comparisons between the latest deep-learning methods yolov3 and ssd. Angle Orthod. 89(6), 903–909 (2019)
Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8 27
Payer, C., Štern, D., Bischof, H., Urschler, M.: Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Med. Image Anal. 54, 207–219 (2019)
Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015)
Riegler, G., Urschler, M., Ruther, M., Bischof, H., Stern, D.: Anatomical landmark detection in medical applications driven by synthetic data. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 12–16 (2015)
Samet, N., Akbas, E.: Hprnet: hierarchical point regression for whole-body human pose estimation. Image Vis. Comput. 115, 104285 (2021)
Song, Y., Qiao, X., Iwamoto, Y., Chen, Y.W.: Automatic cephalometric landmark detection on x-ray images using a deep-learning method. Appl. Sci. 10(7), 2547 (2020)
Stern, O., Marée, R., Aceto, J., Jeanray, N., Muller, M., Wehenkel, L., Geurts, P.: Automatic localization of interest points in zebrafish images with tree-based methods. In: Loog, M., Wessels, L., Reinders, M.J.T., de Ridder, D. (eds.) PRIB 2011. LNCS, vol. 7036, pp. 179–190. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24855-9 16
Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5693–5703 (2019)
Torosdagli, N., Liberton, D.K., Verma, P., Sincan, M., Lee, J.S., Bagci, U.: Deep geodesic learning for segmentation and anatomical landmarking. IEEE Trans. Med. Imaging 38(4), 919–931 (2018)
Vandaele, R., Aceto, J., Muller, M., Peronnet, F., Debat, V., Wang, C.W., Huang, C.T., Jodogne, S., Martinive, P., Geurts, P., et al.: Landmark detection in 2d bioimages for geometric morphometrics: a multi-resolution tree-based approach. Sci. Rep. 8(1), 1–13 (2018)
Verhaegen, Y., Adriaens, D., De Wolf, T., Dhert, P., Sorgeloos, P.: Deformities in larval gilthead sea bream (sparus aurata): a qualitative and quantitative analysis using geometric morphometrics. Aquaculture 268(1–4), 156–168 (2007)
Weinhardt, V., Shkarin, R., Wernet, T., Wittbrodt, J., Baumbach, T., Loosli, F.: Quantitative morphometric analysis of adult teleost fish by x-ray computed tomography. Sci. Rep. 8(1), 1–12 (2018)
Xu, Z., Li, B., Yuan, Y., Geng, M.: Anchorface: an anchor-based facial landmark detector across large poses. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 3092–3100 (2021)
Yeh, Y.C., Weng, C.H., Huang, Y.J., Fu, C.J., Tsai, T.T., Yeh, C.Y.: Deep learning approach for automatic landmark detection and alignment analysis in whole-spine lateral radiographs. Sci. Rep. 11(1), 1–15 (2021)
