Active Learning; Cancer tissues; Computational evaluation; Digital pathologies; Gland segmentations; Human expert; Images segmentations; Real-world; Semi-active; Semi-supervised learning; Artificial Intelligence; Computer Science Applications; Computer Vision and Pattern Recognition
Abstract :
[en] Real-world segmentation tasks in digital pathology require a great effort from human experts to accurately annotate a sufficiently high number of images. Hence, there is a huge interest in methods that can make use of non-annotated samples, to alleviate the burden on the annotators. In this work, we evaluate two classes of such methods, semi-supervised and active learning, and their combination on a version of the GlaS dataset for gland segmentation in colorectal cancer tissue with missing annotations. Our results show that semi-supervised learning benefits from the combination with active learning and outperforms fully supervised learning on a dataset with missing annotations. However, an active learning procedure alone with a simple selection strategy obtains results of comparable quality.
Disciplines :
Computer science
Author, co-author :
Jiménez, Laura Gálvez; Université Libre de Bruxelles, Brussels, Belgium
Dierckx, Lucile; Université Catholique de Louvain, Louvain-La-Neuve, Belgium
Amodei, Maxime ; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Khosroshahi, Hamed Razavi; Université Libre de Bruxelles, Brussels, Belgium
Chidambaran, Natarajan; Université de Mons, Mons, Belgium
Ho, Anh-Thu Phan; Multitel, Mons, Belgium
Franzin, Alberto; Université Libre de Bruxelles, Brussels, Belgium
Language :
English
Title :
Computational Evaluation of the Combination of Semi-Supervised and Active Learning for Histopathology Image Segmentation with Missing Annotations
Publication date :
2023
Event name :
2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
Event place :
Paris, Fra
Event date :
02-10-2023 => 06-10-2023
Main work title :
Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023
Publisher :
Institute of Electrical and Electronics Engineers Inc.
ThisworkwassupportedbyServicePublicdeWal-lonie Recherche under grant n°2010235 - ARIAC by DIGI-TALWALLONIA4.AI. Computationalresourceshave been provided by the Consortium des Équipements de Calcul Intensif (CÉCI), funded by the Fonds de la Recherche ScientifiquedeBelgique(F.R.S.-FNRS) underGrantNo. 2.5020.11 and by the Walloon Region. We thank Prof. Christine Decaestecker and Charlotte Nachtegel for the use-fuldiscussionsandcomments.
Fan Bai, Xiaohan Xing, Yutian Shen, Han Ma, and Max Q-H Meng. Discrepancy-based active learning for weakly supervised bleeding segmentation in wireless capsule endoscopy images. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2022: 25th International Conference, Singapore, September 18-22, 2022, Proceedings, Part VIII, pages 24-34. Springer, 2022.
Wenjia Bai, Ozan Oktay, Matthew Sinclair, Hideaki Suzuki, Martin Rajchl, Giacomo Tarroni, Ben Glocker, Andrew King, Paul M Matthews, and Daniel Rueckert. Semisupervised learning for network-based cardiac MR image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II 20, pages 253-260. Springer, 2017.
Péter Bándi, Rob van de Loo, Milad Intezar, Daan Geijs, Francesco Ciompi, Bram van Ginneken, Jeroen van der Laak, and Geert Litjens. Comparison of different methods for tissue segmentation in histopathological whole-slide images. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pages 591-595. IEEE, 2017.
Gerda Bortsova, Florian Dubost, Laurens Hogeweg, Ioannis Katramados, and Marleen De Bruijne. Semi-supervised medical image segmentation via learning consistency under transformations. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part VI 22, pages 810-818. Springer, 2019.
Gloria Bueno, M Milagro Fernandez-Carrobles, Lucia Gonzalez-Lopez, and Oscar Deniz. Glomerulosclerosis identification in whole slide images using semantic segmentation. Computer methods and programs in biomedicine, 184:105273, 2020.
Lile Cai, Xun Xu, Jun Hao Liew, and Chuan Sheng Foo. Revisiting superpixels for active learning in semantic segmentation with realistic annotation costs. In Conference on Computer Vision and Pattern Recognition, pages 10988-10997. IEEE, 2021.
Arantxa Casanova, Pedro O Pinheiro, Negar Rostamzadeh, and Christopher J Pal. Reinforced active learning for image segmentation. arXiv preprint arXiv:2002.06583, 2020.
Krishna Chaitanya, Neerav Karani, Christian F Baumgartner, Anton Becker, Olivio Donati, and Ender Konukoglu. Semisupervised and task-driven data augmentation. In Information Processing in Medical Imaging: 26th International Conference, IPMI 2019, Hong Kong, China, June 2-7, 2019, Proceedings 26, pages 29-41. Springer, 2019.
Lyndon Chan, Mahdi S Hosseini, and Konstantinos N Plataniotis. A comprehensive analysis of weakly-supervised semantic segmentation in different image domains. International Journal of Computer Vision, 129:361-384, 2021.
Lyndon Chan, Mahdi S Hosseini, Corwyn Rowsell, Konstantinos N Plataniotis, and Savvas Damaskinos. Histosegnet: Semantic segmentation of histological tissue type in whole slide images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10662-10671, 2019.
Veronika Cheplygina, Marleen de Bruijne, and Josien PW Pluim. Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Medical image analysis, 54:280-296, 2019.
Patrick Ferdinand Christ, Mohamed Ezzeldin A Elshaer, Florian Ettlinger, Sunil Tatavarty, Marc Bickel, Patrick Bilic, Markus Rempfler, Marco Armbruster, Felix Hofmann, Melvin D'Anastasi, et al. Automatic liver and lesion segmentation in ct using cascaded fully convolutional neural networks and 3d conditional random fields. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19, pages 415-423. Springer, 2016.
Ekin Dogus Cubuk, Barret Zoph, Jon Shlens, and Quoc Le. Randaugment: Practical automated data augmentation with a reduced search space. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 18613-18624. Curran Associates, Inc., 2020.
Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1-22, 1977.
Shujian Deng, Xin Zhang, Wen Yan, Eric I-Chao Chang, Yubo Fan, Maode Lai, and Yan Xu. Deep learning in digital pathology image analysis: A survey. Frontiers of medicine, 14:470-487, 2020.
Adrien Foucart, Olivier Debeir, and Christine Decaestecker. SNOW: Semi-supervised, noisy and/or weak data for deep learning in digital pathology. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pages 1869-1872. IEEE, 2019.
Adrien Foucart, Olivier Debeir, and Christine Decaestecker. SNOW supervision in digital pathology: Managing imperfect annotations for segmentation in deep learning. Submitted, 2020.
Geoffrey French, Samuli Laine, Timo Aila, Michal Mackiewicz, and Graham Finlayson. Semi-supervised semantic segmentation needs strong, varied perturbations. In British Machine Vision Conference, 2020.
Mélanie Gaillochet, Christian Desrosiers, and Hervé Lombaert. Taal: Test-time augmentation for active learning in medical image segmentation. In Data Augmentation, Labelling, and Imperfections: Second MICCAI Workshop, DALI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 22, 2022, Proceedings, pages 43-53. Springer, 2022.
Mingfei Gao, Zizhao Zhang, Guo Yu, Sercan O Ark, Larry S Davis, and Tomas Pfister. Consistency-based semisupervised active learning: Towards minimizing labeling cost. In European Conference on Computer Vision, pages 510-526. Springer, 2020.
Chengyue Gong, Dilin Wang, and Qiang Liu. Alphamatch: Improving consistency for semi-supervised learning with alpha-divergence. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13683-13692, 2021.
Yanming Guo, Yu Liu, Theodoros Georgiou, and Michael S Lew. A review of semantic segmentation using deep neural networks. International journal of multimedia information retrieval, 7:87-93, 2018.
Neil Houlsby, José Miguel Hernández-Lobato, and Zoubin Ghahramani. Cold-start active learning with robust ordinal matrix factorization. In International conference on machine learning, pages 766-774. PMLR, 2014.
Hanzhe Hu, Fangyun Wei, Han Hu, Qiwei Ye, Jinshi Cui, and Liwei Wang. Semi-supervised semantic segmentation via adaptive equalization learning. Advances in Neural Information Processing Systems, 34:22106-22118, 2021.
Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert, et al. nnu-net: Self-adapting framework for u-net-based medical image segmentation. arXiv preprint arXiv:1809.10486, 2018.
Sara Hosseinzadeh Kassani, Peyman Hosseinzadeh Kassani, Michal J Wesolowski, Kevin A Schneider, and Ralph Deters. Deep transfer learning based model for colorectal cancer histopathology segmentation: A comparative study of deep pre-trained models. International Journal of Medical Informatics, 159:104669, 2022.
Zhengfeng Lai, Chao Wang, Zin Hu, Brittany N Dugger, Sen-Ching Cheung, and Chen-Nee Chuah. A semi-supervised learning for segmentation of gigapixel histopathology images from brain tissues. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 1920-1923. IEEE, 2021.
Zhengfeng Lai, Chao Wang, Luca Cerny Oliveira, Brittany N Dugger, Sen-Ching Cheung, and Chen-Nee Chuah. Joint semi-supervised and active learning for segmentation of gigapixel pathology images with cost-effective labeling. In International Conference on Computer Vision, pages 591-600. IEEE, 2021.
Samuli Laine and Timo Aila. Temporal ensembling for semisupervised learning. In International Conference on Learning Representations, 2017.
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, page 896, 2013.
Haohan Li and Zhaozheng Yin. Attention, suggestion and annotation: A deep active learning framework for biomedical image segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention-MICCAI, pages 3-13. Springer, 2020.
Jiayun Li, Karthik V Sarma, King Chung Ho, Arkadiusz Gertych, Beatrice S Knudsen, and CoreyWArnold. A multiscale u-net for semantic segmentation of histological images from radical prostatectomies. In AMIA Annual Symposium Proceedings, volume 2017, page 1140. American Medical Informatics Association, 2017.
Xiaomeng Li, Lequan Yu, Hao Chen, Chi-Wing Fu, Lei Xing, and Pheng-Ann Heng. Transformation-consistent selfensembling model for semisupervised medical image segmentation. IEEE Transactions on Neural Networks and Learning Systems, 32(2):523-534, 2020.
Shaobo Min, Xuejin Chen, Zheng-Jun Zha, Feng Wu, and Yongdong Zhang. A two-stream mutual attention network for semi-supervised biomedical segmentation with noisy labels. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, pages 4578-4585, 2019.
Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos. Image segmentation using deep learning: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(7):3523-3542, 2021.
Romain Mormont, Mehdi Testouri, Raphäel Marée, and Pierre Geurts. Relieving pixel-wise labeling effort for pathology image segmentation with self-training. In European Conference in Compter Vision (ECCV2022). Springer Cham, Geneve, Switzerland, 2022.
Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. Advances in neural information processing systems, 31, 2018.
George Papandreou, Liang-Chieh Chen, Kevin P Murphy, and Alan L Yuille. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In Proceedings of the IEEE international conference on computer vision, pages 1742-1750, 2015.
Olivier Petit, Nicolas Thome, Arnaud Charnoz, Alexandre Hostettler, and Luc Soler. Handling missing annotations for semantic segmentation with deep convnets. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, pages 20-28. Springer, 2018.
Emmanouil Antonios Platanios, Maruan Al-Shedivat, Eric Xing, and Tom Mitchell. Learning from imperfect annotations. arXiv preprint arXiv:2004.03473, 2020.
Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, and Kaiming He. Data distillation: Towards omnisupervised learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4119-4128, 2018.
Aneesh Rangnekar, Christopher Kanan, and Matthew Hoffman. Semantic segmentation with active semi-supervised learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5966-5977, 2023.
Hatice Catal Reis and Veysel Turk. Transfer learning approach and nucleus segmentation with medclnet colon cancer database. Journal of Digital Imaging, 36(1):306-325, 2023.
Phill Kyu Rhee, Enkhbayar Erdenee, Shin Dong Kyun, Minhaz Uddin Ahmed, and Songguo Jin. Active and semisupervised learning for object detection with imperfect data. Cognitive Systems Research, 45:109-123, 2017.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. Unet: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer, 2015.
Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 29, 2016.
Dhruv Sharma, Zahil Shanis, Chandan K Reddy, Samuel Gerber, and Andinet Enquobahrie. Active learning technique for multimodal brain tumor segmentation using limited labeled images. In Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data: First MICCAI Workshop, DART 2019, and First International Workshop, MIL3ID 2019, Held in Conjunction with MICCAI, pages 148-156. Springer, 2019.
Korsuk Sirinukunwattana, Josien PW Pluim, Hao Chen, Xiaojuan Qi, Pheng-Ann Heng, Yun Bo Guo, Li Yang Wang, Bogdan J Matuszewski, Elia Bruni, Urko Sanchez, et al. Gland segmentation in colon histology images: The glas challenge contest. Medical image analysis, 35:489-502, 2017.
Korsuk Sirinukunwattana, David RJ Snead, and Nasir M Rajpoot. A stochastic polygons model for glandular structures in colon histology images. IEEE transactions on medical imaging, 34(11):2366-2378, 2015.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.
Shuang Song, David Berthelot, and Afshin Rostamizadeh. Combining mixmatch and active learning for better accuracy with fewer labels. arXiv preprint arXiv:1912.00594, 2019.
Chetan L Srinidhi, Ozan Ciga, and Anne L Martel. Deep neural network models for computational histopathology: A survey. Medical Image Analysis, 67:101813, 2021.
Nima Tajbakhsh, Laura Jeyaseelan, Qian Li, Jeffrey N Chiang, Zhihao Wu, and Xiaowei Ding. Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Medical Image Analysis, 63:101693, 2020.
You-Bao Tang, Sooyoun Oh, Yu-Xing Tang, Jing Xiao, and Ronald M Summers. Ct-realistic data augmentation using generative adversarial network for robust lymph node segmentation. In Medical Imaging 2019: Computer-Aided Diagnosis, volume 10950, pages 976-981. SPIE, 2019.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
Hamid Reza Tizhoosh and Liron Pantanowitz. Artificial intelligence and digital pathology: Challenges and opportunities. Journal of pathology informatics, 9(1):38, 2018.
Katrin Tomanek and Udo Hahn. Semi-supervised active learning for sequence labeling. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2, page 1039-1047, USA, 2009. Association for Computational Linguistics. 3
Pratima Upretee and Bishesh Khanal. Fixmatchseg: Fixing fixmatch for semi-supervised semantic segmentation. arXiv preprint arXiv:2208.00400, 2022.
Jeroen Van der Laak, Geert Litjens, and Francesco Ciompi. Deep learning in histopathology: The path to the clinic. Nature medicine, 27(5):775-784, 2021.
Jesper E. van Engelen and Holger H. Hoos. A survey on semi-supervised learning. Machine Learning, 109:373-440, 2019.
Yves-Rémi Van Eycke, Adrien Foucart, and Christine Decaestecker. Strategies to reduce the expert supervision required for deep learning-based segmentation of histopathological images. Frontiers in medicine, page 222, 2019.
Noorul Wahab, Asifullah Khan, and Yeon Soo Lee. Transfer learning based deep cnn for segmentation and detection of mitoses in breast cancer histopathological images. Microscopy, 68(3):216-233, 2019.
Jiazhuo Wang, John D MacKenzie, Rageshree Ramachandran, and Danny Z Chen. A deep learning approach for semantic segmentation in histology tissue images. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19, pages 176-184. Springer, 2016.
Jun Wang, Shaoguo Wen, Kaixing Chen, Jianghua Yu, Xin Zhou, Peng Gao, Changsheng Li, and Guotong Xie. Semisupervised active learning for instance segmentation via scoring predictions. In British Machine Vision Conference (BMVC2020), 2020.
Huimin Wu, Xiaomeng Li, Yiqun Lin, and Kwang-Ting Cheng. Compete to win: Enhancing pseudo labels for barelysupervised medical image segmentation. IEEE Transactions on Medical Imaging, 2023.
Yingda Xia, Fengze Liu, Dong Yang, Jinzheng Cai, Lequan Yu, Zhuotun Zhu, Daguang Xu, Alan Yuille, and Holger Roth. 3d semi-supervised learning with uncertainty-aware multi-view co-training. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3646-3655, 2020.
Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. Advances in neural information processing systems, 33:6256-6268, 2020.
Yi Xu, Lei Shang, Jinxing Ye, Qi Qian, Yu-Feng Li, Baigui Sun, Hao Li, and Rong Jin. Dash: Semi-supervised learning with dynamic thresholding. In International Conference on Machine Learning, pages 11525-11536. PMLR, 2021.
Lihe Yang, Lei Qi, Litong Feng, Wayne Zhang, and Yinghuan Shi. Revisiting weak-to-strong consistency in semi-supervised semantic segmentation. In CVPR, 2023.
Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, and Danny Z Chen. Suggestive annotation: A deep active learning framework for biomedical image segmentation. In Medical Image Computing and Computer Assisted Intervention-MICCAI, pages 399-407. Springer, 2017.
Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, and Pheng-Ann Heng. Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part II 22, pages 605-613. Springer, 2019.
Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Advances in Neural Information Processing Systems, 34:18408-18419, 2021.
Ling Zhang, Vissagan Gopalakrishnan, Le Lu, Ronald M Summers, Joel Moss, and Jianhua Yao. Self-learning to detect and segment cysts in lung ct images without manual annotation. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 1100-1103. IEEE, 2018.
Ling Zhang, Xiaosong Wang, Dong Yang, Thomas Sanford, Stephanie Harmon, Baris Turkbey, Holger Roth, Andriy Myronenko, Daguang Xu, and Ziyue Xu. When unseen domain generalization is unnecessary rethinking data augmentation. arXiv preprint arXiv:1906.03347, 2019.
Ziyuan Zhao, Zeng Zeng, Kaixin Xu, Cen Chen, and Cuntai Guan. Dsal: Deeply supervised active learning from strong and weak labelers for biomedical image segmentation. IEEE journal of biomedical and health informatics, 25(10):3744-3751, 2021.
Yuyin Zhou, Yan Wang, Peng Tang, Song Bai, Wei Shen, Elliot Fishman, and Alan Yuille. Semi-supervised 3d abdominal multi-organ segmentation via deep multi-planar cotraining. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 121-140. IEEE, 2019.
Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, and Tomas Pfister. Pseudoseg: Designing pseudo labels for semantic segmentation. arXiv preprint arXiv:2010.09713, 2020.
