The need for careful data collection for pattern recognition in digital pathology

[en] Effective pattern recognition requires carefully designed ground-truth datasets. In this technical note, we first summarize potential data collection issues in digital pathology and then propose guidelines to build more realistic ground-truth datasets and to control their quality. We hope our comments will foster the effective application of pattern recognition approaches in digital pathology.

Research Center/Unit :

Montefiore Institute - Montefiore Institute of Electrical Engineering and Computer Science - ULiège

Disciplines :

Computer science

Author, co-author :

Marée, Raphaël ; Université de Liège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation

Language :

English

Title :

The need for careful data collection for pattern recognition in digital pathology

Publication date :

2017

Journal title :

Journal of Pathology Informatics

ISSN :

2229-5089

eISSN :

2153-3539

Publisher :

Wolters Kluwer Medknow

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

http://www.jpathinformatics.org/

Name of the research project :

Histoweb

Funders :

DGTRE - Région wallonne. Direction générale des Technologies, de la Recherche et de l'Énergie

Available on ORBi :

since 05 March 2017

Statistics

Number of views

334 (24 by ULiège)

Number of downloads

369 (19 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenAlex citations

Bibliography

Fuchs TJ, Buhmann JM. Computational pathology: Challenges and promises for tissue analysis. Comput Med Imaging Graph 2011;35:515-30.
Kothari S, Phan JH, Stokes TH, Wang MD. Pathology imaging informatics for quantitative analysis of whole-slide images. J Am Med Inform Assoc 2013;20:1099-108.
McCann MT, Castro C, Ozolek JA, Parvin B, Kovacevic J. Automated Histology Analysis: Opportunities for Signal Processing. IEEE Signal Processing; 2014.
Ponce J, Berg TL, Everingham M, Forsyth DA, Hebert M, Lazebnik S, et al. Dataset issues in object recognition. Toward Category Level Object Recognition. Ch. 2. Switzerland: Springer-Verlag Lecture Notes in Computer Scienc; 2006.
Herve N, Boujemaa N. Image Annotation: Which Approach for Realistic Databases? In Proceeding ACM International Conference on Image and Video Retrieval (CIVR); 2007. p. 170-7.
Shamir L. Evaluation of face datasets as tools for assessing the performance of face recognition methods. Int J Comput Vis 2008;79:225-230.
Kumar N, Berg AC, Belhumeur PN, Nayar SK. Attribute and Simile Classifiers for Face Verification. In IEEE International Conference on Computer Vision (ICCV) October, 2009.
Cox DD, Pinto N, Barhomi Y, DiCarlo JJ. Comparing State-of-the-Art Visual Features on Invariant Object Recognition Tasks. In Proceeding IEEE Workshop on Applications of Computer Vision (WACV); 2011.
Torralba A, Efros A. Unbiased Look at Dataset Bias. In Proceeding IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2011.
Shamir L. Assessing the efficacy of low-level image content descriptors for computer-based fluorescence microscopy image analysis. J Microsc 2011;243:284-92.
Model I, Shamir L. Comparison of dataset bias in object recognition benchmarks. IEEE Access 2015;3:1953-62.
Harder N, Batra R, Diessl N, Gogolin S, Eils R, Westermann F, et al. Large-scale tracking and classification for automatic analysis of cell migration and proliferation, and experimental optimization of high-throughput screens of neuroblastoma cells. Cytometry A 2015;87:524-40.
Lezoray O, Elmoataz A, Cardot H. A color object recognition scheme: Application to cellular sorting. Mach Vis Appl 2003;14:166-71.
Marinakis Y, Dounias G, Jantzen J. Pap smear diagnosis using a hybrid intelligent scheme focusing on genetic algorithm based feature selection and nearest neighbor classification. Comput Biol Med 2009;39:69-78.
Bengtsson E, Malm P. Screening for cervical cancer using automated analysis of PAP-smears. Comput Math Methods Med 2014;2014:842037.
Bejnordi BE, Timofeeva N, Otte-Holler I, Karssemeijer N, van der Laak J. Quantitative Analysis of Stain Variability in Histology Slides and an Algorithm for Standardization. Proceeding SPIE - The International Society for Optical Engineering (9041); 2014.
Sahay K, Mehendiratta M, Rehani S, Kumra M, Sharma R, Kardam P. Cytological artifacts masquerading interpretation. J Cytol 2013;30:241-6.
Mcinnes EF. Artefacts in histology. Comp Clin Pathol 2005;13:100-8.
Marien K. The Search for a Predictive Tissue Biomarker for Response to Colon Cancer Therapy with Bevacizumab. Doctoral Thesis, Universiteit Antwerpen; 2016.
Delga A, Goffin F, Kridelka F, Marée R, Lambert C, Delvenne P. Evaluation of CellSolutions BestPrep ® automated thin-layer liquid-based cytology Papanicolaou slide preparation and BestCyte ® cell sorter imaging system. Acta Cytol 2014;58:469-77.
Giusti A, Claudiu D, Caccia C, Schmid-Huber J, Gambardella LM. A Comparison of Algorithms and Humans for Mitosis Detection. In Proceeding International Symposium on Biomedical Imaging (ISBI); 2014.
Kozubek M. Challenges and benchmarks in bioimage analysis. Adv Anat Embryol Cell Biol 2016;219:231-62.
Shamir L, Delaney JD, Orlov N, Eckley DM, Goldberg IG. Pattern recognition software and techniques for biological image analysis. PLoS Comput Biol 2010;6:e1000974.
Bradley AP. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit 1997;30:1145-935.
Marée R, Rollus L, Stévens B, Hoyoux R, Louppe G, Vandaele R, et al. Collaborative analysis of multi-gigapixel imaging data using Cytomine. Bioinformatics 2016;32:1395-401.
Jeanray N, Marée R, Pruvot B, Stern O, Geurts P, Wehenkel L, et al. Phenotype classification of zebrafish embryos by supervised learning. PLoS One 2015;10:e0116989.
Krig S. Ground truth data, content, metrics, and analysis. In Computer Vision Metrics. Ch. 7. Switzerland:Springer; 2016.
Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND. Dataset Shift in Machine Learning. US: The MIT Press; 2008.
Vittayakorn S, Hays J. Quality Assessment for Crowdsourced Object Annotations. In Proceeding of the British Machine Vision Conference, BMVA Press; September, 2011. p. 109.1-109.11.
Shamir L, Macura T, Orlov N, Eckely DM, Goldberg IG. Iicbu 2008 - A Benchmark Suite for Biological Imaging. In 3rd Workshop on Bio-Image Informatics Biological Imaging, Computer Vision and Data Mining; 2008.
Maree R, Geurts P, Wehenkel L. Towards generic image classification using tree-based learning: An extensive empirical study. Pattern Recognit Lett 2016;74:17-23.
Swamidoss IN, Kårsnäs A, Uhlmann V, Ponnusamy P, Kampf C, Simonsson M, et al. Automated classification of immunostaining patterns in breast tissue from the human protein atlas. J Pathol Inform 2013;4 Suppl:S14.
Donida Labati R, Piuri V, Scotti F. All-idb: The Acute Lymphoblastic Leukemia Image Database for Image Processing. In IEEE International Conference on Image Processing (ICIP) September, 2011.
Janowczyk A, Madabhushi A. Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. J Pathol Inform 2016;7:29.
Goodfellow I, Bengio Y, Courville A. Practical methodology. Deep Learning. Ch. 11. US: MIT Press; 2016.