Enhancing Interactive Annotation in Digital Pathology with Foundation Models

Manual annotation of large-scale biomedical images is a major bottleneck in digital pathology, since it requires significant expert time and effort. In this paper, we explore the use of foundation models for segmentation to facilitate and accelerate this annotation process. More particularly, we investigate the Segment Anything Model (SAM) and its successor SAM2, evaluating their performance across three in-house datasets and one public dataset which cover diverse segmentation tasks. In addition, we design and evaluate new architectures that build on these models and also incorporate domain-specific pathology encoders. Our experiments show that domain adaptation substantially improves segmentation results across various structures, with fine-tuning yielding the strongest gains and hybrid designs offering a complementary benefit, paving the way for more effective interactive annotation. Furthermore, we integrate the best model we obtained into an open-source, user-friendly, web platform.