Enhancing Interactive Annotation in Digital Pathology with Foundation Models

Manual annotation of large-scale biomedical images
is a major bottleneck in digital pathology, since it requires
significant expert time and effort. In this paper, we explore the
use of foundation models for segmentation to facilitate and accel-
erate this annotation process. More particularly, we investigate
the Segment Anything Model (SAM) and its successor SAM2,
evaluating their performance across three in-house datasets and
one public dataset which cover diverse segmentation tasks. In
addition, we design and evaluate new architectures that build
on these models and also incorporate domain-specific pathology
encoders. Our experiments show that domain adaptation substan-
tially improves segmentation results across various structures,
with fine-tuning yielding the strongest gains and hybrid designs
offering a complementary benefit, paving the way for more
effective interactive annotation. Furthermore, we integrate the
best model we obtained into an open-source, user-friendly, web
platform.