This technical documentation and research paper introduce SAMRI (Segment Anything Model for MRI), an MRI-specialized adaptation of Meta AI’s Vision Transformer (ViT)-based Segment Anything Model (SAM). Manual MRI segmentation is labor-intensive and prone to inter-observer variability, while traditional CNNs often fail to generalize across variable MRI contrasts and protocols. SAMRI addresses these gaps through an efficient two-stage training strategy: precomputing image embeddings with a frozen SAM encoder followed by fine-tuning only the lightweight mask decoder. Trained on a massive corpus of 1.1 million labeled MRI slices spanning 47 tasks and 36 datasets, SAMRI achieves a state-of-the-art mean Dice Similarity Coefficient (DSC) of 0.87. Notably, it delivers substantial gains in segmenting small, clinically critical structures—such as microbleeds and thin cartilage—surpassing existing benchmarks like MedSAM by up to 42.4% in accuracy while requiring 94% less training time.
Technical Review: SAMRI — MRI-Specialized Segment Anything Model
[Introduction/Problem Statement] Generalization Challenges in MRI: Standard CNN-based architectures (U-Net and variants) lack scalability in dynamic clinical environments due to a requirement for complete retraining when protocols or label sets change. MRI specifically presents challenges including intensity inhomogeneity and the presence of small objects (<10mm) that occupy minimal pixel space.
[Technical Architecture] Two-Stage Training Pipeline: SAMRI utilizes a Vision Transformer (ViT-B/16) backbone. To optimize for MRI, the system employs a two-stage procedure:
Stage 1: Precomputes image embeddings using the frozen SAM encoder once, avoiding redundant computation.
Stage 2: Fine-tunes only the 4M-parameter mask decoder while keeping the image and prompt encoders frozen.
[Training Metrics] Computational Efficiency: This strategy results in a 94% reduction in training time and 96% fewer trainable parameters compared to full-model retraining. Total training was completed in approximately 76 hours on 8× AMD MI300X GPUs (608 GPU-hours).
[Data Scale] 1.1 Million MRI Pairs: The model was validated on a curated corpus encompassing brain, spine, knee, prostate, heart, and abdomen regions. This includes 21 datasets inherited from MedSAM and 15 new MRI-specific datasets, providing the diversity required for robust zero-shot generalization.
[Key Performance Takeaway] Small-Object Superiority: SAMRI yields the most significant gains in small structures (<0.5% image area), achieving a DSC of 0.84 compared to MedSAM’s 0.59. Specific successes include femoral cartilage (0.85 DSC where MedSAM failed entirely) and vestibular schwannoma (45% improvement).
[Loss Function] Hybrid Optimization: The model employs a combined Focal-Dice loss ($20 \times \mathcal{L}{Focal} + \mathcal{L}{Dice}$). Focal loss prevents the rare pixels of small structures from being overwhelmed during training, while Dice loss ensures boundary adherence.
[Zero-Shot Generalization] Cross-Protocol Robustness: SAMRI achieved a 0.85 DSC in zero-shot evaluations on unseen datasets. A key finding was that training on knee cartilage transferred effectively to shoulder cartilage, proving the model exploits structural similarities rather than just anatomical shortcuts.
[Inference and Deployment] Technical Implementation:
Hardware requirements: Training is feasible on a single commercial GPU with minimum VRAM (~2.5 GB for training, 4.5 GB for inference).
Interface: Supports CLI (inference.py) and a Jupyter-based GUI for interactive segmentation using box, point, or mixed prompts.
[Limitations and Future Directions] Road to Clinical Integration: Current limitations include 2D-only processing and storage requirements for embeddings (2.2 TB). Future work aims to extend the model to 2.5D/3D context and native support for DICOM/NIfTI formats for prospective clinical validation.
Reviewing Group Recommendation:
This topic should be reviewed by a multidisciplinary team consisting of Biomedical Imaging Researchers, Senior Machine Learning Engineers (specializing in Transformers/Foundation Models), and Clinical Radiologists. This group can evaluate the technical feasibility of the decoder-only fine-tuning approach against the practical clinical need for high-accuracy small-lesion detection.