01 / Computer Vision · University of Michigan
Adapting a general segmentation model to brain tumor MRI with parameter-efficient LoRA fine-tuning and bidirectional slice propagation.
The model in action
Compare the original MRI, ground truth, and SAM2-Large predictions before and after fine-tuning.
Left to right: MRI slice along the z-axis, ground-truth tumor mask, prediction without fine-tuning, and prediction with fine-tuning.
Approach
BraTS NIfTI volumes are converted into 2D slices. FLAIR inputs are replicated across three channels, then treated as a pseudo-video for continuous segmentation.
Read multimodal BraTS MRI volumes and slice them along the z-axis.
Replicate FLAIR into three channels and save preprocessed .npz files.
Apply rank-16 adapters to selected attention and MLP layers.
Use forward and backward pseudo-video propagation from an initial labeled slice.
Findings & limitations
Lower-resolution training reduced training time. Fine-tuning remained necessary for acceptable segmentation quality; direct prediction without adaptation produced unsatisfactory results.
The model targets brain tumors and still needs one manually labeled slice as a prompt. Future work could explore higher-resolution inputs and a larger trainable parameter budget.
The project report is being finalized and will be added when ready.