Get Started with MedSAM in Medical Image Labeler

Since R2025a

This example shows how to interactively segment objects in medical images and in cross-sections of medical volumes using the MedSAM algorithm in the Medical Image Labeler app.

Download Data to Label

This example labels chest CT data from a subset of the Medical Segmentation Decathlon data set [1]. The size of the subset of data is approximately 76 MB. Run this code to download the MedicalVolumNIfTIData.zip file from the MathWorks® website and unzip the file.

zipFile = matlab.internal.examples.downloadSupportFile("medical","MedicalVolumeNIfTIData.zip");
filepath = fileparts(zipFile);
unzip(zipFile,filepath)
dataFolder = fullfile(filepath,"MedicalVolumeNIfTIData");

Install MedSAM Support Package

Install the Medical Imaging Toolbox™ Model for Medical Segment Anything Model from the Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons. The support package also requires Deep Learning Toolbox™ and Computer Vision Toolbox™. Processing image data on a GPU requires a supported GPU device and Parallel Computing Toolbox™.

Open Labeling Session and Load Data

Open the Medical Image Labeler app and start a volume labeling session. You can create a new volume session or reopen a previous session. If you create a new session, import the file lung_027.nii.gz from the downloaded data, located in the folder specified by the dataFolder variable. For an example that shows how to create a volume session and import data, see Label 3-D Medical Image Using Medical Image Labeler.

Create Label Definitions

In the Label Definitions pane, select Create Label Definition to create a label with the default label name. To change the name of the label, double-click on the label and type in a new name. The label name must be a valid MATLAB® variable name with no spaces. To change the default color associated with the label, double-click on the colored square in the label identifier and select a color from the Color dialog box. For this example, create two labels named Type1 and Type2.

Label Objects Using MedSAM

Medical Segment Anything Model (MedSAM) is a deep learning algorithm for the easy, interactive segmentation of 2-D medical images and 2-D cross-sections of 3-D medial volumes. You can use MedSAM to segment objects in several medical imaging modalities without needing to retrain the model for your data set. MedSAM builds upon the foundation of the Segment Anything Model (SAM). You can segment objects in 2-D medical images and 2-D cross-sections of 3-D medial volumes using the medicalSegmentAnythingModel object. For more information about MedSAM, see Get Started with Medical Segment Anything Model for Medical Image Segmentation.

On the app toolstrip, in the Draw tab, select MedSAM. You can use a GPU, if available, for faster segmentation by selecting Use GPU for segmentation if available. This option is selected by default if a GPU is available, but you can choose to clear it. If a GPU is not available, the option is inactive.

Label Objects in 2-D Slice of 3-D Volume

Select label Type1. Draw a bounding box around the object to segment in the transverse plane of the volume.

If you are segmenting objects in a medical image or a 2-D slice image of a medical volume for the first time in the app, and if you are not using a GPU, the MedSAM algorithm takes some time to extract embeddings for the image. In this case, the app displays the Running MedSAM model dialog box.

Once the MedSAM algorithm extracts the embeddings of the image, you can adjust the bounding box.

The MedSAM algorithm segments the object in the bounding box and labels it.

Label Multiple Objects in Image

Select label Type1. Draw a bounding box around the object to segment in the sagittal plane of the volume. The MedSAM algorithm extracts the embeddings of the image and segments the object in the bounding box. Then, select label Type2. Draw a bounding box around the object to segment in the sagittal plane of the volume. Because this is the second object segmented in the same image during the same labeling session, the MedSAM algorithm does not extracts the embeddings of the image again. The algorithm directly segments the object in the bounding box.

Edit Labels Created Using MedSAM

Select label Type2. Draw a bounding box around the object to segment in the coronal plane of the volume. The MedSAM algorithm extracts the embeddings of the image and segments the object in the bounding box. Right-click the labeled object, and, from the context menu, select Select Drawn Region. You can now edit the label shape using the points, or reassign or delete the label by using the options in the context menu.

Visualize Labels in 3-D

Visualize the labels in different cross-sections of the volume in the 3-D volume rendering pane.

References

[1] Medical Segmentation Decathlon. "Lung." Tasks. Accessed May 10, 2018. http://medicaldecathlon.com/.

The Lung data set is provided by the Medical Segmentation Decathlon under the CC-BY-SA 4.0 license. All warranties and representations are disclaimed. See the license for details. This example uses a subset of the original data set consisting of two CT volumes. The labels shown in this example were created for illustration purposes and have not been verified by a clinician.