Convert Ultrasound Image Series into Training Data for 2-D Semantic Segmentation Network

This example shows how to create training data for a 2-D semantic segmentation network using a groundTruthMedical object that contains multiframe ultrasound image series. To train a semantic segmentation network, you need pairs of data source images and label images stored in an imageDatastore and pixelLabelDatastore respectively. To train a 2-D network using data from a multiframe image series, you must convert the series into individual frames stored in separate files.

Create Ground Truth Object

You can export a groundTruthMedical object from the Medical Image Labeler app or create one programmatically.

This example creates a groundTruthMedicalobject by using the createGroundTruthMed2D helper function. The helper function is attached to this example as a supporting file. The groundTruthMedical object references a multiframe echocardiogram data source and its corresponding label image series. The data source and label image are stored as a single DICOM file and MAT file, respectively.

gTruthMed = createGroundTruthMed2D;

Extract Data from Ground Truth Object

Extract the data source and label image file names from the groundTruthMedical object.

dataSource = gTruthMed.DataSource.Source;
labelData = gTruthMed.LabelData;

Remove any data sources that are missing label images.

noLabelsIdx = labelData ~= "";
dataSource = dataSource(noLabelsIdx);
labelData = labelData(noLabelsIdx);

Extract the label definitions from the groundTruthMedical object. Add an addition label definition for the background region, corresponding to a pixel value of 0.

labelDefs = gTruthMed.LabelDefinitions;
labelDefs(2,:) = {"background",[0 1 0],0};

Convert Multiframe Series into 2-D Frames

The data source is a multiframe ultrasound image series stored in a single DICOM file. Convert the DICOM file into individual 2-D images, stored as MAT files, by using the convertImageSeriesToFrames supporting function. The supporting function is defined at the end of this example.

newDataSource = convertImageSeriesToFrames(dataSource);
newDataSource = string(newDataSource);

The label data is a multiframe image series stored in a single MAT file. Convert the single MAT file into individual MAT files for each frame by using the convertLabelSeriesToFrames supporting function. The supporting function is defined at the end of this example.

newLabelData = convertLabelSeriesToFrames(labelData);

Create Datastores

Load the individual MAT file data sources into a imageDatastore.

imds = imageDatastore(newDataSource,...
    ReadFcn=@readFramesLabels,...
    FileExtensions=[".mat",".dcm"]);

Load the individual MAT file label images into a pixelLabelDatastore (Computer Vision Toolbox). Use the label definitions from the groundTruthMedical object to map pixel values to categorical labels in the datastore.

pxds = pixelLabelDatastore(cellstr(newLabelData),labelDefs.Name,labelDefs.PixelLabelID,...
    ReadFcn=@readFramesLabels,...
    FileExtensions=".mat");

Preview one image and label. Display the labeled image by using the labeloverlay function.

im = preview(imds);
label = preview(pxds);
imOverlay = labeloverlay(im,label);
imshow(imOverlay)

Create a CombinedDatastore that pairs each data source image with its corresponding label image.

trainingData = combine(imds,pxds);

Supporting Functions

The convertImageSeriesToSlices function converts a multiframe DICOM image series, such as ultrasound data, into individual 2-D frames stored in MAT files. The function returns a cell array of the new MAT filenames.

function newDataSource = convertImageSeriesToFrames(labelDataSource)
% Create a data directory to store MAT files
dataFileDir = fullfile(pwd,"GroundTruthData");

if ~isfolder(dataFileDir)
    mkdir(dataFileDir)
end

image = medicalImage(labelDataSource);
data3d = image.Pixels;

% Assumption that time is the third dimension
numFrames =  size(data3d,3);
newDataSource = cell(numFrames,1);

for frame = 1:numFrames
    data = squeeze(data3d(:,:,frame,:));
    [~,name,~] = fileparts(labelDataSource);
    matFileName = strcat(fullfile(dataFileDir,name),"_",num2str(frame),".mat");
    save(matFileName,"data");
    newDataSource{frame} = string(matFileName);
end

end

The convertLabelSeriesToFrames function converts a multiframe image series stored in a single file into individual frames stored as separate MAT files. The function returns a cell array of the new MAT filenames.

function newlabelData = convertLabelSeriesToFrames(labelSource)
% Create a label directory to store MAT files
labelFileDir = fullfile(pwd,"GroundTruthLabel");

if ~isfolder(labelFileDir)
    mkdir(labelFileDir)
end

labelData = load(labelSource);

% Assumption that time is the third dimension
numFrames =  size(labelData.labels,3);
newlabelData = cell(numFrames,1);

for frame = 1:numFrames
    data = squeeze(labelData.labels(:,:,frame,:));
    [~,name,~] = fileparts(labelSource);
    matFileName = strcat(fullfile(labelFileDir,name),"_",num2str(frame),".mat");
    save(matFileName,"data");
    newlabelData{frame} = string(matFileName);
end

end

The readFramesLabels function reads data stored in a variable named data from a MAT file.

function data = readFramesLabels(filename)
d = load(filename);
data = d.data;
end