[YOLOv2] Train the feature layer instead of transfer learning

As explained in the example in https://de.mathworks.com/help/vision/ug/train-an-object-detector-using-you-only-look-once.html, most YOLO network in Matlab context is trained via transfer learning. The feature layer is pretrained, and using trainYOLOv2ObjectDetector I can only adapt the coefficients in detector layers. When I change the

[detector, info] = trainYOLOv2ObjectDetector(augmentedTrainingData,lgraph, opts);

with detector = trainNetwork(dsTrain,lgraph,opts);

I have the following error (info. I have increased the labels to 4 classes)

Error using trainNetwork

Invalid training data. The output size ([23 40 63]) of the last layer does not match the response size ([1 1 4]).

How can I realize training of the complete YOLOv2 network, not just part of it?

Thank you.

[Venu]

Hi @Tai,

If you want to use "trainNetwork" instead of "trainYOLOv2ObjectDetector", you would need to preprocess your data to match the expected response format of the YOLOv2 network, which involves encoding the bounding box and class information into the YOLOv2-specific grid format.

The expected response size is [1 1 4]. This is a mismatch because for YOLOv2, the response size should not just be the number of classes. Instead, it should be a multi-dimensional array based on the grid size and the number of predictions per grid cell, which includes bounding box coordinates, object confidence, and class probabilities.

Can you provide your network's output layer structure, some info about how your training data is organized, including how the bounding boxes and class labels are formatted?

If you're using an "augmentedImageDataStore", can you provide details on how it was created and what preprocessing steps were applied?

Thanks

[Tai]

Hi @Venu:

Thank you very much for your reply. It seems that "trainYOLOv2ObjectDetector" will implicitly incorporate bounding box and anchor information into the training processing. If I want to use "tainNetwork" for training, I have to manually add these information into the "dsTrain", correct?

Actually, my intention is to train the complete YOLOv2 network instead of the layers only reponsible for detector. Is it possible to use "trainYOLOv2ObjectDetector" to realize my intention?

Besides, here are the answers to your question, in case it is impossible to do it by using "trainYOLOv2ObjectDetector":

Can you provide your network's output layer structure, some info about how your training data is organized, including how the bounding boxes and class labels are formatted?

• I use Darknet19 as backbone network, and the feature network is till the layer "leaky_relu_13"
• 
• I have put the training data as mat-file in the attachment.

If you're using an "augmentedImageDataStore", can you provide details on how it was created and what preprocessing steps were applied?

• I have resize the datainto [720 720 3]
• Apply random horizontal flipping, and random X/Y scaling. Boxes that get scaled outside the bounds are clipped if the overlap is above 0.25. Also, jitter image color.

[Venu]

Hi @Tai,

Firstly I'll answer your queries.

"I have to manually add these information into the "dsTrain", correct? "YES.

"Actually, my intention is to train the complete YOLOv2 network instead of the layers only reponsible for detector. Is it possible to use "trainYOLOv2ObjectDetector" to realize my intention? "Unfortunately NO.The "trainYOLOv2ObjectDetector" function is designed for transfer learning and is not intended for training the entire network from scratch. It assumes that the backbone network (in your case, Darknet19 up to "leaky_relu_13") is pre-trained and its weights are frozen during training. This means that only the layers added beyond the backbone (i.e., the detection layers) are trained to adapt to your specific object detection task. If you want to train the complete YOLOv2 network, you need to either train from scratch or fine-tune the entire network, including the feature extraction layers (Darknet19). So you cannot use "trainYOLOv2ObjectDetector" and must proceed with "trainNetwork".

Coming to the data and info that you have provided I would suggest these steps:

1. Apply the same augmentations to both the images and the bounding boxes and add to "trainNetwork" manually. Some entries are empty [ ], which suggests that no object was detected or annotated in that particular image. You'll need to decide how to handle these cases during training - whether to include them as negative examples or to exclude them from the training process.

2. Preprocess the bounding box data into the format expected by YOLO. This involves assigning each bounding box to the appropriate cell in the 45x45 grid based on its coordinates, encoding the bounding box coordinates relative to the cell it's assigned to, encoding the objectness score and class probabilities.

Hope this helps!