I have a question on feature extraction from 2D CNN and classifying features with SVM. First let me introduce what I am trying to do;
1) I use pretrained network AlexNet which is trained with ImageNet.
2) I have a small dataset and use transfer learning for the classification problem. First, I trained my database with AlexNet by retraining all the parameters inside the network (no freezing layers) and observed the accuracy.
3) Now I want to classify the extracted features from the network with SVM. Should I use the initial AlexNet network's layer for feature extraction (default AlexNet) or the retrained network's layer on step 2?
I actually tried both of them and acquired higher accuracy on retrained network with almost %20 difference compared with the initial AlexNet. Is it just because the retrained parameters performed well on SVM? That's why a higher accuracy observed? Which one should be used?