How to improve the training accuracy in example of Train Convolutional Neural Network for Regression?

The example of 'Train Convolutional Neural Network for Regression' shows how to predict the angles of rotation of handwritten digits using convolutional neural networks. However, the accuracy of the CNN network is not good enought. Any ideas to improve the network accuracy, like adjusting learnable parameters or net structures?

Thanks.

Answer 1

[Shounak Mitra]

Great question!

You're right, sometimes the routine augmentations do not add additional value. In such cases, often I rely upon an optimizer to find optimal hyperparameters like learning rate, mini-batch size,momentum etc. See the documentation https://www.mathworks.com/help/deeplearning/examples/deep-learning-using-bayesian-optimization.html.

With regards to your question on finding the best net structures, it is a area of research and often words like AutoML are used for such workflows. What I mentioned earlier, regarding bayesian optimization, is also a part of AutoML. The downside of trying to use an automated technique to find the best network structure is that it is computationally very very expensive. Having said that, we've an example in the documentation that shows how you can do a parameter sweep on the depth of a network --> https://www.mathworks.com/help/deeplearning/examples/use-parfeval-to-train-multiple-deep-learning-networks.html

Answer 2

[Saira]

Hi,

I have 5600 training images. I have extracted features using Principal Component Analysis (PCA). Then I am applying CNN on extracted features. My training accuracy is 30%. How to increase training accuracy?

Feature column vector size: 640*1

My training code:

% Convolutional neural network architecture

layers = [

imageInputLayer([1 640 1]);

reluLayer

fullyConnectedLayer(7);

softmaxLayer();

classificationLayer()];

options = trainingOptions('sgdm', 'Momentum',0.95, 'InitialLearnRate',0.0001, 'L2Regularization', 1e-4, 'MaxEpochs',5000, 'MiniBatchSize',8192, 'Verbose', true);

Answer 3

[o.cefet cefet]

You could try adding dropout layer, regularization, strengthening your database ...

It's fine with your regularization code, but now you have to change the value of these regularizations, and look for "the best value".