How to use tic/toc function efficiently if the code has many loops and every loops takes half an hour?

Hi, I am using tic/toc function to meausre my custom CNN architecture classification time. I try to find fastest(prediction time, in this case in matlab classify function) custom model. To do this i have 7 for loops... In loops i change the input data size, number of filters in first and second conv layer, filter sizes in first and second conv layer, strides and so on.... I used the tic toc function in the loops as shown in below.

for

for

.

.

layers = ....

net = trainNetwork(augimdsTrain,layers,options);

tic ;

[YPred,probs] = classify(net,augtestDS);

toc

time(a)= toc

.

.

end

end

There are 432 possibilities depending on for loops. First 3-4 loops give good results(i mean tic/toc function). But after some hours tic/toc function doesnt give good results. The time is increasing always... i expect max time must be around 3seconds but after 2 days of running code tic/toc time rised to 8 seconds. I stoped the code using ctrl+c and i runed the code again(code starts with clc clear all close all) but the tic/toc time was again around 8 seconds.. then i closed and opend the matlab. ı runed the code just for 1 loop and it gives expected small times around 2 seconds. What should be the problem when i use tic/toc function for type of huge loops and huge running time of codes? How to make tic/toc function working for big loops? I think matlab using more memory depending on runing time. Maybe there is an easy solution.

Thank you for advising me on this topic.

Hakan

[Hari Krishna Ravuri]

I’m unable to reproduce a similar behavior on my side. Please provide reproduction steps.

[Walter Roberson]

I suspect that you are not preallocating your variables and MATLAB is getting slower as it grows the variables.

[Hakan]

Hi Hari and Walter, thanks for your replies. Inside for loops there is a code :

 miniBatchSize = 64;
valFrequency = floor(numel(augimdsTrain.Files)/miniBatchSize);
options = trainingOptions('sgdm', ...
'MiniBatchSize',miniBatchSize, ...
'MaxEpochs',20, ...
'InitialLearnRate',3e-4, ...
 'Shuffle','every-epoch', ...
 'ValidationData',augimdsValidation, ...
'ValidationFrequency',valFrequency, ...
'Verbose',ture); 

I just changed the 'Verbose',ture to 'Verbose',false. Now it works better than before. The code will run 3 days. Attachment you can find the matlab code. You can use any data set and try this code for 6 classes. Your feedbacks are important for me. Maybe i can change something more and the code works better.

I also wonder if tic toc gives me always exact time or not. Is there any other way to measure execution time?

Kind Regards

[Walter Roberson]

accuracy(a)= mean(YPred == testDS.Labels)

a=a+1;

You are not pre-allocating the accuracy array, so every iteration of your seven-deep loops you would be growing accuracy. That will get slower and slower as you go.

You are also not pre-allocating

for v = 1 : numel(varSize)
    for num_of_filt_conv1_j = 1: numel(num_of_filt_conv1)
        for size_of_filt_conv1_j = 1: numel(size_of_filt_conv1)
            for stride1_j = 1: numel(stride1)
                for num_of_filt_conv2_j = 1:numel(num_of_filt_conv2)
                    for size_of_filt_conv2_j = 1: numel(size_of_filt_conv2)
                        for stride2_j = 1:numel(stride2) 

The upper bounds appear to be known ahead of time before the for v loop starts, so you can multiply them all out to find the size of accuracy that you will need.

You have the same issue with time(a)

Also, every iteration you are displaying all of time and all of accuracy, not just the latest value. You should be putting a semi-colon on the end of the assignment statements.

[Hakan]

Dear Walter ,

Thanks for your reply. Yes you are right i have problem with pre-allocating. Now code works better. But still i have some problems with code. Attachment you can find the final version of the code. Withouth pre-allocating time was rising to 7 seconds now it is rising upto 3 seconds. And still cant find the minimum time because of always rising time. What can be the reason? Every iteration i create a new layer and net. Do i have to make pre-allocate for layers strucre also however i am not sure of tihs?

Thanks for helps

Hakan