8 views (last 30 days)

I encountered a strange problem in simulation code today. The problem is that having an initialized, large cell array in memory during a calculation, slows down the calculation, even though the cell array is not used anywhere after its initialization! If I initialize the same variable to approximately the same size, but then in a multi-dimensional array, there is no problem. The array is initialized as follows:

%--------------------------------------------------------------------------

% Cell array initialization (note that SRR is not used anywhere!)

%--------------------------------------------------------------------------

SRR = cell(NKstd, NLstd, NManufacture);

for i = 1:NKstd

for j = 1:NLstd

for k = 1:NManufacture

SRR{i, j, k} = complex(zeros(2, 1, length(y)), ...

zeros(2, 1, length(y)));

end

end

end

%--------------------------------------------------------------------------

% Multi array initialization (note that SRR is not used anywhere!)

%--------------------------------------------------------------------------

SRR = complex(zeros(2, 1, length(y), NKstd, NLstd, NManufacture), ...

zeros(2, 1, length(y), NKstd, NLstd, NManufacture));

% Display size

whos SRR;

length(y) = 2001, NKstd = NLstd = 21, NManufacture = 100, leading to a size of approximately 3 GB after initialization. It is important to again stress that SRR is not used anywhere. I can comment this entire piece of code away and my simulation still runs.

If I comment out the multi array part, I get (with runtime of my simulation, for which timing starts well after the initialization):

Name Size Bytes Class Attributes

SRR 21x21x100 2828750400 cell

Running: 21 21

Done in 136.3 seconds.

If I comment out the cell part, I get (with runtime of my simulation):

Name Size Bytes Class Attributes

SRR 6-D 2823811200 double complex

Running: 21 21

Done in 57.5 seconds.

If I comment out the entire code above (showing SRR is not used anywhere), I get results very similar to multidimensional initialization:

Running: 21 21

Done in 58.6 seconds.

So, how does an unused cell array cause such a large difference in calculation time? Eventually, I would like to used this array to store some calculation details.

The problem may be related to https://nl.mathworks.com/matlabcentral/answers/331930-can-anyone-explain-why-matlab-gets-slower-and-slower-until-restart-if-large-cell-or-struct-arrays-ar , which is still unsolved.

EDIT:

As discussed in some of the comments to answers below, the timing I'm doing specifically excludes the allocation, so this is not the issue. Both types of allocation only take a few seconds at most anyways.

Guillaume
on 27 Nov 2019

Well, with the cell array you're doing 2x44100 matrix allocations with zeros, which all have to be copied into a new complex matrix, so that's 88200 allocations and 88200 copies + 44100 allocation of complex matrices, whereas with the matrix it's just 3 allocation and 2 copies (I assume all this, details of memory management are not public). There's quite a lot of bookkeeping involve with allocating matrices so that could explain the slowdown (although i wouldn't expect it to be that big).

Note that another way of creating that cell array would be with:

SRR = squeeze(num2cell(complex(zeros(2, 1, length(y), NKstd, NLstd, NManufacture), ...

zeros(2, 1, length(y), NKstd, NLstd, NManufacture)), [1 2 3]));

See if it's any faster.

Of course, since all the matrices are identical you could just do:

SRR = repmat({complex(zeros(2, 1, length(y)), zeros(2, 1, length(y)))}, NKstd, NLstd, NManufacture);

This will definitively be faster to create but on the other hand, you'll have a slowdown when first assigning to each matrix when matlab need to split the shared copy between the cells.

Guillaume
on 27 Nov 2019

Sign in to comment.

James Tursa
on 27 Nov 2019

Edited: James Tursa
on 27 Nov 2019

Can you clarify if the above code is included in your timings? For all we know, you are simply showing that creating 44100 separate MATLAB variables takes longer than creating 3 MATLAB variables.

Also, it is rarely the case that you need to pre-allocate cell contents, since these are typically overwritten downstream in your code. It is only if you are modifying the contents by element that it might make sense. And if only some of the contents will be changing, this might be better:

SRR = cell(NKstd, NLstd, NManufacture);

SRR(:) = {zeros(2, 1, length(y),'like',1i)}; % reference copies fast to create

Finally, your 6D array example is better allocated as:

SRR = zeros(2, 1, length(y), NKstd, NLstd, NManufacture,'like',1i);

This avoids those temporary real & imaginary parts that you are currently using.

Sign in to answer this question.

Opportunities for recent engineering grads.

Apply Today
## 2 Comments

## Direct link to this comment

https://se.mathworks.com/matlabcentral/answers/493512-large-unused-cell-aray-in-memory-still-slows-down-calculation-significantly#comment_772633

⋮## Direct link to this comment

https://se.mathworks.com/matlabcentral/answers/493512-large-unused-cell-aray-in-memory-still-slows-down-calculation-significantly#comment_772633

## Direct link to this comment

https://se.mathworks.com/matlabcentral/answers/493512-large-unused-cell-aray-in-memory-still-slows-down-calculation-significantly#comment_772705

⋮## Direct link to this comment

https://se.mathworks.com/matlabcentral/answers/493512-large-unused-cell-aray-in-memory-still-slows-down-calculation-significantly#comment_772705

Sign in to comment.