large file slow running
4 views (last 30 days)
Show older comments
latmin=-90; %chooce your location
latmax=90;
longmin=-180; a=1;
longmax=180;
data=zeros(90,90);
for b=1:length(x(:,1))
if (x(b,8)>=longmin && x(b,8)<longmax && x(b,7)>=latmin && x(b,7)<latmax);
for col=1:13
data(a,col)=x(b,col);
end
a=a+1;
end
end
theres a file called x.mat (601760x13). What might be the reason it runs too slow
1 Comment
KSSV
on 20 Sep 2021
What you are trying to achieve? It can be achived without loop. You need to give an explanation of your problem. What does x.mat have?
Answers (1)
Jan
on 20 Sep 2021
I guess that the iterative growing of the array data slows down the processing.
Remember that the growing of an array required to create a new array and to copy the foermer contents.
x = [];
for k = 1:1e6
x(k) = rand;
end
This creates a new array in each iteration. Although the final array needs 8MB only (8 bytes per double), Matlab has to allocate and to copy sum(1:1e6)*8 bytes, which are more than 4 TB ! The solution is easy: Allocate with the final size:
x = zeros(1, 1e6); % Pre-allocation
for k = 1:1e6
x(k) = rand;
end
In your case you do not know the needed size in advance. You tried to pre-allocate with data=zeros(90,90), but this is not the final size. You could determin the final size:
latmin = -90;
latmax = 90;
longmin = -180;
longmax = 180;
index = (x(:,8) >= longmin & x(:, 8) < longmax & x(:, 7) >= latmin && x(:,7) < latmax);
data = zeros(sum(index), 13);
a = 1;
for b = 1:size(x, 1) % nicer and faster than: length(x(:,1))
if index(b)
for col = 1:13
data(a, col) = x(b, col);
end
a = a + 1;
end
end
But it is easier, nicer and faster to perform the copy directly without a loop:
latmin = -90;
latmax = 90;
longmin = -180;
longmax = 180;
index = (x(:,8) >= longmin & x(:, 8) < longmax & x(:, 7) >= latmin && x(:,7) < latmax);
data = x(index, 1:13);
0 Comments
See Also
Categories
Find more on Dates and Time in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!