How to accumulate values ​​from time 1 to the last time

I have more than 10000 files and I want to accumulate the values of given variable between all these times and then plot it on a single image. How can I do it? The files are 1000x1000 matrices.

[the cyclist]

Let's just suppose, to keep things easy for understanding, that you only have two files, and each one has a 3x3 matrix. Suppose those two matrices are

M1 = magic(3)

M1 = 3×3

8 1 6 3 5 7 4 9 2

M2 = magic(3) + 17

M2 = 3×3

25 18 23 20 22 24 21 26 19

What exactly should the output be?

Also, what is the file type? Excel, CSV, MATLAB mat file?

[pink flower]

This must be my output (M3):

M1 = magic(3);
M2 = magic(3)+17;
M3 = M1 + M2;

Answer 1

[Sulaymon Eshkabilov]

Here is one of the possible routines to read all 1e4 files and collect all data into a signle variable and the rest is relatively simple:

clearvars
FILE = fullfile('C:\Users\...', '*.txt');  % Directory where the files are residing. Your data file extension, e.g.: .txt, .xlsx, .csv, etc 
LIST = dir(FILE);
N = length(LIST);                          % N = 1e4 as you stated that 1e4 files 
D = zeros(1e3, 1e3, N);
for ii = 1 : N   
    FFName = fullfile(LIST(ii).folder, LIST(ii).name);
    DATA = readmatrix(FFName);                
    D(:, :, ii) = DATA;                    % NOW D contains all data from 10000 files
end
...  % Select and process the part of D that is necessary

[pink flower]

I tried these commands, but I was unsuccessful. This was the mistake:

Error using zeros
Requested 1000x1000x48282 (359.7GB) array exceeds maximum
array size preference. Creation of arrays greater than
this limit may take a long time and cause MATLAB to
become unresponsive. See array size limit or preference
panel for more information.

Error in test (line 10)
D = zeros (1e3, 1e3, N);

[Sulaymon Eshkabilov]

These are not coding problems but rather memory problems. That means either you should process your data chunk by chunk or increase the memory in your matlab settings.

Answer 2

[Stephen23]

P = 'absolute or relative path to where the files are saved';
S = dir(fullfile(P, '*.txt'));  % Select the file extension to suit your data files
M = 0;
for k = 1:numel(S) 
    F = fullfile(P, S(k).name);
    M = M + readmatrix(F);                
end

[pink flower]

I got this error:

Undefined function or variable 'readmatrix'.

Error in untitled2 (line 12)
M = M + readmatrix(F);

With this:

P = 'C:\Users\...'
FILE = dir(fullfile(P,'*.bin'));
M = 0;

for k = 1:numel(FILE)
    F = fullfile(P, FILE(k).name);
    M = M + readmatrix(F);
end

My file is binary and I have to open it like this:

fileID = fopen('C:\Users\...\test.bin');
var = fread(fileID,[1000 1000], 'float32');

Is it possible to change this code and adapt it?

[Stephen23]

"Undefined function or variable 'readmatrix'."

In future please select your MATLAB release when asking a question.

" Is it possible to change this code and adapt it?"

Of course, just replace READMATRIX with whatever code you use to import your matrices with.

[pink flower]

Thank you! I managed to do this! But the points that have NaN are not entering the sum. Do you have any suggestions to solve?

[Stephen23]

"But the points that have NaN are not entering the sum"

It is not clear to me what that means. Please give an example of what occurs, and what you want to occur.

[pink flower]

For example, in row 400 and column 5, I have:

M1 = NaN;

M2 = NaN;

M3 = 0.9

And the result it returns to me is NaN. Do you understand now? If not, I can explain it better.

[Stephen23]

To ignore NaN when summing you can use SUM with the appropriate flag, e.g.:

M = zeros(1000,1000);
for k = 1:numel(S)
    F = fullfile(P, S(k).name);
    N = whateverCodeYouUseToImportYourFileData;
    M = sum(cat(3,M,N),3,'omitnan');
end