How to execute the same operation on various .csv files?
5 views (last 30 days)
Show older comments
Simone Speltoni
on 16 May 2021
Commented: Simone Speltoni
on 16 May 2021
I have a list (62 items) of .csv files, all within a single folder. My goal would be to make the same operation on all of these files; I managed to write the code on a single .csv file, which is as follows (actually it is a trapz operation but written extensively):
%my code on a single .csv file
T = readtable('M_20210325-1519_90_l90_.csv','HeaderLines', 11); %read single .csf file and skip 11 lines
[numRows,numCols] = size(T);
cont=0;
tB= T(:,3); %consider 3rd column
B=table2array(tB) %consider it as an array
bvB=B(1,1); %consider the first value
i=1;
intB=zeros(numRows,1); % create an array full of zeros
newB=B(:,1)-bvB; %subtract the first value
z=numRows-1;
for cont=1:z
if (newB(i+1,1)+newB(i,1))>=0
intB(i,1)= (newB(i+1,1)+newB(i,1))/2;
else
intB(i,1)=0;
end
dbstop if error;
i=i+1;
end
sommaB=sum(intB); %useful result
I had been looking for some scripts that allow to consider and read all the data from the .csv files, such as
files = dir('*.csv')
N = length(files) ;
% loop for each file
fileNames = {files.name};
for k = 1:N
data{k} = csvread(fileNames{k});
%%do whatever you want
end
(which gives the Trouble reading 'Numeric' field from file error, because I guess it does not skip the header lines), or
fds = fileDatastore('*.csv', 'ReadFcn', @importdata)
fullFileNames = fds.Files
numFiles = length(fullFileNames)
% Loop over all files reading them in and plotting them.
for k = 1 : numFiles
fprintf('Now reading file %s\n', fullFileNames{k});
% do the operation --------> my single operation
end
Both are useful because I get all the .csv files, already sorted in a proper way, but I'm missing the link between considering them one at a time and execute the operation. Probably it's relatively easy, but I'm a beginner.
I attach two .csv files as an example.
Thank you
0 Comments
Accepted Answer
David Fletcher
on 16 May 2021
Edited: David Fletcher
on 16 May 2021
You would need to incorporate your code into one of the examples, also you would need to store the intended output as the loop progresses thorugh your examples. As a guide: (you will of course have to change the path to where you have the .csv files stored on your system)
path='H:\MATLAB\R2021a\bin\TestDir\' %directory of the csv files on your system
files = dir(strcat(path,'*.csv'))
N = length(files) ;
% loop for each file
fileNames = {files.name};
for k = 1:N
%my code on a single .csv file
T = readtable(strcat(path,fileNames{k}),'HeaderLines', 11); %read single .csf file and skip 11 lines
[numRows,numCols] = size(T);
cont=0;
tB= T(:,3); %consider 3rd column
B=table2array(tB) %consider it as an array
bvB=B(1,1); %consider the first value
i=1;
intB=zeros(numRows,1); % create an array full of zeros
newB=B(:,1)-bvB; %subtract the first value
z=numRows-1;
for cont=1:z
if (newB(i+1,1)+newB(i,1))>=0
intB(i,1)= (newB(i+1,1)+newB(i,1))/2;
else
intB(i,1)=0;
end
dbstop if error;
i=i+1;
end
%useful result
sommaB(k)=sum(intB); %Save output in a vector for each loop iteration
end
3 Comments
David Fletcher
on 16 May 2021
Yes, the strcat just adds the file name to the path to make a valid file location - I had to do this on my system because I put your files in an separate test directory (you might get away without it if everything on your system is in the same diectory). If the files struct is empty it indicates that there is a problem in locating the files. It works on my system, but obviously I have no idea how your system is setup or where you are storing your files. If you've got everything in the same directory you could get rid of the path entirely and revert to your original usage:
files = dir('*.csv')
N = length(files) ;
% loop for each file
fileNames = {files.name};
for k = 1:N
T = readtable(fileNames{k}),'HeaderLines', 11);
% ... and so on
More Answers (0)
See Also
Categories
Find more on Large Files and Big Data in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!