How can I import multiple .CSV files in MATLAB and SISTEMATICALLY process the data of each file?
43 views (last 30 days)
Hi all. I have a couple of csv files I need to analyse (1488 files, each having 3 columns and 2304 rows containing only numerical values). I would like my script to open these csv files one by one and extrapolate the informations I need. For example, the script opens the first file, does its calculations, stores these 2 relevant values (that are unique to that file) and then moves on to the next file repeating the same calculations. The script I have right now does this but just for one file. How do I change the script so that it proceeds file by file and stores the obtained values?
1:All the files are contained in the same foulder
2: The first row of the file is Text, this is why i use :
3: The relevant values I want the script to save before it moves to the next file are: Hav and Tz (at the end of the script).
4: What I'm struggling with here is to give Matlab the right order to do things: ie: open file-->do the calculations--->store the values----> open new file-->repeat the same calculations etc.
eta = vector(:,1)/100; % heave disp in metres
north = vector(:,2)/100; % northing in metres
west = vector(:,3)/100; % westing in metres
%------ TIME-SERIES ANALYSIS ----------------------------------------------------
% de-mean the time series
eta = eta - mean(eta);
sr=length(eta)/(30*60); %sampling frequency estimate based on a full 30minute record
for i=1:length(eta) % time vector for data
% find the zero up-crossings
[ind,t0]=crossing(eta,t); % this finds all crossings (up and down)
n_up = 0;
for i = 1:length(ind)
if eta(ind(i)+1) - eta(ind(i)) > 0
n_up = n_up+1;
ind_up(n_up) = ind(i);
t0_up(n_up) = t0(i);
fprintf(1,'no. of crossings = %g\n',length(ind))
fprintf(1,'no. of up_crossings = %g\n',n_up)
nwaves = n_up-1;
for i = 1:nwaves
wv = eta(ind_up(i):ind_up(i+1));
crest(i)=max(wv); % determine crest of that wave
trough(i)=abs(min(wv)); % determine trough of that wave
ht(i)=crest(i)+trough(i); % determine height of that wave
dpb on 22 Jan 2021
See <Importing-all-files-from-a-specific-folder?> from yesterday. There's a section in the doc with examples on processing multiple files as well.
In addition, at a higher level there's a function FILEFUNCTION on the File Exchange that lets you simply use your function as an argument and will call it with all files in the argument list that can let you avoid actually writing the looping construct yourself.
I forget the author but I've used it successfully several times to save just such effort.
Jeremy Hughes on 25 Jan 2021
You could also look at tabularTextDatastore with transform, and writeall functions. This is the basic way that's done, but you may need to tweak the inputs for these depending on what you need to do
ttds = tabularTextDatastore(fileOrFolderName)
tds = transform(ttds,@myProcessFunction)
myProcessFunction takes the output of tabularTextDatastore's read and outputs the data you want to write.