MATLAB Answers

How can I concatenate multiple CSV file and turn into one CSV file

49 views (last 30 days)
I have 1000 csv.files with table format that I want to merge every 6 files together and turn them into another csv file of matrix.
I wrote this code It does not work. I will get error in the line 6"out = csvread(myFiles(m),1,0);"
Can you help me to make the idea to run?
cd 'path' % folther where the files are
myFiles = dir ('*.csv'); % all csv files
folder = cd;
for m=1: numel(myFiles)
out = csvread(myFiles(m),1,0); % read the first file
for n=m+1 : m+5 % loop over each 6 files
new = csvread(myFiles(n)},1,0); % Read the nth file
out = horzcat(out, new); % Concatenate the first file of the loop with others in rows
m=n
end
file_name = sprintf('f%d.csv',m);
csvwrite(file_name , out);
end
Thanks in advance!
  5 Comments
Zahra Soltani
Zahra Soltani on 28 Jan 2021
The problem is that I first, need to get rid of the table type of the csv files to e.g. matrix because it does not allow me to concatenate data and second I need a proper command that read the files in the for loop.

Sign in to comment.

Accepted Answer

per isakson
per isakson on 28 Jan 2021
Edited: per isakson on 28 Jan 2021
A alternate idea (that wrongly(?) concatenates the the files vertically)
Pro
  • avoids converting to numerical and back to text (good for performance)
  • avoids a variable that increases in size in the for-loop (good for performance)
Con
  • requires that the data files are terminated with ONE newline character
%% Create twenty sample files
for jj = 1 : 20
csvwrite( sprintf( 'zahra_%02d.csv', jj ), randi( [10,99], 6, 4 ) );
end
%%
sad = dir( fullfile( 'd:\m\cssm\', 'zahra*.csv' ) );
len = numel( sad );
pst = rem( len, 6 );
%%
myFiles = reshape( sad(1:end-pst), 6,[] );
count = 0;
for sixpack = myFiles
chr = [ fileread( fullfile( sixpack(1).folder, sixpack(1).name ) ) ...
, fileread( fullfile( sixpack(2).folder, sixpack(2).name ) ) ...
, fileread( fullfile( sixpack(3).folder, sixpack(3).name ) ) ...
, fileread( fullfile( sixpack(4).folder, sixpack(4).name ) ) ...
, fileread( fullfile( sixpack(5).folder, sixpack(5).name ) ) ...
, fileread( fullfile( sixpack(6).folder, sixpack(6).name ) ) ];
count = count + 1;
fid = fopen( fullfile( 'd:\m\cssm\', sprintf('f_%02d.csv',count) ), 'w' );
fprintf( fid, '%s', chr );
fclose( fid );
end
%%
fprintf( 2, 'The %d last file(s) didn''t fit into the six by six scheme\n', pst )
  4 Comments
Zahra Soltani
Zahra Soltani on 1 Feb 2021
Just the line below should be added to the code above after defining "sad" to sort the files based on their names. As I have both text and number in the file names, I have to download and use function "natsort" or "natsortfiles" in order to sort the filenames.
sad = natsort({sad.name})

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!