How can I concatenate multiple CSV file and turn into one CSV file

I have 1000 csv.files with table format that I want to merge every 6 files together and turn them into another csv file of matrix.

I wrote this code It does not work. I will get error in the line 6"out = csvread(myFiles(m),1,0);"

Can you help me to make the idea to run?

cd 'path' % folther where the files are
myFiles = dir ('*.csv'); % all csv files
folder = cd;

for m=1: numel(myFiles)
    out = csvread(myFiles(m),1,0); % read the first file
    
    for n=m+1 : m+5  % loop over each 6 files
        new = csvread(myFiles(n)},1,0);  % Read the nth file
        out = horzcat(out, new);         % Concatenate the first file of the loop with others in rows
    m=n
    end
file_name = sprintf('f%d.csv',m);
csvwrite(file_name , out);
end

Thanks in advance!

[dpb]

What error?

[dpb]

 out = horzcat(out, new);

The above will fail unless every file has identical number of rows...

[Stephen Cobeldick]

As an aside, you should get rid of cd and use absolute/relative filenames (e.g. using fullfile).

[Zahra Soltani]

This is the error I got. File name must be a character vector or string scalar.

The files shape are the same

[Zahra Soltani]

The problem is that I first, need to get rid of the table type of the csv files to e.g. matrix because it does not allow me to concatenate data and second I need a proper command that read the files in the for loop.

Answer 1

[per isakson]

A alternate idea (that wrongly(?) concatenates the the files vertically)

Pro

• avoids converting to numerical and back to text (good for performance)
• avoids a variable that increases in size in the for-loop (good for performance)

Con

• requires that the data files are terminated with ONE newline character

%% Create twenty sample files
for jj = 1 : 20
    csvwrite( sprintf( 'zahra_%02d.csv', jj ), randi( [10,99], 6, 4 ) );
end
%% 
sad = dir( fullfile( 'd:\m\cssm\', 'zahra*.csv' ) );    
len = numel( sad );
pst = rem( len, 6 );
%%
myFiles = reshape( sad(1:end-pst), 6,[] );

count = 0;
for sixpack = myFiles
    chr = [ fileread( fullfile( sixpack(1).folder, sixpack(1).name ) )  ...
        ,   fileread( fullfile( sixpack(2).folder, sixpack(2).name ) )  ...
        ,   fileread( fullfile( sixpack(3).folder, sixpack(3).name ) )  ...
        ,   fileread( fullfile( sixpack(4).folder, sixpack(4).name ) )  ...
        ,   fileread( fullfile( sixpack(5).folder, sixpack(5).name ) )  ...
        ,   fileread( fullfile( sixpack(6).folder, sixpack(6).name ) )  ];
    
    count = count + 1;
    fid = fopen( fullfile( 'd:\m\cssm\', sprintf('f_%02d.csv',count) ), 'w' );
    fprintf( fid, '%s', chr );
    fclose( fid );
end
%%
fprintf( 2, 'The %d last file(s) didn''t fit into the six by six scheme\n', pst )

[Zahra Soltani]

My files have each column name as the first row. It is kind of table.

when I run this code with my own files directory and name , 'sad' is a structure with info is written bellow:

val = 
    name
    folder
    date
    bytes
    isdir
    datenum

So I don't get any response from that.

[per isakson]

"My files have each column name as the first row" Thus my answer will not work. Column header rows will appear six times in the output file.

The doc says: M = csvread(filename) reads a comma-separated value (CSV) formatted file into array M. The file must contain only numeric values. Thus, you cannot read these files with csvread()

"'sad' is a structure with info is written bellow:" Yes, that's what the function, dir, returns. 'sad' stands for Structure Array of Dir.

>> myFiles = dir ('*.csv')
myFiles = 
  36×1 struct array with fields:
    name
    folder
    date
    bytes
    isdir
    datenum
>> 

"So I don't get any response from that." You may use it like this (sixpack is a slice of sad)

fullfile( sixpack(1).folder, sixpack(1).name )

[Zahra Soltani]

It is working well. The only problem is that "sad" is not sorted.

[Zahra Soltani]

Just the line below should be added to the code above after defining "sad" to sort the files based on their names. As I have both text and number in the file names, I have to download and use function "natsort" or "natsortfiles" in order to sort the filenames.

sad = natsort({sad.name})