Readtable For Loop for importing large quantities of csv files in to Matlab
14 views (last 30 days)
Show older comments
Hello,
I am currently working on a project which requires me to import sequential data into matlab so that it can be processed and analysed as one data set. There is multiple csv files because the data logger creates a new file every 40mb or so.
I have put together the following code to import the data.
if true
% code
end
clear all
close all
d = uigetdir();
filePattern = fullfile(d, '*.csv');
file = dir(filePattern);
x={};
for k = 1 : numel(file)
baseFileName = file(k).name;
fullFileName = fullfile(d, baseFileName)
x{k} = readtable(fullFileName);
end
There are 4 things I have not done yet.
1. Preallocate memory for x - I don't know how much to preallocate.
2. I am not appending the data in to one matrix. Rather, a cell array combining each set of data. For Example 3 sets of data creates a 1x3 cell array each including a table of the data.
3. Added a status print in cmd window i.e: "read file xyz"
4. Not added datetime format because the first column has cotents like this: "31/03/2017 07:58:31"
Here is an example of the output.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/197554/image.png)
In this case what I actually need is a 64,803 x 191 table (Need to keep variable names - each row is a specific variable).
How do I achieve this?
For any given data set I will always have the same amount of columns but the number of rows may vary in each file.
0 Comments
Accepted Answer
Guillaume
on 10 Oct 2018
1. Replace
x = {};
by
x = cell(1, numel(file)); %better name for that variable advised. How about: tables
Note that unless the number of files is significant, this won't have any noticeable impact as it only allocates memory for the cells, not for the tables that go into the cells. This is normal and how cell arrays work.
2. That's fine and is the proper way to do it
3. After the readtable:
fprintf('read file %s\n', fullFileName);
4. Assuming all the tables have the same variable names, to concatenate them all into one table, after the loop:
bigtable = vertcat(x{:});
4 Comments
SUMAN
on 21 Feb 2023
if table doesn't have same variable name then how to concatenate them to one table
More Answers (0)
See Also
Categories
Find more on Logical in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!