Importing Data from Text File Loop

I used Matlab's auto-import feature to generate a script that imports data from a text file

%% Setup the Import Options and import the data
opts = delimitedTextImportOptions("NumVariables", 2);

% Specify range and delimiter
opts.DataLines = [1, Inf];
opts.Delimiter = ",";

% Specify column names and types
opts.VariableNames = ["sam12mark", "dashcolorforcolumn2"];
opts.VariableTypes = ["double", "double"];

% Specify file level properties
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";

% Import the data
N9913A6 = readtable("C:\Users\nperson\Desktop\New folder\Scaled\N9913A_6.16.20 04_HF-000013-T01-SCALED.ep", opts);


%% Clear temporary variables
clear opts

This works perfectly but I need it to cycle through a folder full of files (all '.ep' but with different names)

How would I do that?

Answer 1

[per isakson]

First you need to create an array of file specifications. I prefer to use the function dir()

glob = fullfile( "C:\Users\nperson\Desktop\New folder\Scaled", "N*SCALED.ep" ); % I'm guessing 
sad = dir( glob );

Next you need to decide how to store all the tables that will be created. A zillion names like N9913A6 is not a good idea (See TUTORIAL: Why Variables Should Not Be Named Dynamically (eval)). Do you want to keep all the tables? I assume you want that. A structure or a containers.Map are possible (or a cell array, but it's hard to know which file went into cell 17). A structure requires that the field names are legal Matlab names. The keys of a containers.Map can be "any" text.

len = numel( sad );
for jj = 1 : len
    name = ... % you decide
    S.(name) = readtable( fullfile( sad(jj).folder, sad(jj).name ), opts );
end

In response to comment

The code fragment above creates a scalar structure with a zillions of fields. The field names could be something like N9913A6, which is "easily" created from the file name. However, there is little room for meta-data.

A structure array that allows for meta-data could be something like

S.meta_data_1
S.meta_data_2
S.table

and the loop

for jj = 1 : len
    S(jj).meta_data_1 = ...
    S(jj).meta_data_2 = ...
    S(jj).table = readtable( fullfile( sad(jj).folder, sad(jj).name ), opts );
end

Which of these alternatives is the "better" depends on the context. The question provides little of that.

[Stephen23]

Alternatives to containers.Map:

• if names are valid fieldnames, you can use the structure array returned by dir.
• most efficient and easiest to work with: store the meta-data as data in its own right (i.e. in a variable, rather than forcing it into keys/maps/fieldnames). Doing so will make your code simpler and more versatile.