How to pull files from mutiple folders

I have a long list of subject folders (~70) I need to pull an individual summary file from each folder, scan and write the information to a new overall summary file.

I have most of the processing code written, however I am having issues automating the opening of each folder and extracting the data. I would prefer not to individually select each file.

Folder names are 'SubjectXX' and within each is a .txt file called 'Summary_SubjectXX.txt' along with other unnecessary folders.

I am using uigetdir to select the parent folder, but I cant seem to navigate into each folder after establishing the directory. Code I have so far is:

%%%

    subjects_folder = uigetdir('*.txt');
    cd(subjects_folder);

    list_of_subject_folders = dir(subjects_folder);

    for i = 3:numel(list_of_subject_folders)

        file_to_process = dir('Subject[0-9][0-9]*'); %This line seems wrong to me

        subjectID = regexp(file_to_process, 'Subject[0-9][0-9]*','match');
        fid = sprintf('%s_%s', 'Summary', subjectID{1});

        fID = fopen(fid, 'r');
        scanned_data = textscan(fID, ['%*s', '\t', '%s', '\n']);
        data_to_print = [subjectID, scanned_data{1, 1}(2:22)];
        fclose(fID);
        cd ../
    end

%%%

I feel like I am just missing a simple step in identifying the folders, however I am relatively new to MATLAB and cannot seem to figure it out with just the 'help' function.

I appreciate any help or advice!

[Walter Roberson]

MS Windows? I am not certain that [0-9] works for dir() in MS Windows. It should be okay in Linux or OS-X I would think.

[Steven]

I am using windows, Also the extra string was a typo, It will be used in the actual code, I was trying to simplify the wordy variables I use!

Would there be a simpler way to scan and open the folders using dir?

Answer 1

[Jan]

Yes, this line is very wrong. What do you expect it to do?

subjects_folder = uigetdir('*.txt');
list_of_subject_folders = dir(subjects_folder);

% Remove \., \.. and all other files and folders starting with a .: 
name = {list_of_subject_folders([list_of_subject_folders.isdir]).name};
name(strncmp(name, '.', 1)) = [];

for i = 1:numel(name)
  aName = name{i};
  if strcnmp(aName, 'Subject', 7)
    subjectID = sscanf(aName, 'Subject%d');
    file = fullfile(subjects_folder, aName, sprintf('Summary_%s', subjectID));
    fid = fopen(fid, 'r');
    if fid < 0, error('Cannot open: %s', file); end
    scanned_data = textscan(fID, ['%*s', '\t', '%s', '\n']);
    ata_to_print = [subjectID, scanned_data{1, 1}(2:22)];
    ...
  end
end

I'm not sure about the file name, because sprintf('%s_%s %s', 'Summary', subjectID{1}) has 3 format specifiers, but only 2 values.

[Steven]

Thank you Jan,

I don't know what I was trying to do with that original line, it was essentially filler until I could figure it out, but now I understand the process necessary.

In my actual code the file name will contain three strings, I edited out the extra string to simplify the question and missed the extra specifier while doing so, sorry for the confusion!

Thanks again!

Answer 2

[Walter Roberson]

Your line

fid = sprintf('%s_%s %s', 'Summary', subjectID{1});

expects three string inputs to sprintf(), but you are only passing in two.

Answer 3

[Steven]

For anyone else who has had this issue, here is the final code that I was able to get working.

%%%

common_filename = 'Summary';
ext = '.txt';
name = {list_of_subject_folders([list_of_subject_folders.isdir]).name};
name(strncmp(name, '.', 1)) = [];

for i = 1:numel(name)
  aName = name{i};  

      % Open subject folder, search for file
      folder = fullfile(subjects_folder, aName);
      cd(folder)

      subjectID = regexp(folder, 'Subject[0-9][0-9]*','match');
      subject_fileID = sprintf('%s_%s%s', common_filename, subjectID{1}, ext);

      file_path = fullfile(folder, subject_fileID);
      fid = fopen(file_path, 'r');

%%%

Enjoy!

Answer 4

[per isakson]

See the FEX contributions:

• Recursive directory listing - Enhanced RDIR
• Recursive directory listing