Getting stuck after first loop iteration-- Coregistering .nii files to spectroscopy files

Hello,

I am working on editing/modifying a script a colleague wrote. Basically it is a manual (via this script) batch processing of a software run in MATLAB. It runs successfully for the first iteration, but I am stuck on the second (I only have two files in the folder for testing purposes). I am getting the error

"Index exceeds matrix dimensions" at line 154::                
cur_file_NII = NII_filename{l};

Which is when I am trying to set the current .nii file in the loop/iteration to run the Coregistration. It is failing on the attempt to find the second file.

Also, when I try to use "cur_file_NII" the software gives an error about the number of .nii files found. However when I type "curr_file_NII" into the command window I get: B:\BBA\Gannet\Tests\8.13.18_testManualBatchScript\NII_files\RAN133-1a3_T1.nii which makes me think it should be fine...

I believe it has to do with "NII_foldername" because when I run the script that variable shows as 0. However, when I Evaluate Selection for this loop

for i = 1:length(NII_list)
    NII_foldername{i} = NII_list(i).name;

end;

indexing NII_foldername gives me the .nii files in my folder, with a correct length. Not sure why it gets set to zero before the end.

Thanks for the help. Here is the code. Unfortunately I can't include the MRS_struct file because it's too large.

%%Paths- source and final
source_path = 'B:\BBA\Gannet\Tests\8.13.18_testManualBatchScript'; %where the dicoms are; Can be a larger list, even if we are currently only doing a newer/smaller
%batch. The excel list document will take care of only what we want
%fit_path = 'B:\BBA\Gannet\TAP\TEMP\GannetFit_output'; %used for renaming MRS_struct
MRS_struct_path = 'B:\BBA\Gannet\Tests\8.13.18_testManualBatchScript'; %location of MRS_struct file that will be renamed
NII_path = 'B:\BBA\Gannet\Tests\8.13.18_testManualBatchScript\NII_files'; %folder of NII files

scan_code_filename = 'List_to_analyze.csv'; % this is the .csv we look through to get scan codes
% change per analysis, keep this in VOI folder

%%Keywords + setup
%analysis_name = 'BBA_ALC'

study_name = 'RAN'; %change per study/ analysis

%These keywords are from the DICOM files- to set file_water and file_metab
water_keyword = 'TEMP GABA H2O svs_se'; %these should be the same for RAN vs TAP etc.
metab_keyword = 'TEMP GABA svs_edit_RFA'; %most of them with this
NII_keyword = '_T1'; %this may work but need to test
%metab_keyword = 'TEMP GABA svs_edit_859A'; %use this for PRE RAN 69

count = 0;

%%read the scan code
cd(source_path);

% look through CSV file. Open, look through (comma delim), close
fid = fopen(scan_code_filename);
scan_code_file_text = textscan(fid, '%s', 'Delimiter', ',');
fclose(fid);

% extract the list of scan codes from the CSV file

scan_code_exp = strcat(study_name,'\d\d\d-\d');
scan_code_array = regexp(scan_code_file_text{1},scan_code_exp, 'match'); %look for scan_code_exp within scan_code_file_text (.csv file)
%list of PIDs from .csv file

%put scan codes into final array
scan_code_array = scan_code_array(~cellfun('isempty', scan_code_array)); %making a usable array here and below

for i = 1:length(scan_code_array) %for loop to deal with cell array structure?
  
  
  scan_code_array{i} = scan_code_array{i}{1};
  
end

scan_code_array = unique(scan_code_array); %make sure each ID is unique

%%Go to dicom location and find files-- LOOP Start

cd(source_path)

display ('Find DICOM files and set water/metab files');

% make an array for dicom folder name
dicom_list = dir(source_path); %top of layer where all dicom FOLDERS are stored (in TEMP/INS folders)
%DICOM folders
dicom_foldername = cell(length(dicom_list),1); %cell struct w length of dicom list X 1

% make an array for nii folder name
%NII_list = dir([NII_path, '/*.nii']);%list all conent from location of MAT files
NII_list = dir(NII_path); %location where nii files are
NII_foldername = cell(length(NII_list),1); %cell struct w length (rows) of nii_list x 1 column

for i = 1:length(dicom_list) %for the length of ^, get ith name from dicom list
  dicom_foldername{i} = dicom_list(i).name; %setting d_foldername to the name of the list,
  %getting a name of a folder
  
end

for i = 1:length(NII_list)
  NII_foldername{i} = NII_list(i).name;
  
end;

%display ('test11');

% find the files
for i = 1: length(scan_code_array)
  display ('inside for loop');
  cur_scan_code = scan_code_array{i};  %current scan code is the ith of scan code array
  
  %Setup dicom search via cur_scan_code (PID)
  tmp = strfind(dicom_foldername, cur_scan_code); %find current scan code ^ (PID) within dicom folders
  tmp_foldername = dicom_foldername(~cellfun('isempty', tmp)); %folders from ^ scan code (should return metab and water)
  
  %Setup nii search via cur_scan_code (PID)
  tmp_nii = strfind(NII_foldername, cur_scan_code); %find current scan code ^^ (PID) within folder with NII files
  tmp_nii_foldername = NII_foldername(~cellfun('isempty',tmp_nii)); %folders from ^ scan code (should return .nii file)
  
  if (~isempty(tmp_foldername)) %if tmp_foldername comes up with folders/data then.... (i.e is not empty)
    
    % find water and metabolite folders in the list of tmp_foldername based on the key words
    % now we are in a SPECIFIC IDs DICOM folders (both water and metab show up) (tmp_foldername(i))
    
    % find the full name of water file
    water_foldername = tmp_foldername(~cellfun('isempty',strfind(tmp_foldername,water_keyword)));
    N_waterfolder = length(water_foldername);
    water_filename = cell(N_waterfolder, 1);
    for j = 1:N_waterfolder
      
      tmp_waterfile_list = dir([source_path,filesep,water_foldername{j},filesep, '/*.IMA']);
      water_filename{j} = [source_path,filesep,water_foldername{j},filesep,tmp_waterfile_list(1).name];
      
    end
    
    % find the full name of metabolite file-- path, no .ima extension yet
    metab_foldername = tmp_foldername(~cellfun('isempty',strfind(tmp_foldername,metab_keyword)));
    N_metabfolder = length(metab_foldername);
    metab_filename = cell(N_metabfolder, 1);
    for j = 1:N_metabfolder %getting a .ima file to paste at end of metab path (1/320 files)
      
      tmp_metabfile_list = dir([source_path,filesep,metab_foldername{j},filesep, '/*.IMA']);
      metab_filename{j} = [source_path,filesep,metab_foldername{j},filesep,tmp_metabfile_list(1).name];
      
    end
    
    %find the full name/ path of nii file
    NII_foldername = tmp_nii_foldername(~cellfun('isempty',strfind(tmp_nii_foldername,NII_keyword)));
    N_NIIfolder = length(NII_foldername);
    NII_filename = cell(N_NIIfolder, 1);
    for j = 1:N_NIIfolder
      
      tmp_niifile_list = dir([NII_path,filesep,NII_foldername{j}]);
      NII_filename{j} = [NII_path,filesep,tmp_niifile_list(1).name]; %here NII_filename is not empty and is the first nii in folder
      
    end
    
    
    display('found all the files');
    
    for j = 1: N_waterfolder
      for k = 1:N_metabfolder
        for l = 1:N_waterfolder %this should be N_NIIfolder but it is giving zero for length...
          
          cur_file_water = water_filename{j};
          cur_file_metab = metab_filename{k};
          cur_file_NII = NII_filename{l}; %HERE, UNABLE TO INDEX BECAUSE NII_filename COMES UP AS EMPTY
          
          cd(source_path) %VOI folder
          
          % Run Load
          MRS_struct=GannetLoad({cur_file_metab},{cur_file_water});
          
          % Run Fit
          MRS_struct=GannetFit(MRS_struct);
          
          % Run Coregister
          MRS_struct = GannetCoRegister(MRS_struct, (NII_filename)); %not able to run 2nd image/iteration
          
          % Run Segment
          %MRS_struct = GannetSegment(MRS_struct);
          
          % Run Quantify
          %MRS_struct = GannetQuantify(MRS_struct);
          
          display ('fail test 222');
          %Rename MRS_struct to include unique ID w water+metab filenames
          %doing this to override Gannet batch and create a unique/single MRS_struct file for each P
          
          cd(source_path); %the MRS_struct_vox1 file in in source_path location (main folder)
          
          [water_filepath, water_name,water_ext] = fileparts(cur_file_water); %use cur_file so it's the PID we're on, not j/k
          [metab_filepath, metab_name,metab_ext] = fileparts(cur_file_metab);
          %[NII_filepath, NII_name, NII_ext] = fileparts(cur_file_NII);
          
          
          display ('Changing name of MRS_struct_vox1 file');
          old_filename = 'MRS_struct_vox1.mat';
          new_filename= strcat(water_name,'_',metab_name,'_','MRS_struct.mat');
          %new_filename= strcat(water_name,'_',metab_name, '_',NII_name,'_','MRS_struct.mat'); %need '.mat' when saving newfile
          %^ saving new MRS_struct file w/ water,metab,NII files used in name
          movefile(old_filename, new_filename);
          
          
        end
      end
    end
    
    
    
  end
end

[Image Analyst]

What is your directory structure? What files, with what extensions, are expected to be in what folders?

[Holly Pothier]

I have my nifti .nii files in the 'NII_path' and the dicom .ima folders are in 'source_path', the MRS_struct file produced after the second command (GannetFit) is in 'MRS_struct_path'.

[Dennis]

I might be mistaken but i think NII_list is a structure and length(NII_list) is bigger than 2, even with only 2 files in your folder.

[Holly Pothier]

Yes, that is correct. Is that causing a problem? NII_list contains 4 elements:

>> NII_list.name

ans =

.

ans =

..

ans =

RAN133-1a3_T1.nii

ans =

RAN154-1a4_T1.nii

[Holly Pothier]

After the first iteration, when all the variables are set and ready to run the software, it seems all the variables are correct. The arrays are working and have the correct data in them, and the first iteration is works fine. For the second iteration I either get the indexing error (as the code is above) or it just stops before running the Gannet commands (when I change for l = 1:N_waterfolder to for l = 1:N_NIIfolder). After the first iteration, after it stops, when I check N_waterfolder, it shows as zero. However it shows as 1 if i check after the first successful iteration, before the stop. Not sure if that helps...

[Holly Pothier]

Hello all,

I have tried messing around with this and I can't figure it out. I get a few different errors (all same line) when I try different things. Here is a summary of my errors that occur in the line

MRS_struct = GannetCoRegister(MRS_struct, (NII_filename{l}));

When I use "cur_file_NII" I get an error from the software (Gannet) saying the number of nifti files does not match the number of datasets loaded. At this point it has loaded 1 dataset, and "cur_file_NII" is the path of a single Nifti. I get the same error when I use "NII_filename {l}"

When I use "NII_filename" without the l index, the first iteration is succesful-- no error from software, and the output is fine. However, when in the second iteration, it stops at the line "display ('found all the files')". So it does not enter the loop. I think it is because of the

for:l = 1:N_NIIfolder
loop. If I stop it before the first iteration is done, this value is 1. If I check this variable (N_NIIfolder) after the loop is over I get 0 which makes me think that is why it is stopping but I'm not sure.

When I take out that for loop (for:1 = N_NIIfolder) I am unable to index the correct NII file and get the error that the number of NIFTI files does not match the loaded MRS_struct files (1). Consequently, 'cur_file_NII' comes up as an empty cell array 0 by 1.

Any help would be appreciated- I'm pretty stuck.

[Image Analyst]

This is quite a long script for us to try to debug without being able to run it. Can't you attach some data or give us the file/folder structure at all? Otherwise, just follow normal debugging methods. It looks like when you do this:

NII_list = dir(NII_path);

that you might be getting folders in there, so you mgiht want to skip those

if NII_list(i).isdir
    continue; % Skip directories
end

Or else go back to getting a list of only files with the .nii file extesions.

[Holly Pothier]

Unfortunately I can't attach all the files because the software uses 320 + 1 .ima files to run. Also, you would need the software (Gannet) installed. However, I did go back to getting only files with the .nii file extension

NII_list = dir([NII_path, '/*.nii']);%list all conent from location of NIFTI files

This seems to be fine now, I can't remember why I strayed away from using it. Although, I am running into the same problem. The first iteration works fine, and all the variables are what they are supposed to be. But when I get to the second iteration when GannetCoRegister is called, all the variables are empty arrays when indexed in the command window. I'm not sure how/why this happens in the second iteration?

for i = 1:length(NII_list)
    NII_foldername{i} = NII_list(i).name; %setting NII_foldername (which is empty before this) to contain the names/files from NII_list
                                          %after this NII_foldername should have the nii files

end;

Is it possible this is causing the second iteration to come up empty? length(NII_list) comes up as 2 which seems correct because NII_list comes up as a struct array 2x1-- 2 NII files in that folder. NII_list.name comes up with the correct names of the NII files in that folder. __

What do you mean by file/folder structure? Do you mean what is in each folder? I will try to provide whatever I can...

NII_path- folder with NII files in there.

source_path- main folder that contains folders of dicoms and NIIs (located in ^ folder). The dicom files are grouped in folders by ID. For example, ID #123 will have a folder for water dicoms (1), metabolite dicoms (320), and one NII file within NII_path.

So the script finds the water dicoms, the metabolite dicoms, and the NII file for ID123 and then runs the software. It works fine for iteration 1, but when I get to the second it fails due to the arrays coming up empty.

I hope this helps. And thank you for the help, it is much appreciated.