Import from data from messy text file

Hi, I have looked through the questions and answers that are already available for this topic but wasn't able to apply those to my situation.

For the attached file, I need to extract the data for each "Specimen." The data that i need are the three columns corresponding to the 500+ rows for each specimen. If possible, I would like to save it in to different arrays. One for each specimen.

Answer 1

[dpb]

Pretty straightforward, just use the features in the file to find the wanted sections...it's quite regular in its construction so don't even have to do very much looking for what want--should get you started:

fid=fopen('Sample_DATA.TXT','r');  % open file 
N=cell2mat(textscan(fid,'Number of specimens: %d %*[^\n]','headerlines',11));
for i=1:N                                % loop over number specimens in file
  l=fgetl(fid);                          % initialize string to find start of section
  while isempty(strfind(l,'Specimen:')),l=fgetl(fid);end  % find the Specimen: n start
  while isempty(strfind(l,'_')),        l=fgetl(fid);end  % then line break before data
  d(i)=textscan(fid,'','delimiter',',','collectoutput',1); % the nth specimen data set
end
fid=fclose(fid);

Result will be a cell array of N nx3 data arrays. Above at command line here leaves:

> whos d
Name      Size            Bytes  Class    Attributes
d         1x3             39564  cell               
>> d
d = 
  [532x3 double]    [561x3 double]    [548x3 double]
>>

Looks about right...

If the sections of auxiliary inputs, etc., are always a fixed length could use line counts in 'headerlines' parameter and avoid the while loops looking for section start, but the above will work for different sizes of those elements as long as the sections remain the same. For files of this size, that won't be that much overhead to process on a record-by-record basis anyway...

[Zekeftw]

Thanks dpb. This worked great. However, d is a cell array and I'm not familiar with those. How would I be able to access specific collunms in each d?

For example, I need to divide the second and third column of each d by a scalar. Then plot the second vs third column of each d.

[dpb]

Under help for cell arrays in Data Types section is "Access the contents of cells by indexing with curly braces, {}. For more information, see <Access Data in a Cell Array>. Also note cellfun to do operations on cell arrays w/o explicitly dereferencing them which would serve for step 1 above although plot will need need to have the arrays and while cellfun will work given plot as the function, it doesn't have enough granularity to let you set hold on in between or create a new axes unless you write a complementary function to use that can handle the details so "use the curlies, Luke!"

Answer 2

[Shameer Parmar]

Here is that can help you..

 Data = textread('SAMPLE_DATA.txt', '%s', 'delimiter', '');
 count=1;
 newData = {};
 for i=1:length(Data)
    if ~isempty(strfind(Data{i},','))
        newData{count,1} = Data{i};
        count=count+1;
    end
 end

 count1=1;
 count2=0;
 Specimen = {};
 for i=1:length(newData)
        linedata = newData{i};
        if ~isempty(strfind(linedata(1:2),'1,'))
            count2=count2+1;
            count1=1;
            Specimen{count1,count2} = linedata;
        else
            count1=count1+1;
            Specimen{count1,count2} = linedata;
        end
 end

to check the output

type

 Specimen(:,2);