import data function breaks at NaN or not a number
4 views (last 30 days)
Show older comments
Sanwal Yousaf
on 10 Dec 2015
Commented: Sanwal Yousaf
on 17 Dec 2015
I am trying to use import_data function to import dataset from the data file. This data set is generated from iterations of an experiment based upon the response from the subjects. This kind of data set is bound to end up have a Not a Number or a missing response and not extract anything after that. I am trying to figure out a way to import data and not have the function break at Not A Number or a missing response. The function extracts the row before the Not a Number and not after the NaN row. I am attaching the data file that i am interested in. Any help would be greatly appreciated.
P.S. The experiment running the experiment generates a .txt file everytime and i am not allowed to mess with that program.I am including the .txt file as well
0 Comments
Accepted Answer
Guillaume
on 11 Dec 2015
Edited: Guillaume
on 11 Dec 2015
With format like this you have no choice but to write your own parser. Here is a draft:
function testresult = import_testresult(filename)
%import_testresult imports an individual test results stored in a text file
%
% filename: full path of the file to import
% tesresult: a structure containing the test results
filecontent = fileread(filename); %read whole file at once
sections = strsplit(filecontent, ''''''''''''''''''''''''''''''''''''''''''''''''''''''''); %split into individual sections
%the use of quotes for delimiting section is unfortunate as each quote needs to be escaped in matlab
sections = cellfun(@(s) strsplit(s, {'\n', '\r'}), sections, 'UniformOutput', false); %split each sections into lines
%at this point, you have 4 sections (+1 empty one for the last line of the file) split into individual lines
%parse each section.
%
%section 1: assume all values are text. Note that the last cell is empty (due to the way the file is parsed earlier)
propvalues = regexp(sections{1}(1:end-1), '\[([^\]]+)\] (.+)', 'tokens', 'once'); %extract property names and values
propvalues = cellfun(@(pv) {matlab.lang.makeValidName(pv{1}), pv{2}}, propvalues, 'UniformOutput', false); %convert property names into valid matlab variable names
propvalues = vertcat(propvalues{:})'; %concatenate into a single cell array
testdescription = struct(propvalues{:}); %convert into structure
%
%section 2: assume all values are numbers. Note that the first and last cell are empty (due to the way the file is parsed earlier)
propvalues = regexp(sections{2}(2:end-1), '\[([^\]]+)\] (.+)', 'tokens', 'once'); %extract property names and values
propvalues = cellfun(@(pv) {matlab.lang.makeValidName(pv{1}), str2double(pv{2})}, propvalues, 'UniformOutput', false); %convert property names into valid matlab variable names
propvalues = vertcat(propvalues{:})'; %concatenate into a single cell array
testsummary = struct(propvalues{:}); %convert into structure
%
%section 3: assume it's a table of numbers with two header lines. missed and no response are converted to NAN
%Note that the first and last cell are empty (due to the way the file is parsed earlier)
results = cellfun(@(l) strsplit(l, '\t'), sections{3}(4:end-1)', 'UniformOutput', false); %split each line at the tab character
results = str2double(vertcat(results{:})); %convert to a matrix
%
%section 4: assume it's a column of number with a header.
%Note that the first and last cell are empty (due to the way the file is parsed earlier)
rawtimes = str2double(sections{4}(3:end-1)');
%put everything together into one structure
testresults = struct('description', testdescription, 'summary', testsummary, 'results', results, 'rawtimes', rawtimes);
end
There's no validation anywhere that the file conforms to the expectations, you would have to add that to make the code robust.
0 Comments
More Answers (1)
Walter Roberson
on 10 Dec 2015
I have no problem with that file even on a Mac (which cannot use Excel to open the file.)
[num, txt, raw] = xlsread('time-react.xlsx');
The num array has NaN for the missing or string responses. txt or raw can be used to get to the corresponding strings.
7 Comments
Walter Roberson
on 13 Dec 2015
There was originally an excel file that would have been imported without difficulty; that was replaced with the tricky text file.
See Also
Categories
Find more on Large Files and Big Data in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!