Fastest way to read mixed-type and position-delimited text files

3 views (last 30 days)
Blue on 7 May 2020
Commented: Star Strider on 7 May 2020
I need to convert mixed-type text files that are position-delimited into cell arrays. To do this, I have a reference file which contains the name of each variable as well as the start and end position of each variable. My problem is that the code below is, of course, far too slow, particularly when comes the time to read files that have 200-300k lines. So. What would be the fastest way to convert mixed-type and position-delimited text files into cell arrays ?
% Mixed-type file
file = {'10036N', '12456J', '56786N', '89006N'}'
% reference file (variable name, start position, end position)
reference_file = {'var1', 1, 1; 'var2', 2, 4; 'var3', 5, 6;}
% empty cell array for output
c = cell(length(file), length(reference_file));
% Loop over each line and extract each variable based on position (obviously very slow)
for j = 1:length(file)
for k = 1:length(reference_file) % start position; stop position
c(j, k) = extractBetween(file(j, 1), reference_file(k, 2), reference_file(k, 3))
Thank you,

Accepted Answer

Star Strider
Star Strider on 7 May 2020
If you have R2016b or later, see if the FixedWidthImportOptions function will do what you want. (I have no experience with it, so my ability to help you with it is therefore somewhat limited. If you have problems with it, attach a representative sample file for me to experiment with.) You will probably need to use it with readtable , since that is the only function that FixedWidthImportOptions references in its documentation.

More Answers (0)


Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!