I have a .txt file which is not in a 'MATLAB' friendly format and I am trying to manipulate the data so that I can then delimit it and plot it later

Hi,

I have attached the .txt file I am trying to parse.

The data is set out in a way where the Date, Time, Latitude and Longitude are on the line above. I then want the timestamp along with the >FFT response (which I could then delimit later on). So far I have got to identifying where the data I need is, but cannot get it to display what those lines are and then copy that to a new text file to import/manipulate. Anything I do I get back "Undefined operator '==' for input arguments of type 'cell'." I want to keep this as simple code as possible.

clear all

%% chopping data up
filename = 'AMT_20181115082845.txt';
delimiterIn = ' ';
Raw = importdata(filename,delimiterIn);
Copy = Raw(2:end);
Raw = Raw(1:end-1);
Table = [Raw(3:end), Copy(3:end)];


%% FFT find
FFTindex = strfind(Table,'>FFT','ForceCellOutput', true);
A = [FFTindex(:,2), FFTindex(:,2)];

Answer 1

[Guillaume]

I don't know what format you want ultimately, nor what can change in the format of the file. This is how I'd start:

filecontent = fileread('AMT_20181115082845.txt');  %read whole file at once
ffts = regexp(filecontent, '([^\r\n]+)[\r\n]+>FFT:([^\r\n]+)', 'tokens');  %get line before >FFT and content on that line
ffts = vertcat(ffts{:});  %convert to Nx2 cell array
datlonglat = cellfun(@(datlonglat) textscan(datlonglat, '%f/%f/%f,%f:%f:%f,%f,%f'), ffts(:, 1), 'UniformOutput', false); %parse line before ffts
datlonglat = cell2mat(vertcat(datlonglat{:})); %convert to Nx8 matrix
data = table(datetime(datlonglat(:, 1:6)), datlonglat(:, 7), datlonglat(:, 8), ffts(:, 2), 'VariableNames', {'datetime', 'latitude', 'longitude', 'ffts'})

I have no idea how the ffts part should be parsed, I've left as is.

[Damsel in Distress]

Hi,

Thanks for that. I have had a go at parsing the FFT bit using your logic above.

fft = cellfun(@(fft) textscan(fft, '%f,E%f,N%f,F%f,A%f,GAIN%f,%f;%f;%f;%f;%f;%f;%f;%f;f;%%f;%f;%f;f;%f;%f;%f'), ffts(:, 2), 'UniformOutput', false); %parse fft line
fft = cell2mat(vertcat(fft{:})); %convert to Nx8 matrix 

Thanks for your help :)

[Guillaume]

Possibly, the simplest is to change the initial regular expression so that the 2nd token only captures the relevant part of that FFT line and textscan that:

filecontent = fileread('AMT_20181115082845.txt');  %read whole file at once
ffts = regexp(filecontent, '([^\r\n]+)[\r\n]+>FFT:[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,([^\r\n]+)', 'tokens');  %get line before >FFT and content on that line after the first 6 comma separated entries
ffts = vertcat(ffts{:});  %convert to Nx2 cell array
datlonglat = cellfun(@(datlonglat) textscan(datlonglat, '%f/%f/%f,%f:%f:%f,%f,%f'), ffts(:, 1), 'UniformOutput', false); %parse line before ffts
datlonglat = cell2mat(vertcat(datlonglat{:})); %convert to Nx8 matrix
fftvalues = cellfun(@(fftv) textscan(fftv, '%f', 'Delimiter', ';'), ffts(:, 2), 'UniformOutput', false)
fftvalues = cell2mat([fftvalues{:}])';
data = table(datetime(datlonglat(:, 1:6)), datlonglat(:, 7), datlonglat(:, 8), fftvalues, 'VariableNames', {'datetime', 'latitude', 'longitude', 'fftvalues'})