Read Date time series format

I want to read Date and time series from below format

   % file.txt has time formatted data
   [0803/135825.745523:GGBB:main.cc(182)] 
   [0803/135825.746634:NNFF:boat_control_xhramyos.cc(137)]
   % My Code is as follows   
    VariableNames          = {'Date'};                     
    VariableWidths         = [20] 
    VariableTypes          = {'datetime'};
    opts                   = fixedWidthImportOptions('VariableNames','SelectedVariableNames', [1]);
    opts                   = setvaropts(opts, 'Date', 'InputFormat', 'MMdd/HHmmss.SSSSSS');
    Content.update_engine  = readtable('file.txt', opts);

I am getting following error as I am unable to read the time properly.

Warning: Unable to convert one or more variables to datetime. Specify the correct
InputFormat and DatetimeLocale properties in the DatetimeVariableImportOptions. 
> In matlab.io.internal.functions.ReadTableWithImportOptionsText/executeImpl (line 82)
  In matlab.io.internal.functions.ReadTableWithImportOptions/executeImpl (line 18)
  In matlab.io.internal.functions.ReadTableWithImportOptionsText/execute (line 122)
  In matlab.io.internal.functions.ReadTableWithImportOptions/execute (line 25)
  In matlab.io.internal.functions.ExecutableFunction/validateAndExecute (line 98)

[Walter Roberson]

Why is your width 20 when your inputformat for the date is 18 characters? 20 would include the leading '[' and trailing ':' that you do not account for in your format.

It might make more sense to not used fixed width, and instead tell it to use : as the delimiter, and possibly specify prefix of '['

[Jeremy Hughes]

"I am getting following error"

"Warning: ..."

warning ~= error (Just a bit of MATLAB nit picking)

The warning is saying it's not able to convert the datetime variable, i.e. the text didn't match the input format. It's a warning since it did something... maybe not what you wanted. Probably the whole variable is NaT.

To see what text you're trying to convert, try reading 'string' instead of 'datetime'.

If the file is large, you can use the DataLines property or the preview function to see a handful of rows.

Also, Walter is right on the money about not using fixed width, the lines are not the same length, so this is not what fixed width is meant for. I like his suggestion of using ':' as delimiter, but I'd say just set the other characters as whitespace. e.g.

opts = detectImportOptions('file.txt','Delimiter',':','Whitespace',"[ ]")

t = preview('file.txt', opts)

Then you can see what you're getting, and play with the settings until you like it.

[Life is Wonderful]

Thanks a lot. Will get back to you once I test full file .

[Life is Wonderful]

I tried your solution, but I see problem in assigning variable name and variable type. They take system assigned nomenclature

Time series is not picked up properly. missed out initials fews lines

Can you please send me the right sequence order . For Reference I have added my input file.

[Guillaume]

It was me who suggested to use fixed width import in a previous question, but that was for a different file where the first three columns of the file were indeed fixed width, the 4th one wasn't.

But this new file is formatted differently, so the fixed width format indeed doesn't make sense.

Why do the format of your files keep on changing? I think I've now seen at least three different formats in your questions.

You won't be able to parse this file with readtable. It is neither a proper fixed width file, nor a proper delimited file (the delimiter can also appear as text in the 3rd variable, readtable can't cope with that).

You will have to write your own parser again as I did for this question. regexp should work. you will have to adapt it since it's again a different format.

[Life is Wonderful]

With your help i can move a lot quicker .Thanks a ton .

If can't use fixed width import nor delimiter in readtable , now i am lost.

regexp is much difficult than readtable and i won't prefer since I have to maintain similar coding style for my project if not this is a hard option to do so .Can you please suggest someting else that is easy to do .

Thanks

[Guillaume]

Can you please suggest someting else that is easy to do

Stick to one file format, the fixed width one in your previous question. Or use an even better one (e.g. comma delimited with text in quoted strings)

Otherwise, I'm afraid there's no easy way to do it. You'll have to use regexp or write your own parser.

[Life is Wonderful]

OK if no other choices left .Can you please help me with sample script here

Answer 1

[Life is Wonderful]

milliseconds information is in place(.SSSSSS)but conversion is not done properly.

[Walter Roberson]

The conversion looks perfect when I try.

timestamps.Format = 'dd-MMM-uuuu HH:mm:ss.SSSSSS';
>> timestamps(1:2)

ans = 

  1×2 datetime array

   03-Aug-2019 13:58:25.745523   03-Aug-2019 13:58:25.746634

>> S(1:100)

ans =

    '[0803/135825.745523:INFO:main.cc(182)] A/B Update Engine starting
     [0803/135825.746634:INFO:boot_cont'

745523 and 746634 exactly match the input text.

[Life is Wonderful]

Thanks a lot for your efforts.

Answer 2

[Walter Roberson]

S = fileread('update_engine.txt');
timestamps = regexp(S, '^.(.{18})', 'lineanchors', 'match');
ts_dt = datetime(timestamps, 'InputFormat', 'MMdd/HHmmss.SSSSSS');

[Life is Wonderful]

This does not work.

Console error message

Error using datetime (line 635)
Unable to convert the text to datetime using the format 'MMdd/HHmmss.SSSSSS'.

The other thing is , there is no support to get message section apart from timestamp grab. I need time as well as message content. Thanks

[Walter Roberson]

S = fileread('update_engine.txt');
pieces = regexp(S, '^\[(?<timestamp>[^:]+):(?<category>[^:]+):(?<filename>[^\(]+)\((?<line>\d+)\)\]\s*(?<details>.*)', 'names','lineanchors','dotexceptnewline');
timestamps = datetime({pieces.timestamp}, 'InputFormat', 'MMdd/HHmmss.SSSSSS');

pieces will be a struct array with fields timestamp (containing the text of the timestamp), category (e.g., INFO), filename (e.g., 'main.cc'), line (e.g., '182'), and details (e.g., 'A/B Update Engine starting')

[Life is Wonderful]

Thanks a lot Walter.

Why I don't see the milliseconds information "timestamps" ? Any help

[Walter Roberson]

You can change the Format property to display the milliseconds.