Reading data from csv or tsv file starting from some line

Hi All,

I need to read data from a file openable in notepad. The file format can be csv or tsv. But the data will start from some line say 3 or 4.

Similar file has been attached. In this file the values are starting from line 5 and format is tsv. So line 5 will be read first and there will be five values which should be stored in a matrix and then line 6 values and so on.

I have tried too many ways but nothing is working.

Thanks all

Answer 1

[Cris LaPierre]

Is the format of your data the same in all your files? If yes, then use the import tool. This will allow you to interactively configure your import. You can then generate the corresponding code (as a script or a function) that can be used to automate the import process of all files with the same formatting.

If you do want to create an importing function, this video shows you how to generate and reuse your code.

And if you have a lot of files, this video shows you how to use the import function you have created in conjunction with a datastore to import multiple files.

I'd suggest reading your data in as a table. This short video shows how to access data in a table.

[Cris LaPierre]

Just a comment - the autogenerated code is not always the most succinct. If you want to do this yourself, the following code can get you started.

opts = detectImportOptions("RSN6_IMPVALL.I_I-ELC180.DT2","FileType","text");
opts.DataLines=5;
data = readtable("RSN6_IMPVALL.I_I-ELC180.DT2",opts);
data.Properties.VariableNames = {'A','B','C','D','E'}

stackedplot(data)

[Cris LaPierre]

And even shorter

data = readtable("RSN6_IMPVALL.I_I-ELC180.DT2","NumHeaderLines",4,'FileType',"text")

[Ahmed Anas]

Thanks Cris LaPierre,

But which version of Matlab you are using, am using 2017a, But your code is not working for me.

[Ahmed Anas]

And can we use "dlmread" for reading such files. The format should be same for each line.

[Cris LaPierre]

Sorry for not catching that. The import capabilities constantly improve, which is why my code, tested is 2020a, doesn't work in 2017a.

There are several ways you could import this file. I found that dlmread did not have all the options needed to read in the data correctly. The file appears to be fixed width and not tab delimited. For me, textscan worked the best.

fileID = fopen('AhmedAnas_RSN6_IMPVALL.I_I-ELC180.DT2');
data = textscan(fileID, '%f%f%f%f%f', 'HeaderLines', 4);
data = cell2mat(data)
fclose(fileID);

[Ahmed Anas]

Cris LaPierre, Thanks Bro, the last one worked perfectly..

But i have one question here, is it possible to get the number of lines where the data is not properly formatted,just as in this case its value is 4.

[Cris LaPierre]

I'm not aware of a simple, automatic way to do this. You can search MATLAB Answers for other posts on automatically detecting the number of headerlines, but each solution is typically tailored to the use case.

One idea that comes to mind would be to use fgetl to read the file one line at a time. Use strsplit to separate the columns of data in that line. You would then need to write some sort of logic to determine if the first value can be converted to a valid number (it will be read in as a string).