Trouble reading csv file

My files are saved from the microscope as .csv, but there maybe a problem with the format because opening the the file in and clicking save as, it shows "save as type" as Unicode Text (*.txt).

Anyways, I am using the below

[nAme, PATHNAME, FILTERINDEX] = uigetfile('*.csv');
 Dataset = csvread(nAme,1,0);

but I get the following error;

Error using dlmread (line 147)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1, field number
1) ==>  \n
Error in csvread (line 48)
    m=dlmread(filename, ',', r, c);

However, when I resave the file and change it from Unicode text to CSV (comma delimited), the code runs. I have so many files and having to resave all is a huge task. Can some suggest what I can do? PS: I have tried reading it as a (*.txt) but the call up doesn't see the file as (*.txt)

EDIT: Ifechukwu Michaels's "Answer" and uploaded file moved here:

Opening the file in Excel and clicking "Save as". I have attached a sample. Thanks.

[per isakson]

"opening the the file in and clicking save as" in what program?

Could you upload (with the paperclip button) a sample file?

[per isakson]

I've downloaded you sample file, 7000.csv. The editor, Notepad++, identifies the text encoding as UCS-2 LE BOM.

fid = fopen( '7000.csv', 'r', 'ieee-le', 'UCS-2 LE' );
cac = textscan( fid, '%f%f%f%f%f%f%f', 'Headerlines',1          ...
                ,   'CollectOutput',true, 'ReturnOnError',false  );
fclose( fid );

returns the error

Error using textscan
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1, field number 1) ==>  
 1    - 4 2 9 . 1 1 2 7    6 9 9 7 . 2 8 9 9    0 . 0 0 0 0    - 7 4 . 0 4 3 2       \n
Error in Untitled (line 3)
cac = textscan( fid, '%f%f%f%f%f%f%f', 'Headerlines',1          ...

Yes, each digit is coded with two bytes.

Google "windows text file change encoding" returned this Changing source files encoding and some fun with PowerShell and more

Your answer to my comment should have been entered as a comment after my comment. Not as an answer to your own question. One reason is that some potential contributors will not look at a question on reading a text file, which has already received an answer.

Answer 1

[per isakson]

This hack reads and parses your file

str = fileread( '7000.csv' );
iszero = double(str) == 0;
str(iszero) = [];
cac = textscan( str, '%f%f%f%f%f%f%f', 'Headerlines',1          ...
                ,   'CollectOutput',true, 'ReturnOnError',false  );

and

>> cac{1}
ans =
   1.0e+03 *
    0.0010   -0.4291    6.9973         0   -0.0740       NaN       NaN
    0.0020   -0.4278    6.9973    0.0013   -0.0739         0    0.0068
    0.0030   -0.4265    6.9973    0.0026   -0.0735         0    0.0163
    0.0040   -0.4252    6.9973    0.0039   -0.0730         0    0.0194
    0.0050   -0.4239    6.9973    0.0052   -0.0725         0    0.0213

[Ifechukwu Michaels]

Even though it reads the data, textscan reads it as a cell array rather than a list of numbers and therefore I cannot assign it to a simple variable.

[per isakson]

"a list of numbers" In exactly what form do you want the data? Seven column vectors, <nx1 double>? One matrix, <nx7 double>? Or what?

[Ifechukwu Michaels]

Honestly, I am still trying to figure out where the error is from. It reads the data as nx7 double, which is fine. But when I assign x = cac(:,4); y = cac(:,5);

it bring back this error;

Index exceeds matrix dimensions. Error in (line 18) x = cac(:,4);

[per isakson]

Try

>> x = cac{1}(:,4); y = cac{1}(:,5);
>> plot(x,y)
>>

and

>> M = cac{1};
>> plot(M(:,4),M(:,5))

[Ifechukwu Michaels]

Works perfectly now!! Thanks for the assistance. I appreciate.

[Arshey Dhangekar]

I have two csv file WT and both have fixed have 52 header. I tried to use below command to read however headers are showing as Var1,Var2..to Var52 instead of Store no,Date,Time....till 52th header. How can I fix the header problem with importing all the datas from those files and without doing any edit in excel file? Using opts = detectImportOptions()

table = readtable('WT_201120.csv'); 

[Walter Roberson]

As discussed in https://www.mathworks.com/matlabcentral/answers/858175-how-can-i-add-path-in-matlab-to-read-files-csv-and-excel-files-from-specific-folder-and-read-the-dat#answer_726490 your list of variables is separated by semi-colon instead of comma.

Answer 2

[Walter Roberson]

See https://www.mathworks.com/matlabcentral/answers/337326-import-mixed-format-text-file-of-unknown-number-of-columns#answer_264655

(Note: that version does not have my enhancements to deal with duration objects)

[Ifechukwu Michaels]

Thanks for this input. I am still building my competence in MATLAB, hence I am wondering how the "function" form works? How can I incorporate it to my code? My code simply uses "uigetfile" to call up the file, it then uses csvread to read and then assigns variables and then does some calculation and plotting. Please give me a little more direction. Thanks.

[Walter Roberson]

Datatable = csv2table(nAme);

Provided you have R2013b or later, this would return a table() object. The table variable names would be derived from the first line of the file, which could have the effect of modifying them, such as replacing spaces with underscores, or adding a leading character if the word begins with a digit. If you want to see the original version of the first line, then

[Datatable, ~, headers] = csv2table(nAme);

Answer 3

[Alec Jacobson]

I ran into this issue copy-pasting data in excel into a new sheet and saving as .csv.

Seems on mac, this maneuver triggers Excel to save as "CSV UTF-8 (Comma delimited) (.csv)" which Matlab (2019a at least) doesn't understand. Specifically there seems to be an initial problem character. I know, I know, UTF-8 has been around since 1993 and MATLAB doesn't support it.

Here's a work around. Save as in excel as "Comma Separated Values (.csv)". This will probably remove any unicode text so watch out. For me, I just had numbers so it was fine.

[Walter Roberson]

MATLAB does not understand Byte Order Mark on UTF-8, which is something that the standards recommend against including (I do not understand their reasoning; my feeling is that it is better to include it.)