convert 47,000rows * 1000columns .mat file to .txt file

Hi Friends,

I have totally 1000 variables, each has 47,000*1 double values. Now, I need to store all in a txt file.

Please help me to find a solution Thanks in advance

Answer 1

[Image Analyst]

Use

fid = fopen(filename, 'wt')
% Write out variable 1
fprintf(fid, '%f, ', data1);
fprintf(fid, '\n');
% Write out variable 2
fprintf(fid, '%f, ', data2);
fprintf(fid, '\n');
% Write out variable 3
fprintf(fid, '%f, ', data3);
fprintf(fid, '\n');
.
.
.
% Write out variable 1000
fprintf(fid, '%f, ', data1000);
fprintf(fid, '\n');
fclose(fid);

You really should learn how to use arrays and not have 1000 variables. I don't think I've ever written a program in my entire life that had 1000 separate variables.

[Abhishek H P]

Thanks for your quick response, Image Analyst.

Your solution works but as you adviced me, it seems that I should keep writing the variable name 1000 times.

Instead, If I store all the 1000 variables in workspace as one .mat file. I was wondering if this could solve the problem of reentering the variables every time.

[Image Analyst]

Well somehow you already have typed them all because you're using them in your code, aren't you? If it's too cumbersome to do that here while saving them, then why is it not to cumbersome to use them all in your code. I mean, let's say your variables area called var1, var2, var3, ...var1000. Is there someplace in the code where you access each one, like (for example)

var438 = 10 * var219 - var934 + log(var791);

And then you do that again for all 999 other variables? Everyone will tell you this is not a good way to program.

To not have to type them all again you'd have to use dynamic field names, as Walter showed you. But it's best to just avoid that in the first place. Don't use 1000 separate variables. Use one array instead. Then you just have one variable to worry about. What are your variable names and why do you think you need 1000 of them.

[Walter Roberson]

Note that this does not write the data out as columns as was requested in the original question.

Answer 2

[Walter Roberson]

Install https://www.mathworks.com/matlabcentral/fileexchange/10959-sort-nat--natural-order-sort . Then

data_struct = load('TheFileName.mat');
fn = fieldnames(data_struct);
sorted_fn = sort_nat(fn);
num_var = length(sorted_fn);
one_var = data_struct.(sorted_fn{1});
size_of_var = length( one_var );
all_data = zeros(size_of_var, num_var, class(one_var));

for K = 1 : num_var
  all_data(:,K) = data_struct.(sorted_fn{K});
end

Now all of the data has been stored in all_data. But you have not been specific about the output format. Some of the possible outputs include

save OutputFileName.txt all_data -ASCII                     %space, low precision

or

save OutputFileName.txt all_data -ASCII -DOUBLE              %space, higher precision

or

csvwrite('OutputFileName.csv', all_data)                     %commas

or

dlmwrite('OutputFileName.txt', all_data, 'delimiter', '\t')   %tabs

or

xlswrite('OutputFileName.csv', all_data)                      %commas

[Abhishek H P]

Thanks Walter Roberson for efficient code and quick response however, I got an error within for loop

''Conversion to double from struct is not possible.''

[Walter Roberson]

Which line are you getting that message on? If you are getting it on the line

    all_data(:,K) = data_struct.(sorted_fn{K});

then the implication is that your variables are struct rather than vectors. If they are struct then it does not make sense to write them out as 47000 by 1000.

[Walter Roberson]

Please show us the first several lines from the output of

whos -file TheFileName.mat

(using the appropriate filename)

[Abhishek H P]

Thanks Walter Roberson for your time.

I followed your instruction. Output is as follows:

   Name       Size      Bytes    Class     Attributes
     A1      47213*1    377704   double
     A2      47213*1    377704   double
      .           .       .         .
      .           .       .         .
      .           .       .         .
      .           .       .         .
      .           .       .         .
     A996      47213*1    377704   double
     A997        1*1          8    double
     xy          1*1        897284   struct
     gl          1*1          5576   struct

Actually 990 variables are double with 377704 Bytes, and 4 variables are double but 8 Bytes, and rest are structs.

[Image Analyst]

Oh my gosh! You really did create 1000 variables like the FAQ STRONGLY recommends against, and like virtually all other MATLAB programmers strongly discourage. OK, the fix is to just NOT DO THAT in the first place. Do you have control over the bad code that made that .mat file in the first place? If so, that is what we need to fix, rather than trying to fix up a bad situation. An ounce of prevention is better than a pound of cure.

[Walter Roberson]

Modify

   data_struct = load('TheFileName.mat');

to

data_struct = load('TheFileName.mat', 'A*');

[Abhishek H P]

Thanks Image Analyst for your time and concern. I appreciate it...

Actually, the variables Name are not A1,A2,....A990. I gave those for example. The variables are unique For eg: 1st variable = diesel_engine 2nd variable = temperature_room ..... like this there are totally different variables. So data_struct = load('TheFileName.mat', 'A*'); wouldn't be appropriate here.

Also, the .mat file code is not accessible for me.

[Walter Roberson]

data_struct = load('TheFileName.mat');
fn = fieldnames(data_struct);
sorted_fn = sort_nat(fn);
num_var = length(sorted_fn);
one_var = data_struct.(sorted_fn{1});
size_of_var = length( one_var );

all_data = zeros(size_of_var, num_var, class(one_var));
col_names = cell(1, num_var);

used_cols = 0;  
for K = 1 : num_var
  this_data = data_struct.(sorted_fn{K});
  if isnumeric(this_data)
    used_cols = used_cols + 1;
    all_data(:,used_cols) = this_data;
    col_names{used_cols} = sorted_fn{K};
  end
end

all_data(:,used_cols+1:end) = [];  %trim out unused
col_names(used+cols+1:end) = [];   %trim out unused

item_hdr_fmt = '%-23s'
item_num_fmt = '%+23.16e';
hdr_fmt = [repmat([item_hdr_fmt ' '], 1, used_cols-1), item_hdr_fmt '\n'];
item_fmt = [repmat([item_num_fmt ' '], 1, used_cols-1), item_num_fmt '\n'];

fid = fopen('TheOutputFile.txt', 'wt');
fprintf(fid, hdr_fmt, col_names{:});
fprintf(fid, item_fmt, all_data .');   %the transpose is important
fclose(fid);

[Sara]

Thank you so very much Walter!!! :O) :OD

So Awesome!!!

Tiny typo on the line below: the+ should be an _ ;O)

col_names(used+cols+1:end) = []; %trim out unused

col_names(used_cols+1:end) = []; %trim out unused

Thank you!!!

Sara