How to find variance and std in matlab without using zeros in matrix?

I have a matrix (pm2d), and i need to calculate the std and (population) variance in each column without using the zero values. I was wondering if i could use a for loop or an if statement?

For my variance i used:

var = sum(pm2d.^2)/(length(pm2d)-1) - (length(pm2d))*mean(pm2d).^2/(length(pm2d)-1)

But that took the zeros into account...

And for the standard deviation i used:

S = std(pm2d)

which definitely used the zeros.

Every code i try to write is not working. Any assistance would be appreciated! Thanks!

Answer 1

[Vandana Rajan]

Hi,

You can use nanvar and nanstd functions in statistics toolbox.

>> b = pm2d; % just to retain the original matrix
>> b(b==0) = NaN;
>> nz_var = nanvar(b);
>> nz_std = nanstd(b);

Of course, this solution works only if you have license to statistics toolbox.

[Leesy]

I do not, is there another way to do it?

[Rik]

Yes. My solution, or the one Jan Simon suggested (which should have better performance).

Answer 2

[Rik]

In my solution, I abuse the option of omitting NaNs when using mean and std

pm2d_temp=pm2d;%create a copy
pm2d_temp(pm2d_temp==0)=NaN;%overwrite zeroes with NaN
var = sum(pm2d.^2)/(length(pm2d)-1) - (length(pm2d))*mean(pm2d_temp,'omitnan').^2/(length(pm2d)-1)
S=std(pm2d_temp,'omitnan');

[Vandana Rajan]

That's cool :)

Answer 3

[Jan]

It works with replacing the zeros by NaNs and ignoring the NaNs, but you can do this directly also:

function [m, v, s] = StatsNonZeros(x, dim)
if nargin < 2  % Default: first non-singelton dimension
  dimv = [find(size(x) ~= 1), 1];  %#ok<MXFND>
  dim  = dimv(1);
end

n  = sum(x ~= 0, dim);     % Number of non zero elements along dim
m  = sum(x, dim) ./ n;     % Zeros are neutral in the sum
v  = sum(bsxfun(@minus, x, m) .^ 2, dim) ./ (n - 1);
s  = sqrt(v);
end

This is what happens inside nanmean and nanstd also, after the NaNs have been replaced by zeros. Therefore it is an indirection to replace the zeros by NaNs at first.

Call it as:

[m,v,s] = StatsNonZero(pm2d)

[Franck Eitel]

What's the meaning of 's' here? I think we were looking for the variance and standard deviation. Pls could you clarify it for me?

[Rik]

[m, v, s] are the mean, variance, and standard deviation, although I presume you will have found that by now.