Why do I get correlation result NaN?

3 views (last 30 days)
I have two matrices A is of 1*1058 and B is 1*1058, both matrices have some NaN values included in them. Is there any way to get correlation between these two matrices.

Accepted Answer

Ive J
Ive J on 15 Aug 2021
Unless you have a good reason to impute your missing data, you can remove missing values from both vectors.
nanidx = isnan(A) | isnan(B);
corr(A(~nanidx), B(~nanidx))
Walter Roberson
Walter Roberson on 16 Aug 2021
That's what Ive J's code does: removes all locations for which X is nan or Y is nan.

Sign in to comment.

More Answers (1)

Chunru on 13 Aug 2021
Use "fillmissing" to fill up the nans before computing the correlation. doc fillmissing for more details.
Walter Roberson
Walter Roberson on 17 Aug 2021
Mathematically, if you have vectors A and B, then
cAB = corr(A,B);
P = randperm(numel(A));
pA = A(P);
pB = B(P);
cpAB = corr(pA, pB);
then cAB needs to equal cpAB to within round-off. The order of the elements relative to each other in their same vectors do not matter: only the correspondance between the two vectors matter.
If, though, you were to fillmissing(A) and compare that to fillmissing(pA) then you would get different results, because fillmissing works based upon nearby values, under the assumption there is some kind of smooth continuity. This is not really compatible with the mathematics of correlation which does not care about order within the sequence.
If you have some prediction function for your vectors, then Yes, it might make sense to apply that prediction function. It might even make sense to apply something like narx to predict in some cases. But that would have to be done based upon knowledge of what the vectors represent. fillmissing() has no knowledge of what they represent.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!