Why is my p-value matrix after using corrcoef filled with NaN?

Hi I have a code where I want to calculate the correlation coefficient and p-value of the columns of two matrices, instead of the matrices as w whole. I use the following code for that:

for jj = 1:size(locname,1)

[errormatrix, pvaluematrix] = corrcoef(Hxb(:,jj),Hmeas(:,jj), 'rows','complete');

error.R(1,jj) = errormatrix(1,2);

error.P(1,jj) = pvaluematrix(1,2);

end

This works for the correlation coefficient, however, my pvaluematrix is only NaN. How do I solve this?

Answer 1

[Adam Danz]

First check whether you have NaNs in your inputs. If so, that explains why there are NaNs in your outputs.

If you do not have NaNs in your inputs, chec, that your inputs do not have identical values.

See this answer for details.

[Milva Bon]

I do have NaN in my input, but I thought the 'rows','complete' would take care of that.

[Adam Danz]

If you share the data so I an reproduce the results, I can dig deeper.

[Milva Bon]

here are thw two matrices. So I want to compare column 1 of hmeas to column 1 of hxb, and hten column 2 to column 2 ect.

[Adam Danz]

@Milva Bon, I was about to dig in but I see that another user has showed you why there are nans in the results. If you want more background information on this, see the link I provided in my answer and go to section "NaN values in the inputs spreading to the outputs".

[Milva Bon]

thank you!

Answer 2

[Voss]

Columns 5, 6 and 7 of Hmeas are all NaN, so even with 'rows','complete', there is nothing you can get but a NaN result.

load('hmeas.mat')
load('hxb.mat')

nanxb = sum(~isnan(Hxb),1);
nanmeas = sum(~isnan(Hmeas),1);

for jj = 1:size(Hxb,2)
    fprintf(1,'number of non-NaNs in column %d of Hxb:   %d\n',jj,nanxb(jj));
    fprintf(1,'number of non-NaNs in column %d of Hmeas: %d\n',jj,nanmeas(jj));
    [errormatrix, pvaluematrix] = corrcoef(Hxb(:,jj),Hmeas(:,jj), 'rows','complete')
%     error.R(1,jj) = errormatrix(1,2);
%     error.P(1,jj) = pvaluematrix(1,2);
    fprintf(1,' \n');
end

number of non-NaNs in column 1 of Hxb:   249

number of non-NaNs in column 1 of Hmeas: 242

errormatrix = 2×2

1.0000 0.9859 0.9859 1.0000

pvaluematrix = 2×2

1.0000 0.0000 0.0000 1.0000

number of non-NaNs in column 2 of Hxb:   249

number of non-NaNs in column 2 of Hmeas: 233

errormatrix = 2×2

1.0000 0.9480 0.9480 1.0000

pvaluematrix = 2×2

1.0000 0.0000 0.0000 1.0000

number of non-NaNs in column 3 of Hxb:   249

number of non-NaNs in column 3 of Hmeas: 226

errormatrix = 2×2

1.0000 0.8579 0.8579 1.0000

pvaluematrix = 2×2

1.0000 0.0000 0.0000 1.0000

number of non-NaNs in column 4 of Hxb:   249

number of non-NaNs in column 4 of Hmeas: 227

errormatrix = 2×2

1.0000 0.8670 0.8670 1.0000

pvaluematrix = 2×2

1.0000 0.0000 0.0000 1.0000

number of non-NaNs in column 5 of Hxb:   249

number of non-NaNs in column 5 of Hmeas: 0

errormatrix = 2×2

NaN NaN NaN NaN

pvaluematrix = 2×2

NaN NaN NaN NaN

number of non-NaNs in column 6 of Hxb:   249

number of non-NaNs in column 6 of Hmeas: 0

errormatrix = 2×2

NaN NaN NaN NaN

pvaluematrix = 2×2

NaN NaN NaN NaN

number of non-NaNs in column 7 of Hxb:   249

number of non-NaNs in column 7 of Hmeas: 0

errormatrix = 2×2

NaN NaN NaN NaN

pvaluematrix = 2×2

NaN NaN NaN NaN