# How can I use detrend in a matrix with NaN?

49 views (last 30 days)
Sara Antonio on 4 May 2016
Answered: Naomi Krauzig on 14 May 2019
Hello,
So, I have a matrix 5551*1601 (time*distance) containing numbers and NAN’s. Some rows are just with NaN’s. I want to use linear detrend for every row. Apparently the detrend function does not work with matrix containing NaN. To solve this problem, I tried it to make it step by step using polyfit function for a linear regression. Then just remove the linear regression from my data. Here is the code I used:
% % Applying linear detrend for each barline
for i = 1:numel(E_blimline_1_m(:,1));
% Creates a matrix time as x in polyfit funtion:
E_along_M(i,:) = -2000:2.5:2000;
% Determines the nan position:
E_Bl1_nan = ~isnan(E_blimline_1_m(i,:));
% Execute polyfit function with NaN's for linear trend (y = b*x+m) giving b and m
E_Bl1_polyfit(i,:) = polyfit(E_along_M(i,E_Bl1_nan),E_blimline_1_m(i,E_Bl1_nan),1);
% Attributes values for y of linear equation
E_Bl1_polyval(i,:) = polyval(E_Bl1_polyfit(i,:),E_along_M(i,:));
end
% Create a matrix with detrend lines by removing the trend from the original values
E_Bl1_detrend = bsxfun(@minus,E_blimline_1_m, E_Bl1_polyval);
With this code it seems to give the correct result. However, because I have some rows only with NaN’s, while running it constantly appear this message when using the polyfit function: Warning: Polynomial is not unique; degree >= number of data points.
So my question is, there is some way avoid that message?
And, there is a simplest way of doing a linear detrend on a matrix with NaN’s without using a loop?
Sara

Naomi Krauzig on 14 May 2019
Hi Sara,
here is an available function for detrending with NaNs.
function detrended = detrendNaN3(A,t)
%DETRENDNAN3 Detrends a matrix with (or without) NaNs into the third dimension using linear least squares.
%
% Input Arguments:
% - A: NxMxK matrix (double)
% - t: Optional 1xK time vector (double) indicating the measurement
% time of each slice. If not given, the slices are assumed
% to be evenly spaced.
%
%% Default time and data formatting
% create default t if not given
if nargin<2
t = 1:size(A,3);
end
% time to same format as A and same NaN positions
t = bsxfun(@times,permute(t,[3 1 2]),ones(size(A)));
t(isnan(A)) = NaN;
%% Calculation
% mean of time
xm = nanmean(t,3);
% mean of A
ym = nanmean(A,3);
% calculate slope using least squares
a = nansum(bsxfun(@times,bsxfun(@minus,t,xm),bsxfun(@minus,A,ym)),3)./nansum(bsxfun(@minus,t,xm).^2,3);
% calculate intercept
b = ym - a.*xm;
% calculate trend
trend = bsxfun(@plus,b,bsxfun(@times,a,t));
% remove trend
detrended = A-trend;
end
Hope it helps,
Naomi