Improving my forwardfill call

I want to clean missing data by using the previous available data to fill gap and only a given gap length 'n' can be filled.

E.g. I have the following vector:

A = [0.1 0.2 0.3 NaN NaN NaN 0.7 NaN 0.9 1];

If n = 1

A_ffilled = [0.1 0.2 0.3 0.3 NaN NaN 0.7 0.7 0.9 1];

If n = 2

A_ffilled = [0.1 0.2 0.3 0.3 0.3 NaN 0.7 0.7 0.9 1];

So, this is exactly what is done in the fillmissing help page example:

function y = forwardfill(xs,ts,tq,n)
y = NaN(1,numel(tq));
y(1:min(numel(tq),n)) = xs;
end

n = 2;
gapwindow = [10 0];
[F,TF] = fillmissing(A,@(xs,ts,tq) forwardfill(xs,ts,tq,n),gapwindow,SamplePoints=t);

So far so good. However, I am wondering if I am not missing something as what I am doing seems so basic. E.g. with python in pandas, one can simply use the .ffill() method and provides the desired value for limit. Therefore, I am wondering if there is nothing more simple that the example above if I do not what the apply a specific function but only to take the previous value.

Thanks in advance.

Answer 1

[Matt J]

A = [0.1 0.2 0.3 NaN NaN NaN 0.7 NaN 0.9 1]

A = 1×10

0.1000 0.2000 0.3000 NaN NaN NaN 0.7000 NaN 0.9000 1.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

A_ffilled = forwardfill(A,1)

A_ffilled = 1×10

0.1000 0.2000 0.3000 0.3000 NaN NaN 0.7000 0.7000 0.9000 1.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

A_ffilled = forwardfill(A,2)

A_ffilled = 1×10

0.1000 0.2000 0.3000 0.3000 0.3000 NaN 0.7000 0.7000 0.9000 1.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

EDIT:

function [B,I]=forwardfill(A,n)
 [I,I0]=deal( 1:numel(A) );
 I(ismissing(A))=nan;
 I=min(movmax(I,[n,0]) , I0); 
 
 B=A(I);
end

% function B=forwardfill(A,n)
% 
%  B=min(A,  movmax(A,[n,0]) );
% 
% end

[frlby]

Thanks, that's an interesting approach. However, it seems to me that it does not work if the data is not ordered :

A = [0.1 0.2 0.1 NaN NaN NaN 0.7 NaN 0.9 1];
A_ffilled = forwardfill(A,2)

A_ffilled = 1×10

0.1000 0.2000 0.1000 0.2000 0.1000 NaN 0.7000 0.7000 0.9000 1.0000

While it should be:

A_ffilled = 1×10

0.1000 0.2000 0.1000 0.1000 0.1000 NaN 0.7000 0.7000 0.9000 1.0000

Further, it needs to also works for string. I did not mention it initially because it naturally works with fillmissing, with just a tweak in the function:

function y = forwardfill_str(xs,ts,tq,n)
    y = repmat({''}, 1, numel(tq));
    if ~isempty(xs)
        y(1:min(numel(tq),n)) = xs;
    end
end

I would just use one or the other depending on my data type.

[Matt J]

Here's a version that fixes the ordering problem you mentioned, and also works for strings.

A = [0.1 0.2 0.1 NaN NaN NaN 0.7 NaN 0.9 1]

A = 1×10

0.1000 0.2000 0.1000 NaN NaN NaN 0.7000 NaN 0.9000 1.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

A_ffilled = forwardfill(A,2)

A_ffilled = 1×10

0.1000 0.2000 0.1000 0.1000 0.1000 NaN 0.7000 0.7000 0.9000 1.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

A_ffilled = forwardfill(string(A),2)

A_ffilled = 1×10 string array
    "0.1"    "0.2"    "0.1"    "0.1"    "0.1"    <missing>    "0.7"    "0.7"    "0.9"    "1"

function [B,I]=forwardfill(A,n)

 [I,I0]=deal( 1:numel(A) );

 I(ismissing(A))=nan;

 I=min(movmax(I,[n,0]) , I0); 
 
 B=A(I);

end