fillmissing function with makima method

Hi!

I'm trying to fill the missing data with fillmissing function and makima method for gaps that are less than 12000 samples. This is the code I used:

trial8 = fillmissing(trial8,'makima','MaxGap',12000); 

It removes NaNs, but for some reason gives weird and wrong negative values. I'm attaching the screens, first is my data before filling and the second after filling. Could somebody tell me what do I do wrong?

[KSSV]

Attach your data and full code.

[Matt J]

In what way are the filled values "wrong"? Your original data has a very sharp downward drop.The fillmissing operation seems to extend ithe drop very smoothly.

[katie_km]

@Matt J The point of using the function was getting rid of the drop, not extending it.

[Steven Lord]

Is that drop in some sense "real" or is it an artifact of some issue with the data gathering process? Do you really want to use fillmissing here or would either rmmissing or rmoutliers be a better match for what you're hoping to do with this data?

If I was collecting data from a thermometer and it accidentally kept recording once I took it from under my tongue and put it back in its case, that data shouldn't be filled it should be removed entirely.

Answer 1

[Star Strider]

Thje 'makima' method is likely best for filling NaN gaps in varying data. Yours appear to be relatively constant, other than for that one instance.

See if using 'nearest' instead will improve the result. .

[katie_km]

Hi, @Star Strider! Thank you for your suggestion! Using 'nearest' worked a little bit better, the drop didn't go to the negative values, but it is still there. Do you have any other suggestions on how to get rid of it?

[Star Strider]

As always, my pleasure!

I’m not certain what your data are, so I’m not certain how to fill the gap (if you want to completely get rid of the ‘dip’ in the data). One option might be to find the value of the ends of the NaN data (this whould be relatively straightforward), then fill it —

x = 0:16;
y = 5+rand(size(x));
y([12 15]) = rand(2,1);
y(13:14) = NaN;

figure
plot(x, y)
ylim([0 6])

nanidx = find(isnan(y))                                     % Indices Of 'NaN' Values

nanidx = 1×2

13 14

to_fill = nanidx + [-2 2]                                   % Extend To Reliable Data (This May Require Experimentation)

to_fill = 1×2

11 16

to_fill_idx = to_fill(1) : to_fill(2)                       % Create Continuous Index Range

to_fill_idx = 1×6

11 12 13 14 15 16

gap = interp1(x(to_fill), y(to_fill), x(to_fill_idx))       % Interpolate (Using 'linear' Here)

gap = 1×6

5.2339 5.3644 5.4949 5.6254 5.7559 5.8864

y(to_fill_idx) = gap;                                       % Assign 'y' Range

figure
plot(x, y)                                                  % Plot Result
ylim([0 6])

An approach similar to this would likely work with your data. There might be easier ways to accomplish this, however none come to mind just now.

.

[katie_km]

@Star Strider, thank you, it worked! :D

[Star Strider]

As always, my pleasure!