Is it possible to detect and replace abnormal/wrong numbers in an array?

3 views (last 30 days)
Rikke on 20 Mar 2019
Commented: Chris Turnes on 21 Mar 2019
I have a dataset which is an array containing values, and which are plotted in the figure below. A section of the array for the figure is: A = [.... 25.9, 25.9, 26.2, 27, 28, 29, 29.3, 29.6, 3, 30.4, 30.5, 30.4, 30.3, 30.3, ....]; Here is number 3 an abnormality in the array. Is it possible to detect these abnormal numbers and replace them with the average value of the number before and after? So for the number 3 in A I want it to be (29.6+30.4)/2.

Kevin Phung on 20 Mar 2019
A = [25.9, 25.9, 26.2, 27, 28, 29, 29.3, 29.6, 3, 30.4, 30.5, 30.4, 30.3, 30.3]
abnorm =mean(A) - 3*std(A); % you can set however many standard deviations to constitute 'abnormal'
n = find(A<abnorm); %locations of abnormals
for i = 1:numel(n)
if or(n(i) == 1, n(i)==numel(A)) %if first or last index, replace with NaN;
A(n(i)) = nan;
else
A(n(i)) = (A(n(i) - 1) + A(n(i) + 1)) / 2;
end
end
A(isnan(A)) = []; %remove the endpoints that are NaN
let me know if this works for you
Kevin Phung on 20 Mar 2019
Edited: Kevin Phung on 20 Mar 2019
happy to help!
edit: if number of lines of code is important to you at all, you can actually condense the first 3 lines to one
n = find(isoutlier(A,'movmedian',5)); %locations of abnormals
for i = 1:numel(n)
if or(n(i) == 1, n(i)==numel(A)) %if first or last index, replace with NaN;
A(n(i)) = nan;
else
A(n(i)) = (A(n(i) - 1) + A(n(i) + 1)) / 2;
end
end
A(isnan(A)) = []; %remove the endpoints that are NaN
Rikke on 20 Mar 2019
Aha, perfect!

Walter Roberson on 20 Mar 2019
Rikke on 20 Mar 2019
Thanks!
Chris Turnes on 21 Mar 2019
You may also be interested in the filloutliers function, which lets you not only identify outliers but replace them as well.