Wishing to remove invalid noisy data from a graph.
39 views (last 30 days)
Show older comments
Joshua Wakefield-Luno
on 24 May 2023
Edited: Mathieu NOE
on 24 May 2023
I have plotted a graph of some Tempertaure vs Time data.
I have used 'hampel' and 'medfilt1' to minimise and reduce noise that occurs, while most have been cleaned up there is a period of continious noise I am wanting to remove or mitigate so it doesn't make other data hard to read.
Looking for suggestions to remove said continuous noise besides manually adding in an x-axis cut off point as this experiment will be further continued with multiple results and could become tedious.
0 Comments
Accepted Answer
Mathieu NOE
on 24 May 2023
Edited: Mathieu NOE
on 24 May 2023
My first idea would be to analyse the ratio of variance vs mean value of a buffer of samples (repeat if until the end of the data) , so the noisy sections should show clearly a higher ratio vs a clean one
then replace this secton of data with a smoothed / filtered version of it (or replace with NaN's)
after some trials , maybe this code is a good starting point :
% dummy data
n=1000;
x=(0:n-1)/n;
ynf = 0.3+ max(0,sin(12*x+5*x.^2)).*exp(-3*x); % signal without noise
% add local noise
y = ynf;
l = 300;
y(l:n) = y(l:n) + 0.05*randn(1,1000-l+1); % add some noise in the trailling portion
%% main code
y_out = y;
% create the low pass filtered / smoothed version of y
k = 60;
ys = smoothdata(y,'movmean',k); % smoothed version of y
% let's detect when we have too much noise content and replace y with ys in
% those sections
[B,A] = butter(2,0.03,'high');
yhp = filtfilt(B,A,y);
% logic signal
ls = abs(yhp);
k = 50;
ls = movmean(ls.^2,k);
ls = ls./max(ls);
idx = (ls>0.1);
y_out(idx) = ys(idx);
figure(1)
plot(x,y,'b',x,y_out,'r',x,ynf,'g--')
legend('noisy signal','denoised signal','noise free signal');
0 Comments
More Answers (0)
See Also
Categories
Find more on Filter Analysis in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!