Smoothing a noisy signal

I have a large data set that produces a wide range of values and want to smooth it to follow the curve of the orange line in the graph. I've tried use the sgolay function but it seems to get really strange results!

Please help!

[Dyuman Joshi]

Please attach your data using the paperclip icon.

[Damaris Litton]

@Dyuman Joshi I have attached the data now! I want 'pressure' to look more similar to the 'measured' curve. Thank you

Answer 1

[Cris LaPierre]

I haven't looked at your data or tested anything, but thought you might be interested in the Smooth Noisy Data task, available in the Live Editor. This adds a user interface to the live script, in which you can interactively adjust your smoothing parameters and inspect the results. This might make it easier to find the settings that give you the desired results.

[Damaris Litton]

Thank you for this! When I try it using the sgolay setting, the input data and the smooth data remain the same no matter what I change? Can you help me on this?

Thanks

[Cris LaPierre]

I don't know enough about your data to be able to recommend a solution. My first impression is that the measured data trend does not appear to follow the pressure data, at least not obviously. I also suspect the spikes are not real data, but more an artifact of how the data was collected, so I might start by first cleaning the data.

Answer 2

[Image Analyst]

I don't see how the orange curve follows any of your data. It doesn't seem to follow the spikes or even the non-spikes. What part of the signal is noise and what is not noise?

If you want to get rid of the spikes, do this:

badIndexes = abs(signal) > 0.3;
signal(badIndexes) = [];

If you want to get rid of the non-spikes, do this:

badIndexes = abs(signal) < 0.3;
signal(badIndexes) = [];

Answer 3

[Star Strider]

Experiment with this approach —

LD = load(websave('pressure','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1309950/pressure.mat'));
dP = LD.dP;
L = size(dP,1)

L = 2633

Fs = 1;                                                     % Sampling Frequency (Use Corredt Value)
Fn = Fs/2;                                                  % Nyquist Frequency
t = linspace(0, L-1, L)/Fs;                                 % Time Vectror
NrNans = nnz(isnan(dP))                                     % Missing Values

NrNans = 95

dP_edit = fillmissing([t(:),dP],'linear');                  % Interpolate Missing Values

te = dP_edit(:,1);                                          % Edited Time Vector
dPe = dP_edit(:,2);                                         % Edited Signal Vector

figure
plot(te, dPe)
grid
xlabel('Time')
ylabel('Amplitude')

dPe = filloutliers(dPe, 'linear','grubbs');                 % Interpolate Outliers

NrFrames = 15;                                              % Choose Number Of Frames In Signal
framlen = fix(L/NrFrames);
if rem(framlen,2) == 0
    framlen = framlen+1;
end
dPe_filt = sgolayfilt(dPe, 3, framlen);                     % Filter

figure
yyaxis left
plot(te,dPe)
ylim([-max(ylim) max(ylim)])
ylabel('Amplitude Of Edited & Trimmed Original Signal')
yyaxis right
plot(te, dPe_filt)
ylim([-max(ylim) max(ylim)])
ylabel('Amplitude Of Edited & Trimmed Filtered Signal')
grid

figure
plot(te,dPe)
hold on
plot(te, dPe_filt)
hold off
grid
xlabel('Time')
ylabel('Amplitude')
ylim([-1 1]*1E+3)

This is an extremely difficult signal to process, and requires sevral steps prior to filtering.

.