How can I smooth data around a specific region?

I have data. It is fairly noisy just as it is. But I only really want to smooth the oscillations. In the figure attached, the dashed red line is what I hope to achieve. I assume that there may be a need for a piecewise smoothing function. But I am not sure how to accomplish this. The data is also attached for anyone that wants to play. Thank you ahead of time for any advice.

Answer 1

[Image Analyst]

Try a median filter. It smooths out local oscillations while keeping true steps intact and steep:

s = load('matlab_question.mat');

s = struct with fields:
    time_: [2437×1 double]
     ids_: [2437×1 double]

time_ = s.time_;
ids_ = s.ids_;
plot(time_, ids_, 'b-', 'LineWidth', 2)
xlabel('Time');
ylabel('I')
grid on;
% Take median filter
smoothIds = medfilt1(ids_, 451);
hold on;
plot(time_, smoothIds, 'r--', 'LineWidth', 2)

[Steven Manz]

Never knew about this capability. Thank you so much!

[Image Analyst]

You could also use movmedian which is in base MATLAB in case you don't have the Signal Processing Toolbox (which has medfilt1):

s = load('matlab_question.mat')
time_ = s.time_;
ids_ = s.ids_;
plot(time_, ids_, 'b-', 'LineWidth', 2)
xlabel('Time');
ylabel('I')
grid on;
% Take median filter
smoothIds = movmedian(ids_, 451);
hold on;
plot(time_, smoothIds, 'r--', 'LineWidth', 2)

[Steven Manz]

@Image Analyst Referring to all other comments, your first one (medfilt1) is exactly what I needed. I needed oscilations removed. And medfilt1 did this perfectly.

Answer 2

[Joel Van Sickel]

Here is a simple script that gives you an easy way to implement filtering on specific pieces of data. You can accomplish this by writing something that goes inside a conditional expression looking at time. I'm just creating my own time and data vector as an example.

t = 0:0.1:1;
data = sin(t);
for i = 1:length(t)
    if t(i) > 0.5 && t(i) < 0.75 % only filter data for times between 0.5 and 0.75
        data(i) = (data(i-1)+data(i)+data(i+1))/3        
    end
end

now, that just averages the 3 points next to each other, which is a very small amount of filtering, but you can replace that line with a more sophisticated filter or averaging approach.

[Steven Manz]

As a more brute force method, this is a wonderful approach. Thank you!

[Image Analyst]

Or you could use movmean:

s = load('matlab_question.mat');
time_ = s.time_;
ids_ = s.ids_;
plot(time_, ids_, 'b-', 'LineWidth', 2)
xlabel('Time');
ylabel('I')
grid on;
% Take median filter
smoothIds = movmean(ids_, 201);
hold on;
plot(time_, smoothIds, 'r--', 'LineWidth', 2)

[Alex Sha]

Try a fitting function like below:

y=-1.04519296+(1/(1+exp(-(p21*(1/(1+exp(-(p6*(1/(1+exp(-(p1*(-1+2*(x-1.047808e-5)/7.7952e-7)+p24))))+p9*(1/(1+exp(-(p2*(-1+2*(x-1.047808e-5)/7.7952e-7)+p25))))+p12*(1/(1+exp(-(p3*(-1+2*(x-1.047808e-5)/7.7952e-7)+p26))))+p15*(1/(1+exp(-(p4*(-1+2*(x-1.047808e-5)/7.7952e-7)+p27))))+p18*(1/(1+exp(-(p5*(-1+2*(x-1.047808e-5)/7.7952e-7)+p28))))+p29))))+p22*(1/(1+exp(-(p7*(1/(1+exp(-(p1*(-1+2*(x-1.047808e-5)/7.7952e-7)+p24))))+p10*(1/(1+exp(-(p2*(-1+2*(x-1.047808e-5)/7.7952e-7)+p25))))+p13*(1/(1+exp(-(p3*(-1+2*(x-1.047808e-5)/7.7952e-7)+p26))))+p16*(1/(1+exp(-(p4*(-1+2*(x-1.047808e-5)/7.7952e-7)+p27))))+p19*(1/(1+exp(-(p5*(-1+2*(x-1.047808e-5)/7.7952e-7)+p28))))+p30))))+p23*(1/(1+exp(-(p8*(1/(1+exp(-(p1*(-1+2*(x-1.047808e-5)/7.7952e-7)+p24))))+p11*(1/(1+exp(-(p2*(-1+2*(x-1.047808e-5)/7.7952e-7)+p25))))+p14*(1/(1+exp(-(p3*(-1+2*(x-1.047808e-5)/7.7952e-7)+p26))))+p17*(1/(1+exp(-(p4*(-1+2*(x-1.047808e-5)/7.7952e-7)+p27))))+p20*(1/(1+exp(-(p5*(-1+2*(x-1.047808e-5)/7.7952e-7)+p28))))+p31))))+p32)))-0.1)*115.62874345

Sum Squared Error (SSE): 964.641639921989
Root of Mean Square Error (RMSE): 0.629151502109372
Correlation Coef. (R): 0.99983711862103
R-Square: 0.999674263772404

Parameter Name	Parameter Value
p1	76.4338521993061
p2	-113.215568747206
p3	18.1344896498787
p4	-6.31054275508802
p5	-8.64796259966414
p6	-44.3878849868977
p7	146.499448370489
p8	1.09313510000424
p9	2862262434.75547
p10	3.48445302484705
p11	-51.6709661320154
p12	44.7143171093621
p13	158.031873252179
p14	35.4382835361441
p15	82.7925117134803
p16	127.969217480314
p17	24.2118582072076
p18	852.333705669537
p19	0.2437035177173
p20	51.4328207881327
p21	0.500675788745402
p22	-29.4454763550884
p23	4.29032435859503
p24	26.1028273001908
p25	-31.1044326301403
p26	10.061593797352
p27	10.5707992246764
p28	0.980627900492599
p29	-83.4042390095949
p30	-435.873830153447
p31	-57.8935287049736
p32	-2.55896641004643

[Image Analyst]

@Alex Sha he said "I only really want to smooth the oscillations" and that formula has the oscillations still in there.