Piecewise differentiation, automatic identification of pieces

3 views (last 30 days)
I have a time series defined by gradual changes interrupted by large jumps; the time series also contains random noise. It also has irregular time steps. With my eye, I can separate the time series into uniform segments of the gradual changes and ignore the noise. I would like to automate this as I actually have several hundred such time series. Are there existing techniques to do this kind of task? Otherwise I will trial and error criteria to break the time series into segments, smooth down the noise, then differentiate. My attempts have not matched my eye's results so I want to make sure I am not missing any known approaches.
  1 Comment
KAE on 5 Mar 2021
Edited: KAE on 5 Mar 2021
In case it helps someone with a similar problem: if my signal was only gradual trend and noise, but no large jumps, a nice technique is Savitzky-Golay differentiation and there are many FEX contributions to implement it.

Sign in to comment.

Accepted Answer

Bjorn Gustavsson
Bjorn Gustavsson on 5 Mar 2021
I'd try with a lowess/loess filtering aproach, and then check the residuals between the data and the filtered/approximated values to find the transitions, the lowess should also give you a neatly differentiable curve for the gradual pieces. Maybe you'll get better results by repeating the process, first once to find the jumps and cut the data into smooth pieces, then to filter and estimate gradients.
See for example data-driven-fitting for inspiration...

More Answers (1)

Bruno Luong
Bruno Luong on 5 Mar 2021

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!