Find peaks in a specific portion of data?

Hello,

I have sets of data with a group of artifacts at the begging and end (see below - ignore the middle artifact):

I need to isolate the portions of the data with the start/end artifiacts as I'll be removing them. Right now I'm identifying the group of artifcats manually by literally zooming in on them and seeing exactly which point they start and end:

xlim_br_start = [1000 9000];
xlim_br_end = [320000 350000];

But because I have multiple sets of data like this that have these start/end artifacts but, with slightly varying start/end times, I would like to automate the process of isolating them so I can remove them.

Does someone know how to go about doing this? Any advice is appreciated.

Thanks in avance.

Answer 1

[Adam Danz]

Option 1: set threshold

If the outliers always have such an enormous amplitude relative to the signal, an easy approach would be to set a threshold value that is several times larger than the expected signal. Any points that exceed the threshold are identified as outliers.

% Create data
y = rand(1,10000) ./ 50; 
y(randi(200,1,30)+300) = rand(1,30)/2 + 0.5; 
x = linspace(0, 3.5, 10000); 
threshold = 0.2; 
% Plot data
figure
subplot(2,1,1)
plot(x,y, 'b-')
% Mark threshold
hold on
plot(xlim,threshold*[1,1], 'r-')
%identify outliers
isOut = y > threshold; 
% Plot data without outliers
subplot(2,1,2)
plot(x(~isOut), y(~isOut), 'b-')

Option 2: use isoutlier()

isoutlier(): https://www.mathworks.com/help/matlab/ref/isoutlier.html

This function identifies outliers. Pros: It's more flexible if the thresholding method doesn't work. Cons: It may require tweeking (see input options) and if the SNR is low, it may not be sufficient.

%Same as my example above except...
%identify outliers
isOut = isoutlier(y);

[Maria Y]

Thank you! This seems to be what I'm looking for, except my data is just a vector of signal data that I'm plotting so I don't really have an x and y to put in the last line. So how would I exclude the isOut if it's just a vector like I have?

I'm new to matlab so I'm sorry if that wasn't clear.

[Adam Danz]

Glad it helped! Just ignore my x vector.

If your signal data are stored in a vector named 'y' you can either permanently remove the outliers like this

y(isOut) = []; 

or replace them with NaN values (to preserve indexing)

y(isOut) = NaN; 

or you can just ignore them when plotting or doing subsequent calculations like this

plot(y(~isOut))
mean(y(~isout))