How to Extract ROI on a signal with additional rows before and after threshold

12 views (last 30 days)
I have data with the following details :
a) Three columns (Time, Signal 1, Signal 2) and 950000 rows.
b) My Threshold (Th) = 0.5
c) RT (Raising Threshold) = FT (Falling Threshold) = Th
I would like to extract the ROI between RT and FT for both signals simultaneously, but I also would like to include 300 rows Before RT (BRT) and 300 rows After FT (AFT).
Until now, I could extract only the values that are greater than my Threshold for S1 (Signal 1), but it was not enough, as it does not include S2 (Signal 2) and BRT and AFT either.
If someone could help, I would greatly appreciate it.
Thank you in advance.

Accepted Answer

dpb on 6 Aug 2022
Edited: dpb on 6 Aug 2022
It's gotten too convoluted -- yes, the original answer does, as noted in the last comment, only return the first/last -- your original post/illustration didn't mention that there are multiple it went on to say, one has two ways to then approach it; I took the latter here...
load A A
hL=plot(A(:,2:3)); xlim([0 1E4]); yline(th) % just to see what's we got...
tmp=int8([0; diff(A(:,2)>th)]); % the engine -- find the transition points
hold on
hT=plot(tmp); % show the flag variable, too...
ix1=find(tmp==1); % now find +ive transitions...
plot(ix1,th,'*r') % show them on the plot, too...
ix2=find(tmp==-1); % then the negative
ix1=ix1-PreTrigNo; % adjust to pre-, post-trigger ranges
S=arrayfun(@(i1,i2) A(i1:i2,:),ix1,ix2,'uniform',0); % and pick up the data in those sections
ROI=cellfun(@(c,i)i*ones(size(c,1),1),S,num2cell([1:numel(S)].'),'UniformOutput',0); % create ROI grouping variable
tS=array2table([cell2mat(ROI),cell2mat(S)],'VariableNames',{'ROI','Time','S1','S2'}); % and put into a table
The above results in
>> [head(tS);tail(tS)]
ans =
16×4 table
ROI Time S1 S2
___ ______ _________ ______
1 3.1488 0.010937 1.3819
1 3.1488 0.009375 1.3775
1 3.1488 0.032813 1.3813
1 3.1489 0.0125 1.3794
1 3.149 0.009375 1.375
1 3.149 0.00625 1.3725
1 3.149 0.0078125 1.375
1 3.1491 0.00625 1.3781
10 4.6906 0.0078125 1.2581
10 4.6907 0.010937 1.2628
10 4.6908 0.025 1.2631
10 4.6908 0.009375 1.265
10 4.6909 0.015625 1.2641
10 4.6909 0.010937 1.2691
10 4.6909 0.017188 1.2744
10 4.691 0.010937 1.2737
NB: However, the above also shows your time data don't contain enough significant digits to be unique; you'll need to either record those with more precision originally or regenerate new values knowing a fixed sample rate. They're perhaps immaterial, anyways...??? But, they're not useful as recorded here.
NB Second: It'll be far easier to handle using grouping variables and the splitapply work flow or rowfun and friends with the table rather than the suggested cell array horizontally
It's not a difficult task conceptually, one just has to approach it in logical manner thinking about how to identify the transitions above/below the threshold. Converting to a logical variable to make the values fixed and known to be 0|1 then lets you also know what the value of the difference is when the state changes, precisely. This is an easy case because the signals are so clean; you don't have to worry about noise or large real fluctuations that make the segmentation much more difficult -- the second channel would be more of a challenge, for example.
dpb on 7 Aug 2022
Edited: dpb on 7 Aug 2022
I have no way to comment/know anything at all about the data collection parameters since, to quote Sgt Schultz "I know NOTHINK!" about the process under study.
From the traces you've shown here the sampling rate of 20 kHz seems reasonable -- if you blow up the time axis to expand one transient, the rise and fall occurs within just 2-3 sample periods, but there are one or two samples in that time -- unless the fine structure inside the transient (which I presume must be an input of some sort to the DUT) is significant, you could probably safely cut the sampling rate in half or at least by 1/4 or 1/3 without sacrificing much since the response trace has a much slower frequency response.
The only real advantage to this would be the size of the output files but if your systems can handle the ones you're getting now, then it wouldn't seem to matter; you could always choose to decimate the files before processing, but you can't get back data you didn't collect in the first place.
If, indeed, the whole secondary transient until it reaches essentially steady-state isn't of real interest and your 'scope supports it, you could alternatively set it up to trigger on the transient with a set pretrigger hold time and also set a fixed capture/record time and essentially do the same thing as doing now except only collect the data around the impulses.
That all just depends on the capabilities of your instrumentation and how drastic/expensive the loss would be if you didn't capture the whole transient. Those are decisions that only you/your organization has the ability to make.

Sign in to comment.

More Answers (2)

dpb on 3 Aug 2022
Edited: dpb on 5 Aug 2022
Revised to match the figure and new understanding of intent to isolate one transient and select ROI around it -- the second trace is immaterial in setting the ROI limits; however, one will have to somehow ensure the ROI pre- and post-trigger intervals are wide enough to capture the entire ROI for the second trace as well -- in the later figure there appears to be a transient before S2 settles down to its final SS value that isn't captured in the shown ROI after S1 has returned to its baseline. This is a different kind of search than thresholding...
But, for the Q? as asked, it's pretty simple
tmp=int8(A(:,2)>Th); % convert to logical, save memory with int8
ix1=find(tmp,1)-PreTrigNo; % +ive crossing (first "1"); subtract pretrigger
ix2=find(tmp,1,'last')+PostTrigNo; % -ive crossing (+1, -1) add posttrigger
S=A(ix1:ix2,:); % select all the array between
clear tmp
To be completely robust, one should check that the result of the find operations are >=1 or <=size(A,1) BEFORE trying to address the S array with them; that's where the previous failed because there were no elements that matched for the second trace if it looked like the example figure.
dpb on 5 Aug 2022
Edited: dpb on 5 Aug 2022
I was very rushed the other day and didn't look at the figure carefully enough -- in fact, the second trace in the above doesn't cross the threshold and the ROI is based ONLY on the first signal -- I thought the intent/need was to isolate two transients at different triggering times -- remove the tests for the second channel entirely and all should be resolved.
I edited the Answer to match the problem...

Sign in to comment.

Mino.G on 5 Aug 2022
Edited: Mino.G on 5 Aug 2022
Thank you for your update and feedback.
You are right; the second trace in the above doesn't cross the threshold, and the ROI is based ONLY on the first signal.
But I need to Extract S2 the same as S1 only between the cursor A and B. The idea here is to use S1 as a reference to also extract S2 at the same time.
These two signals represent two different sensors that operate simultaneously. I want to apply this ROI to improve the classification of my machine learning model.
The signals you see vary from "Normal" to "Anomalous" at the end of the manufacturing process.
Thus, the idea is to clean up the noise, minimize the data point, and extract only the informative zone (in this case, the area between the cursor AB) and label them correctly.
In addition, the threshold represents the crucial section that my learning algorithm will be focused on. However, when the manufacturing process approaches the end /"Anomalous" additional fluctuations appear in the signals from both sides, which I believe need consideration and that I want to include in the ML learning process. It is the reason why I want to include additional rows/points from both sides, what you called them (PreTrigNo and PostTrigNo)
What I want to obtain as the final result or what I want S to look like is the following attached figure.
Mino.G on 6 Aug 2022
Edited: Mino.G on 7 Aug 2022
Thanks again, dpb for taking the time to help with this problem.
I don't have an error now, but the S output prints all without distinction.
In addition, I've tried to change the value of PreTrigNo and PostTrigNo to see if it is possible to extend or shrink the cursor area, but it seems working only on the left side.
As final result if S result is successful, I want to add a final line of code that built a vertical matrix where each row will hold only one ROI at a time (it will be very helpful for labeling and classification purposes)

Sign in to comment.





Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!