Vectorize averaging a timerange in a large timetable

I have a timetable of data and a counter of sample times.

When there is a non NaN "sample_times" variable, I want to average the previous four minutes of data.

currently, I loop through the times when there is a sample_times variable but this ends up taking longer than I would like.

Is there a way to do this without using a loop?

The current code using a for loop is below.

clc
load question_TT.mat
sample_times = ~isnan(question_tt.counter_array); % when is the counter a number?

sample_times = question_tt.Time(sample_times); % get times when a sample period begins

dt_1 = question_tt.Time(sample_times)-minutes(5);
dt_2 = question_tt.Time(sample_times)-minutes(1); % get beginning and end of averaging time

for n = 1:height(dt_1)
    S0(n,:) = mean(question_tt(dt_1(n):dt_2(n),1:4),"omitmissing"); % average the window of background time
end

[Dyuman Joshi]

Do you want to get the mean for all the times between dt_1(n) and dt_2(n)?

Also, the code is a fair bit fast here.

You should pre-allocate the output variable for increased code performance.

clc
load question_TT.mat

tic
sample_times = ~isnan(question_tt.counter_array); % when is the counter a number?

sample_times = question_tt.Time(sample_times); % get times when a sample period begins

dt_1 = question_tt.Time(sample_times)-minutes(5);
dt_2 = question_tt.Time(sample_times)-minutes(1); % get beginning and end of averaging time


for n = 1:height(dt_1)
    S0(n,:) = mean(question_tt(dt_1(n):dt_2(n),1:4),"omitmissing"); % average the window of background time
end
toc

Elapsed time is 0.115147 seconds.

[Poison Idea fan]

On this reduced size timetable it runs pretty well. My actual data table is quite large though. The elapsed time when I run it on the real data is >10 minutes.

Yes the goal is to average the data in that window.

Answer 1

[Taylor]

Preallocating your variables before a for loop is a great way to help with performance. arrayfun, cellfun, and structfun are also useful tools for vectorizing code. Try replacing the for loop with this:

window_indices = arrayfun(@(n) dt_1(n):dt_2(n), 1:height(dt_1), 'UniformOutput', false);
S0 = cellfun(@(indices) mean(question_tt(indices, 1:4), 'omitmissing'), window_indices, 'UniformOutput', false);
S0 = cat(1, S0{:});

[Poison Idea fan]

Using this method, there is a ~50 second improvement. But still takes a while. Maybe I am just asking for too much!

Thanks.