How can I speed-up 3 functions without terminating workers?

1 view (last 30 days)
ahmed gamal
ahmed gamal on 27 Oct 2020
Commented: Thomas Falch on 30 Oct 2020
I have four different functions in which each function runs three superimposed for-loops with 3D arrays, (3 are in dependent and 1 is dependent on those 3 independent functions)
for i=1:nx
for j=1:ny
for k=1:nz
statement or my calculations
All of these loops are just a part of two while loops: 1) inner while loop, and 2) outer while loop. The previous two loops are dependent loops, which means i can't use parallel computing functions for them.
What is the best way to run them all in parallel with the minimum computational cost as this process is a very time consuming and computationally expensive process. I'm wondering, if i can run each function on a worker without terminating the worker everytime i enter the next iteration for the inner while loop!! as terminating and executing MATLAB worker takes few seconds, like 30-60 seconds, which means every inner while loop might take more than 2 minutes to be calculated and it is not the case which i'm looking for. I'm just trying to speed-up the calculations which takes about 60-90 seconds for each inner loop, if you know what i mean.

Answers (1)

Thomas Falch
Thomas Falch on 29 Oct 2020
If you use a parpool you can start up the MATLAB workers once, and then use them to execute e.g. parfor loops, or execute functions in parallel with parfeval, without having to pay for the cost of starting up MATLAB.
For example:
% This starts up 4 MATLAB workers and will take 10s of seconds
parpool('local', 4)
% Run the function pause(1) on a worker, the worker is already running, this wil take 1 second
wait(parfeval(@() pause(1), 0));
The pool remains open (with the workers running idle in the background) until you close it.
In your case, you may also consider using parfor loops. These can often be used to parallelize for loops with parpools (with some restrictions)
Thomas Falch
Thomas Falch on 30 Oct 2020
Essentially, parfor loops require that the iterations are independent, that is, no iteration should depend on the result of another iteration (if it did, those two iterations could not be executed in parallel at the same time).
If you have the Parallel Computing Toolbox installed, you can just replace for with parfor, if the iterations are not independent, it will error.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!