How to pass structure data into parfor loop

Hi all,

I have a simple parfor test code like this:

inpt.a1 = rand(100);

inpt.a2 = rand(100);

parfor i = 1:10000

      inpt_pass = inpt;
      otpt = sin(inpt_pass.a1);

end

I wrote inpt_pass in order to pass the structure inpt into the parfor loop. However, if inpt is a very large structure, passing it into every parfor loop would be cumbersome, and cost loads of storage.

If I do:

inpt.a1 = rand(100);

inpt.a2 = rand(100);

parfor i = 1:10000

      inpt_pass = inpt.a1;
      otpt = sin(inpt_pass.a1);

end

which only pass part of the structure into parfor loop (ideal), I got the broadcast variable warning.

So how can I do it? Many thanks!

[corentin]

I am in a similar situation but I would like to pass to each worker a different part of the struct -- i.e. the content of a given field and only that content (not the whole struct). Following the example above, I'd like to write something like:

inpt.a1 = rand(100);
inpt.a2 = rand(100);
% ...
inpt.aN = rand(100);
parfor i = 1:N
      fieldname = sprintf('a%i',i);
      inpt_pass = inpt.(fieldname);
      otpt = sin(inpt_pass);
end

Is it true that the entire struct is not passed into the parfor loop in that case, thus saving memory? If the answer is negative, what alternatives do I have?

Answer 1

[Brendan Hamm]

You can simply extract the data that you want to pass to the workers:

inpt.a1 = rand(100);
inpt.a2 = rand(100);

A1 = inpt.a1; % Only inpt.a1 is sent to the workers

parfor i = 1:10000
      otpt = sin(A1);
end

Now there is no extra overhead of transferring data. This of course assumes a 1-by-1 vector. If you have more elements in your struct you could just make a copy of only the data that needs to be sent:

inpt.a1 = rand(100);
inpt.a2 = rand(100);

tempVar.a1 = inpt.a1;

parfor i = 1:10000
     % You will still have a warning, but you can ignore it as you only pass the data you need.
      otpt = sin(tempVar.a1); %#ok<PFBNS>     <-- Ignore the warning
end

If you get an error about how MATLAB cannot classify the variable (common with structs), place the body of the parfor loop into a local function and this can overcome this issue easily.

Answer 2

[Sean de Wolski]

Use a cell array instead of a structure - they can be easily sliced by index. Or, less preferably, in newer releases turn the structure into a parallel.pool.Constant so that it persists on the workers.