saving within SPMD or parfor

11 views (last 30 days)
Why matlab does not allow to save something to a file when within spmd or parfor?
let's say I have:
parfor i=1:n or spmd(nWorker)
VarA=some calculations
save(filename(i), 'VarA');
end
Matlab gives me an error "SAVE can not be called in an SPMD block. (the same for parfor except it says within parfor)
Does anyone knows why is that?
The funny thing is that if I create a new function like
function savetofile(data,fullfilename)
save(fullfilename,'data');
end
and then use this function, everything works fine. Well, of course, this is not recognized by matlab to flag an error and stop running the code. But I am trying to show that it means that the operation is perfectly natural and ok.
I have so many files, that some operation needs to be done, which each file could be processed completely independently. The output needs to be stored also in separate files. So, I want multiple worker, each opening one file at a time, and then storing their results once they are done, and then moving to next available file to process. But MATLAB somehow thinks it is not allowed to use save command within spmd and parfor and I need to trick it by kinda changing the function name. Why is that matlab thinks save shouldn't be used?
edit: writetable works just fine. It seems somehow it is save command that is not sitting well.

Accepted Answer

Edric Ellis
Edric Ellis on 8 May 2015
The problem with the save command is that it needs to inspect the contents of the workspace in which it is running. That's not allowed inside parfor and spmd. There's more on this constraint in the documentation. As you note, the constraint is about the body of the block itself, not a fundamental constraint on workers calling save or load, and that's because it's an analysis constraint.

More Answers (1)

Thomas Koelen
Thomas Koelen on 8 May 2015
Edited: Thomas Koelen on 8 May 2015
Transparency is violated by the SAVE command because in general MATLAB cannot determine which variables from the workspace will be saved to a file.
The solution is to move the call to SAVE to a separate function and to call that function from inside the PARFOR loop. Pass any variables that are to be saved as arguments to the function. That way MATLAB can determine which ones will be saved and transparency is not violated.

Categories

Find more on Startup and Shutdown in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!