Saving and resuming the parfor states

12 views (last 30 days)
Is it possible to save the states of parfor loop, stop/break and restart from the same state. For example, in may case some time Windows crashes and I can not run the simulaiton from the same state, I have to run from initial level. In a simple for loop, I can save the variable i and ourval then I can run the for loop. In the code shown below, I have only variable i chaning in the parfor. Can I save parfor states after some interval and after restarting Matlab I can have all the variables in the wokspace resume parfor from same states.
% cubeL=51;
% XYZw=[90 100 108];
% P_labs, xyz nx3 matrix
% some_function is a function
ourval=zeros(size(xyz));
parfor i=1:cubeL*cubeL*cubeL
ourval(i,:)= some_function(xyz(i,:),P_labs,rgb,XYZw);
endd

Accepted Answer

Nicolas B.
Nicolas B. on 26 Aug 2019
Hi Imran,
I would suggest 2 solutions:
  1. at each iteration of your parfor loop, you log your progress into a file and then, if MATLAB or Windows crashes, you read that file.
  2. you use the DataQueue class to send back data to your main MATLAB process and you log the information in a file (see this documentation).
Does it helps you?
  2 Comments
Abdulllah
Abdulllah on 27 Aug 2019
Edited: Abdulllah on 27 Aug 2019
HI Nicolas,
Thank you for reply and time.
  1. Using the first approach
I can not use the first approach becasue, I also have to save "ourval" variable. As we can not save values within a parfor so I tried saving within a function. For this purpose, I had to pass the varible "ourval" to the function. parfor does not allow to pass the this varible.
isave=zeros(1,cubeL*cubeL*cubeL);
parfor i=1:cubeL*cubeL*cubeL
ourval(i,:)= some_function(xyz(i,:),P_labs,rgb,XYZw,isave,i,cubeL,method,ourval);
end
porfor error: can not run parfor due the due to the way the variable ourval is used. in "some_function" I check if the varialbe "i" is already present in the "isave" then I return otherwise I do the task and added the "i" to "isave" to save the "isave" and "ourval". This was good idea but I can not send "ourval" within parfor.
function RGB= some_function(XYZ,P_labs,rgb,XYZw,isave,i,cubeL,method,ourval)
if ismember(isave,i)
return;
end
%;
%;actual taks
%;
if (i/100==0)
save('sony_xyzdatai_51crm.mat','ourval','XYZw','cubeL','method') %save invserse data file
isave(i)=i;
save isave
end
end
2. Using the second approach
I am not yet able to do the second approach. Is it possible that you guide further
q = parallel.pool.DataQueue;
afterEach(q, @disp);
isave=zeros(1,cubeL*cubeL*cubeL);
parfor i=1:cubeL*cubeL*cubeL
ourval(i,:)= some_function(xyz(i,:),P_labs,rgb,XYZw,isave,i,cubeL,method,ourval);
isave(i)=i;
send(q, i,ourval,isave);
end
send(q, 0);
Thank you for your time.
Nicolas B.
Nicolas B. on 27 Aug 2019
Hi,
I haven't much time so I haven't tested your code. However, when calling a parfor loop, you must already have created the data at the right size (no resizing in parfor for common data). So try to create ourval before parfor.
For the code 2, try to write send(q, num2str(i)) in the loop instead of what you have. If what I suspect is right, I think that it should call disp(i) at the end of each loop.
I let you try. If I find time this afternoon, I will try to run your code.
Regards

Sign in to comment.

More Answers (0)

Categories

Find more on Loops and Conditional Statements in Help Center and File Exchange

Products


Release

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!