Code that processes a Byte structure in parfeval crashes the parallell workers

I have a GUI that connects to an external camera and processes frames in real time.

The camera produces a stream of objects that are defined by an external SDK that relies on a variety of .NET libraries

Before starting the GUI, I run

p = parpool(2)

I wait for this to load, and then start the GUI

I have a listener that responds to each new frame from the camera

In that listener I call:

handles.f = parfeval(@analyzeFrameParallel, 0, img, handles.D);

img is a Byte structure, which is a .NET object generated by the Camera SDK, handles.D is the DataQueue, which has an afterEach associated with it to display the processed image

Now in the CLIENT, I can run

X = uint8(img)

And it generates a matlab array of 0-255 values that can be sent to imshow to display the image

But, in analyzeFrameParallel:

function analyzeFrameParallel(img, D)
    X = uint8(img);
    send(D, X);
end

causes the workers to crash. The crash dump file has a stack trace with first element:

"C:/Wndows\System32\KERNELBASE.dll+00239673 RaiseException+00000105"

Could it be that the parallel workers can't process .NET objects the same way that the CLIENT can? I tried running

pctRunAll NET.addAssembly("mscorlib.dll")

after recapturing the data, but it still crashes

Are there .NET limitations to parallel computing toolbox?

Thanks to anyone out there with idea!

Answer 1

[Edric Ellis]

I would suggest that you need to run whatever code you're currently using to initialize img on the desktop client, you instead run on the worker, and use it to build a parallel.pool.Constant. Perhaps something a little bit like this:

imgC = parallel.pool.Constant(@buildImg, @closeImg);

%% Here's the functions needed to build / destroy img
function img = buildImg()
NET.addAssembly("mscorlib.dll");
img = <build img>;
end
function closeImg(img)
<do whatever's needed to clean up img>;
end

%% Then, use the Constant like this:
f = parfeval(@doStuff, 0, imgC, myQ);

%% with
function doStuff(imgC, q)
img = imgC.Value; % Extract the actual 'img' object
send(q, ...); % send stuff back down the data queue.
end

[Leonid Shmuylovich]

Thanks for the response, I will try it out and get back to you.

Can you confirm though that the worker does not have mscorlib.dll or other dependencies included by default? Did you include that because I had it in my sample code, or because in fact it's not there as default? I'm just trying to understand what (if any) .NET compatibility the parallel workers have.

[Edric Ellis]

Workers are separate MATLAB processes, and they pick up only a few things automatically from the client (like the MATLAB path, and the current working directory). If your client MATLAB process needs the NET.addAssembly stage, then the workers will need it too.