Iterate to filter signal in a parfor

Hi everyone

I am using MATLAB 2015 and I am using a student version with all toolboxes installed.

I have a third-dimensional order tensor containing the data coming from a thermographic measurement (in the form of pixel-pixel-time) and I want to apply a custom made low pass filter which I have created with the matlab filter designer. I want filter every signal (so the signal corresponding to every pixel in time) and I want to use a parfor.

In one .m code I am defining the following command:

test_hyper_cube = zeros(512,640,3000)
hyper_cube_filtered_lowpass = lowPassHyperCube(test_hyper_cube,LP); %% LP is a filter object

%% Then I define the following code in another external function that should process each signal from each pixel in time. The function receives 2 inputs: inputdata which is a 3rd order tensor and filterVar which is filter object

function [outputData] = lowPassHyperCube(inputData,filterVar)
outputData = [];
parfor i=1:size(inputData,1)
    tempvar = [];
    for j=1:size(inputData,2)
        newvar = filter(filterVar,squeeze(inputData(i,j,:))-mean(squeeze(inputData(i,j,:))))+mean(squeeze(inputData(i,j,:)));
        newvar = reshape(newvar,1,[]);
        tempvar = [tempvar;newvar];
    end
    outputData(i,:,:) = tempvar;
    disp(['Iteration n.', num2str(i)])
end
end

But if I run it I get the following error:

Error using lowPassHyperCube>(parfor body) (line 9)
An UndefinedFunction error was thrown on the workers for 'filter'.  This might be because the file containing 'filter' is not
accessible on the workers.  Use addAttachedFiles(pool, files) to specify the required files to be attached.  See the documentation
for 'parallel.Pool/addAttachedFiles' for more details.
Error in lowPassHyperCube (line 6)
parfor i=1:size(inputData,1)
Caused by:
    Undefined function 'filter' for input arguments of type 'struct'.

Any idea why?

Let me know you can try this out. Attached you have the mat file containing the filter object I have created but you shouldn't need any specific data, you can just try with fake ones.

Greetings

Luca

[Ridwan Alam]

Hi Luca, does the response below work?

[Luca Pecoriello]

Hi.

Sorry for my late reply, but I was doing some more in-depth tests.

I am still not 100% sure. I have noticed that the computation gets stuck at some point. Maybe you can help me out understanding some points.

Here is the code, following the integration of your suggestions. Because in a parfor it is not possible to use the "clear" command to clear a variable, I am freeing some memory after each iteration with the following lines

inputVector = [];

and

newvar = [];

This is the following code. Maybe can you try it and tell me what could be eventually improved?

function [outputData] = lowPassHyperCube(inputData,filterObj)

tic
outputData = [];
filterVar = filterObj.Numerator;
disp('Preparation for filtering data with parallelized loop');

parfor i=1:size(inputData,1)
    
    tempvar = [];
    disp(['Filtering data at outer loop step: ', num2str(i)]);
    
    for j=1:size(inputData,2)
        
        inputVector = squeeze(inputData(i,j,:));        
        newvar = filter(filterVar,1,inputVector-mean(inputVector))+mean(inputVector);
        newvar = reshape(newvar,1,[]);
        tempvar = [tempvar;newvar];
        disp(['Filtering data at internal loop step: ', num2str(j)]);
        inputVector = [];
        newvar = [];
        
    end
    outputData(i,:,:) = tempvar;
    tempvar = []; %% In a parfor the variable cannot be cleared
    disp(['Filtering iteration n.', num2str(i)])
    
end

toc
disp('EOP. Data have been filtered');

end

For your information, this is the PC I am using. I am actually surprised the code gets stuck, seen the powerful hardware I have

• Name: ASUS DESKTOP PC G11CB_G11CD Series. DESKTOP-I3TV9GG
• Processor: Intel(R) Core(TM) i7-6700 CPU @ 3.40 GHz - 3.40 GHz
• RAM 32.0 GByte

Greetings

Luca

[Ridwan Alam]

Luca, I don't think it has much to do with freeing up those temp memory. You can certainly try to not use newvar, inputVector at all. I used those only for ease of reading.

Maybe you can try to upgrade the Matlab version. Earlier versions needed something called matlabpool. I am not sure you need that or are already using that.

Answer 1

[Ridwan Alam]

The error is caused by the way global digital filter objects are handled by filter() in parfor loop.

function [outputData] = lowPassHyperCube(inputData,filterObj)
    outputData = [];
    filterVar = filterObj.Numerator;
     
    parfor i=1:size(inputData,1)
        tempvar = [];
        for j=1:size(inputData,2)
            inputVector = squeeze(inputData(i,j,:));
            
            newvar = filter(filterVar,1,inputVector-mean(inputVector))+mean(inputVector);
            newvar = reshape(newvar,1,[]);
            tempvar = [tempvar;newvar];
            
        end
        outputData(i,:,:) = tempvar;
        disp(['Iteration n.', num2str(i)])
    end
end