Parpool in Pattern search with temp directories

I am currently trying to parpoll in pattern search. The problem is that my objective function is editing a txt file and runs a simulation. I have tried to make a separate directory for each worker but it doesnt seem to work as expected. this is my code right now.

delete(gcp('nocreate'));  % Clean up any existing pool
pool = parpool;  % Start parallel pool
numWorkers = pool.NumWorkers;  % Get the number of parallel workers
% Define bounds
lb = [0; 0; 0.5; 2/3; 0.8; 0.75];
ub = [1.3; 4; 2; 4/3; 1.1; 1.1];
% Define initial guess for each worker
x0_all = repmat([1 1 1 1 1 1], numWorkers, 1);  % Same x0 for all workers initially
% Perturb each worker's x0 slightly to avoid conflict
rng('shuffle');  % Set random seed based on the current time
for i = 1:numWorkers
    x0_all(i,:) = x0_all(i,:) + 0.1 * randn(1, 6);  % Perturb by small random values
end
% Path to bin folder (adjust this path if necessary)
bin_folder = fullfile(pwd, 'bin');
% Parallel execution
parfor i = 1:numWorkers  % Iterate over workers
    workerID = i;
    
    % Create unique temporary directory for each worker
    tmpdir = fullfile(pwd, sprintf('tmpdir%d', workerID));  % Unique temp folder
    mkdir(tmpdir);  % Create directory for worker
    % Copy the 'bin' folder into the worker's temporary directory
    copyfile('bin', tmpdir);
    % Change the current directory to the worker's temporary directory
    cd(tmpdir);
    
    % Get the initial guess for the worker
    x0 = x0_all(workerID, :);  % Get the initial guess for this worker
    
    try
        % Run pattern search optimization with the unique initial guess
        options = optimoptions('patternsearch', ...
            'UseParallel', true, ...      % Enable parallel execution
            'PlotFcn', @psplotbestf);     % Plot best function value
        [x_opt, fval] = patternsearch(@Motionview, x0, [], [], [], [], lb, ub, [], options);
        % Display results
        fprintf('Worker %d - Optimized variables: %s\n', workerID, mat2str(x_opt));
        fprintf('Worker %d - Optimal function value: %.6f\n', workerID, fval);
        
    catch ME
        fprintf('Worker %d encountered an error: %s\n', workerID, ME.message);
    end
    
    % Clean up: Remove the worker directory after the computation
    rmdir(workDir, 's');  % Delete worker's folder after work is done
end

The process continous but i get these warnings:Warning: Unable to create a parallel pool. Continuing with evaluations in serial. To avoid this warning, set the 'UseParallel' option to false.

> In setOptimFcnHandleOnWorkers (line 98)

In patternsearch (line 278)

Note that the Motionview objective function, need the necessary file which are located in bin folder, thats why its getting copied.

[Walter Roberson]

cd(tmpdir);

If you must cd(), then cd() back again later, especially since you are relying on pwd

Answer 1

[Matt J]

The process continous but i get these warnings:Warning: Unable to create a parallel pool. Continuing with evaluations in serial. To avoid this warning, set the 'UseParallel' option to false.

You should heed the warning and set UseParallel to false.

I have tried to make a separate directory for each worker but it doesnt seem to work as expected.

We don't know what "as expected" means here. Are the directories not getting created? Are the workers overwriting each other? If the processing is serial as the warning claims, that might explain it.

[Pantelis]

Thank you for your answer. I fixed the code and now each worker (when assigned) creates his own directory so there are no conflicts.

Have a nice day.

[Matt J]

That is interesting. Was it fixed by setting UseParallel=true?

[Pantelis]

No, I approached it differently. I implemented a piece of code within my objective function that detected which worker was involved in the process and automatically created a directory named after that worker. Inside each directory, all the necessary files were stored. This way, each worker had their own dedicated folder, avoiding any confusion and ensuring proper file organization.