error when running ray to parallelise a simple matlab function wrapped as python application
1 view (last 30 days)
Show older comments
I would like to parallelise a matlab function using python and ray.
In this example
- I used MathWorks to wrap a very simple matlab function as Python application (the function just returns as result the input integer - see below)
- The application (the function) is then called in python (see below). The function is called using the ray package to parallelise the call.
- The function is called four times using as inputs some integers: 0, 1, 2 and 3. The result should be a list: [0, 1, 2, 3]
- The call fails with error: ray/cloudpickle/cloudpickle_fast.py, TypeError: 'str' object is not callable
- The same function (test_function_for_python_1) works well in a simple for loop and returns [0, 1, 2, 3], so the problem is not in the MathWorks wrapping of the matlab function into python.
In short, ray needs to pickle the matlab/python application, and the pickling encounters a problem I can't solve. Any help would be much appreciated.
The operating system is MacOSx Catalina, the python (and mwpython) version is 3.7.9, the ray python module version is 1.0.0
== Matlab function ==============================================
function result = test_function_for_python_2(an_integer_input)
result = an_integer_input;
end
== Python script that parallelises the matlab function ========================
import TestTwo # a class with a simple function returning the input integer
import matlab
import ray # for parallelisation
# Initialise ========================================================
ray.shutdown() # shutdown any open CPU pool
ray.init() # create a pool of parallel threads
# initialise the matlab app containing the code of a simple test function (returns the same integer that is input)
my_test_function = TestTwo.initialize()
print(type(my_test_function))
print(type(my_test_function.test_function_for_python_2))
@ray.remote
def function_two(an_integer):
'''Here a single string is input and returned.
The function responsible for it has been written in Matlab and wrapped as a python application in Mathworks.'''
result = my_test_function.test_function_for_python_2(an_integer, 1) #, nargout=1)
return result
some_integers = [0, 1, 2, 3]
# run function and collect results with ray for parallelisation
ray_objects = [function_two.remote(an_integer=i) for i in some_integers]
future_results = ray.get(ray_objects)
print('Results: {}'.format(future_results))
# Terminate ====================================
my_test_function.terminate() # close the matlab app containing the code for the test function
ray.shutdown() # shutdown any open CPU pool
== Error from mwpython console =====================================
% /Applications/MATLAB/MATLAB_Runtime/v99/bin/mwpython ~/Documents/TestTwo/for_redistribution_files_only/run_test_2_matlab_wrapped_in_python_parallel.py
2020-11-06 21:09:00,142 INFO services.py:1166 -- View the Ray dashboard at http://127.0.0.1:8265
/Applications/MATLAB/MATLAB_Runtime/v99/bin/maci64/mwpython3.7.app/Contents/MacOS/mwpython3.7: can't find '__main__' module in ''
<class 'matlab_pysdk.runtime.deployablepackage.DeployablePackage'>
<class 'matlab_pysdk.runtime.deployablefunc.DeployableFunc'>
Traceback (most recent call last):
File "/Users/me/Documents/TestTwo/for_redistribution_files_only/run_test_2_matlab_wrapped_in_python_parallel.py", line 32, in <module>
ray_objects = [function_two.remote(an_integer=i) for i in some_integers]
File "/Users/me/Documents/TestTwo/for_redistribution_files_only/run_test_2_matlab_wrapped_in_python_parallel.py", line 32, in <listcomp>
ray_objects = [function_two.remote(an_integer=i) for i in some_integers]
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/remote_function.py", line 99, in _remote_proxy
return self._remote(args=args, kwargs=kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/remote_function.py", line 207, in _remote
self._pickled_function = pickle.dumps(self._function)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 70, in dumps
cp.dump(obj)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 656, in dump
return Pickler.dump(self, obj)
TypeError: 'str' object is not callable
1 Comment
Answers (0)
See Also
Categories
Find more on Call Python from MATLAB in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!