Why doesn't parfeval(@splitapply) improve splitapply's performance?
7 views (last 30 days)
Show older comments
I want to readtable many html-files to extract tables. I wrote a function extract_sheet to do just that. I had used parfor to perform this task, and it runs decently fast. Then it occurs to me that those html-files can be grouped according to their foder and filename segments. So, I try splitapply(extract_sheet, input variables, groupNumber), and it works. Then I want to see if parfeval would improve the speed. I do something like parfeval(@splitapply, extract_sheet, input variables, groupNumber.)
For a small testing file list, both methods spend almost the same amount of elapsed time, around 27.5 +/- .1 seconds. My question is why parfeval doesn't improve the performance?
0 Comments
Accepted Answer
Matt J
on 31 Aug 2023
Edited: Matt J
on 31 Aug 2023
It probably means that Matlab's internal parallellization already does what parfeval does.
6 Comments
Sam Marshalik
on 5 Sep 2023
Edited: Sam Marshalik
on 5 Sep 2023
@Matt J: You bring up a good point that the doc page is lacking information on this topic. I put in an enhancement request to improve that. In the meantime, I would suggest to call our Technical Support - they can investigate this further and reach out to the relevant Dev team.
More Answers (1)
Matt J
on 5 Sep 2023
Edited: Matt J
on 5 Sep 2023
If you're going to be using PCT functions anyway, I wonder if a parfor loop might do better than splitapply. I.e., instead of,
splitapply(func,X,G)
one might instead do,
I=splitapply(@(x){x}, 1:numel(G), G);
parfor j=1:numel(I)
results{j}=func( X(I{j}) );
end
See Also
Categories
Find more on Performance and Memory in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!