How much overhead cost is incurred by using a function handle as a broadcast variable in a parfor loop?

I'm trying to parallellize the process of fitting to an (arbitrary) custom function, which I want to be able to specify beforehand:

fitfun = @function
parfor i=1:N
  fitparams{i} = fitfun(data(i));
end

However, in the function I'm writing, I get warnings about putting a broadcast variable (fitfun) in a parfor loop. Is this overhead significant or comparable to, say, broadcasting a short vector?

[Matt J]

The title of your post asks about broadcasting function handles, but your example is about broadcasting fittype objects.

[Ty Easley]

Edited. Thanks!

Answer 1

[Matt J]

You'll have to try and find out. It's hard to know how much memory an object or function handle is dragging around with it. For broadcasting anonymous function handles, I wrote this cleaning tool to sift out unwanted memory.

[Ty Easley]

Oh, yikes, ok. After reading a little more (thanks for the AFSLIM link), it looks like the problem of dragging around workspace variables may not apply to me, but I'm not sure. I intend to have "fitfun" defined in a separate '.m' file somewhere on the user path, and then passed into the function that does the parallel fitting, e.g. 'EMM4fit.m' is the code for a function taking 'data' as an input and returning 'params' as an output and

fitfun = @EMM4fit
parfor i=1:N
  fitparams{i} = fitfun(data(i));
end

Do you know if calling a function from a file like this has an effect on what it will try to drag along with it in the handle?

[Matt J]

I think that should be fine. Only nested and anonymous function handles drag stuff along.

[Ty Easley]

Excellent. Thanks!

Answer 2

[Edric Ellis]

It's not necessarily a problem to have broadcast data in a parfor loop. If you're using R2016b or later, you can assess the amount of data being transferred to and from the workers using ticBytes and tocBytes.