Slow access of large composite element

I am using spmd to run some parallel algorithms which periodically communicate. At the end, each process has a structure which I would like to pull back to the client for further analysis. Unfortunately, the size of the structure is large (but I don't think they are excessive), and trying to pull just one of the composite elements leads to a timeout of the cluster (30 min) and therefore fails. Is there any way to have the client as one of the spmd processes, or a more efficient way to transfer data from one or more of the workers back to the client in a more efficient way?

EDIT:

A clunky workaround is to have each process save their workspace to a directory. This seems to work pretty quickly, but it's obnoxious to have to import all the workspaces to do the diagnostics. I would appreciate more elegant solutions.

[Walter Roberson]

I just now noticed psave() and pload(); perhaps those would make it more convenient?

[rp]

It looks like psave won't work in an spmd block.

[Walter Roberson]

rp, if you look at the documentation https://www.mathworks.com/help/distcomp/psave.html notice that it references labIndex. labIndex only exists for spmd, so this is a facility specific to spmd.

[rp]

@Walter -- yes you must be right. I quickly read through and saw that the example wouldn't work in an spmd block, but there are probably other examples which may.

[Walter Roberson]

Looking more closely, it is for "communicating jobs", of which spmd is one kind.

Answer 1

[Walter Roberson]

Perhaps the new (R2017a) send() and poll() and afterEach() might help?