Parallel for loop problem
2 views (last 30 days)
Show older comments
Commented: Simone Fiumi on 12 Aug 2022
I'm running the following code in parallel. The variables of interes are w_b and alpha_b. The problem is that if I pause at a random time and then quit debugging the variables computed up to now are not saved. If i do that without the parallel cicle but with a simple for loop the variables are saved. For example, let's say I stop after 1 hour and I have 5 variables saved using a standard for loop, while with a parfor loop there are no saved variables (they are still equal to 0 as preallocated).
[LL, ~, P, obsmat, nrIterations]=dhmm_em(seq, prior, P_start_b, E,N_btstrp,Y_btstrp);
[w_b(k),alpha_b(k)]=parameters(P,ne,nn,N_btstrp,Y_btstrp,1); % VARIABLES OF INTEREST
Raymond Norris on 12 Aug 2022
@Simone Fiumi think of the code running in the parfor as running on some entirely different machine. There is no "debugging" per se. So pausing/stopping the parfor terminates the work entiring including the ability to checkpoint variables.
What you could consider doing is creating a data queue that sends data back to the MATLAB client. The client (outside of the parfor) could view the data queue. For instance:
D = parallel.pool.DataQueue;
parfor idx = 1:10
[wb, ab] = parameters(P,ne,nn,N_btstrp,Y_btstrp,1);
% Send data back to client
w_b(k) = wb;
alpha_b(k) = ab;
% Evaluate a snapshot of wb, ab
Not a perfect solution, but might help.
Benjamin Thompson on 12 Aug 2022
You may want to review this thread and use parfeval if the partial results option is something you want.
Find more on Loops and Conditional Statements in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!