Issues with running genetic algorithm (GA) in parallel mode

Firstly, I want to describe the outline/idea of my code.

I am using GA to optimize a quite complex problem.

(1) I have a main funtion. Within this main function I am using GA function.

The code is like this:

[x,fval,exitflag] = ga(@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,...
            FluidViscosity,PropDensity_0,PropDiameter_0,NoofClusters,Opt_PerfNo,Opt_PerfDia,...
            Opt_ClusterSpacing,Opt_ProppSize,Opt_ProppDen,Opt_MaxInjRate,Opt_MaxPropConc,Opt_RampUpSpeed,...
            PerfNo_Option,PerfDia_Option,ProppSize_Option,ProppDen_Option,NoofStages,...
            ProppPerCluster,SlurryPerCluster),6+2*NoofClusters,A,b,[],[],...
        lb,ub,@(Vars)windows_nlcon(Vars,NoofClusters,ProppPerCluster,SlurryPerCluster),IntCon,opts);

(2) In the GA objective function:

function [windows_objective] = windows_objective(Vars, Case0Input_path, Engine_path, Input_path, security_count, FluidDensity, FluidViscosity,PropDensity_0,PropDiameter_0,NoofClusters, Opt_PerfNo,Opt_PerfDia,Opt_ClusterSpacing,Opt_ProppSize,Opt_ProppDen,Opt_MaxInjRate,Opt_MaxPropConc,Opt_RampUpSpeed,PerfNo_Option,PerfDia_Option,ProppSize_Option,ProppDen_Option,NoofStages,ProppPerCluster,SlurryPerCluster)

an external .exe program will be called and multiple output files will be created by this external program. Therefore, within this objective function, I create a folder called "MatlabOpt" which will store all the output files created by the external program. After finish running this external program, by reading these output files, the objective function will calculate the objective function value and return it to the main function.

Whenever the objective function is called, it will check "MatlabOpt" folder like this (within the objective function):

X = sprintf('I am worker %d',labindex);
disp(X)
MatlabOpt_path = char(strcat(Input_path,'\MatlabOpt',num2str(labindex)));
MyFolder = MatlabOpt_path; % this is a folder
if ~isdir(MyFolder)            % The first time to call objective function
    mkdir(MatlabOpt_path) % this is a folder
    copyfile(Case0Input_path, MatlabOpt_path)
else
    rmdir(MatlabOpt_path, 's')
    mkdir(MatlabOpt_path) % this is a folder
    copyfile(Case0Input_path, MatlabOpt_path)
end

Secondly, I want to describe my issues.

(1) One run of the external program will take around 40 seconds. I give the GA function population size "100" and for my problem usually GA needs to run at least 20 generations to get the optimal solution. Which means that it will take around "20*100*40=80,000s". Therefore, I want to speed up my code.

Q1: Is my undestanding/calculation of the total run time correct or not?

(2) Therefore, I want to use GA in parallel mode through the following settings:

opts = optimoptions('ga','MaxStallGenerations',10,'FunctionTolerance',1e-5,...
    'MaxGenerations',200,'PlotFcn',@gaplotbestf,'PopulationSize',100,...
    'InitialPopulationRange',[initiallb;initialub],...
    'UseParallel',true);

(3) I am thinking about the mechanism/algorithm/idea of GA in parallel mode.

Q2: I am thinking that, for example, in my case, my poupulation size is 100, which means GA needs to calculate 100 objective function values for one generation, yes?

Q3: If I have 4 workers in parallel mode, which means one worker will calculate 100/4=25 objective function values, yes?

If I am correct, to make sure multiple workers could calculate the objective function simultaneously, within the objective function, I create the MatlabOpt folder with the worker id.

X = sprintf('I am worker %d',labindex);
disp(X)
MatlabOpt_path = char(strcat(Input_path,'\MatlabOpt',num2str(labindex)));

(4) When I run GA in serial, I can see that the folder is called "MatlabOpt1" and my codes could be run without problems.

(5) However, when I run GA in parallel mode, I will have the following error:

I am worker 1
I am worker 1
I am worker 1
I am worker 1
Error using windows_objective (line 16)
No directories were removed.

Error in
windows_frac_optimization>@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,FluidViscosity,PropDensity_0,PropDiameter_0,NoofClusters,Opt_PerfNo,Opt_PerfDia,Opt_ClusterSpacing,Opt_ProppSize,Opt_ProppDen,Opt_MaxInjRate,Opt_MaxPropConc,Opt_RampUpSpeed,PerfNo_Option,PerfDia_Option,ProppSize_Option,ProppDen_Option,NoofStages,ProppPerCluster,SlurryPerCluster)
(line 192)
    [x,fval,exitflag] =
    ga(@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,...
    
Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11)
fcn_handle = @(x) fcn(x,FcnArgs{:});

Error in fcnvectorizer (line 16)
        parfor (i = 1:popSize)

Error in gaminlppenaltyfcn

Error in gapenalty

Error in makeState (line 64)
            Score = FitnessFcn(state.Population(initScoreProvided+1:end,:));

Error in galincon (line 17)
state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);

Error in gapenalty

Error in gaminlp

Error in ga (line 393)
    [x,fval,exitFlag,output,population,scores] = gaminlp(FitnessFcn,nvars, ...

Error in windows_frac_optimization (line 192)
    [x,fval,exitflag] =
    ga(@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,...
    
Caused by:
    Failure in user-supplied fitness function evaluation. GA cannot continue.
    Failure in initial user-supplied fitness function evaluation. GA cannot continue.

Finally, your help/comments/suggestions would be greatly appreciated!!

Thanks and best regards,

Min

Answer 1

[Min Zhang]

Based on this post (https://www.mathworks.com/matlabcentral/answers/368486-using-parallel-computing-with-external-software), I have modified my code accordingly.

(1) Main function:

% run in parallel
spmd
  mkdir(sprintf('worker%d', labindex));
  copyfile(Case0Input_path,sprintf('worker%d',labindex));
  cd(sprintf('worker%d', labindex));
end

addAttachedFiles(gcp,{'windows_nlcon.m','windows_generatePumpingSchedule.m'})

(2) GA objective function:

X = sprintf('I am worker %d',labindex);
disp(X)
%MatlabOpt_path = char(strcat(Input_path,'\MatlabOpt',num2str(labindex))); % this line has been commented out
MatlabOpt_path = char(strcat(pwd,'\MatlabOpt')); % this line has been added

MyFolder = MatlabOpt_path; % this is a folder
if ~isdir(MyFolder)            % The first time to call objective function
    mkdir(MatlabOpt_path) % this is a folder
    copyfile('Case0.txt', MatlabOpt_path)
else
    rmdir(MatlabOpt_path, 's')
    mkdir(MatlabOpt_path) % this is a folder
    copyfile('Case0.txt', MatlabOpt_path)
end

Then, I can run my code in parallel properly.

However, I have a new issue.

From my GA objective function, you can see that the folder "MatlabOpt" in floder "worker%d" will be created, deleted, created, deleted... The code works properly at the beginning. However, after sevearl hours, after around 20 generations of GA, the code gave me an error:

Error using windows_objective (line 23) % comment by Min: this line is "mkdir(MatlabOpt_path) % this is a folder"
Access is denied.



Error in
objAndConVectorizer>@(Vars)windows_objective(Vars,Engine_path,security_count,FluidDensity,FluidViscosity,PropDensity_0,PropDiameter_0,NoofClusters,Opt_PerfNo,Opt_PerfDia,Opt_ClusterSpacing,Opt_ProppSize,Opt_ProppDen,Opt_MaxInjRate,Opt_MaxPropConc,Opt_RampUpSpeed,PerfNo_Option,PerfDia_Option,ProppSize_Option,ProppDen_Option,NoofStages,ProppPerCluster,SlurryPerCluster)

Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11)
fcn_handle = @(x) fcn(x,FcnArgs{:});

Error in objAndConVectorizer (line 37)
        parfor (i = 1:popSize)

Error in gaminlppenaltyfcn

Error in gapenalty

Error in stepGA (line 46)
    nextScore = FitnessFcn(nextPopulation);

Error in galincon (line 62)
        [score,population,state] =
        stepGA(score,population,options,state,GenomeLength,FitnessFcn);

Error in gapenalty

Error in gaminlp

Error in ga (line 393)
    [x,fval,exitFlag,output,population,scores] = gaminlp(FitnessFcn,nvars, ...

Error in windows_frac_optimization (line 234)
        [x,fval,exitflag] =
        ga(@(Vars)windows_objective(Vars,Engine_path,security_count,FluidDensity,...

Caused by:
    Failure in user-supplied fitness function evaluation. Cannot continue.
 
IdleTimeout has been reached.
Parallel pool using the 'local' profile is shutting down.

This error tells me that, after hundreds of successful "creating, deleting, creating, deleting, ..." of this "MatlabOpt" folder, it cannot create this folder anymore after deleting it.