How to save and load data in parallel while optimizing?
Show older comments
Hello every one,
I am optimizing 30 equations with Genetic Algorithm and need to save data in each time step and load it in the next time step. This makes my optimization too long, which is not efficient. I ran it in parallel, but it messed up the data. Would you help me with solving this issue if you have any similar experiences?
here is the part of the code that shows how I save data and reload them:
Data = [t ; y(15) ; y(22) ; y(29) ; y(30)];
if t ==0
fileID = fopen('Data.txt','w'); fprintf(fileID, '%f %f %f %f %f \n', Data); fclose(fileID);
else
fileID = fopen('Data.txt','a'); fprintf(fileID, '%f %f %f %f %f \n', Data); fclose(fileID);
end
if t >Tau_D
Data = load ('Data.txt','Data');
for i = 1 : size (Data,1)-1
if t - Tau_D < Data(i+1,1) && t - Tau_D >= Data(i,1)
CD = Data(i,2);
end
end
end
if t > Tau_t
for i = 1 : size (Data,1)-1
if t - Tau_t < Data(i+1,1) && t - Tau_t >= Data(i,1)
CD8T = Data(i,3);
IgM = Data(i,4);
IgG = Data(i,5);
end
end
end
6 Comments
Raymond Norris
on 7 Jul 2020
I'd be best to post your code, using the code formatting, which i've done here for you.
Data = [t ; y(15) ; y(22) ; y(29) ; y(30)];
if t == 0
fileID = fopen('Data.txt','w'); fprintf(fileID, '%f %f %f %f %f \n', Data); fclose(fileID);
else
fileID = fopen('Data.txt','a'); fprintf(fileID, '%f %f %f %f %f \n', Data); fclose(fileID);
end
if t > Tau_D
Data = load ('Data.txt','Data');
for i = 1 : size (Data,1)-1
if t - Tau_D < Data(i+1,1) && t - Tau_D >= Data(i,1)
CD = Data(i,2);
end
end
end
if t > Tau_t
for i = 1 : size (Data,1)-1
if t - Tau_t < Data(i+1,1) && t - Tau_t >= Data(i,1)
CD8T = Data(i,3);
IgM = Data(i,4);
IgG = Data(i,5);
end
end
end
Assigning t, y, Tau_D, and Tau_t to random valaues, doesn't show any issues. You should post what issue you're running into as well.
Mohammad Sami
on 7 Jul 2020
If your processing of timestep t+1 is dependent on the output of t, i don't see how you can parallelize your code.
To parallelize ideally you will need steps that can be undertaken independently of each other.
I also do not understand why you need to save and load data on every iteration.
Hamideh Hayati
on 7 Jul 2020
Edited: Hamideh Hayati
on 7 Jul 2020
Hamideh Hayati
on 7 Jul 2020
Edited: Hamideh Hayati
on 7 Jul 2020
Hamideh Hayati
on 7 Jul 2020
Edited: Hamideh Hayati
on 7 Jul 2020
Mohammad Sami
on 8 Jul 2020
I would suggest you need to centralise the reading and writing of the data to the file.
Multiple threads trying to read and write from the same file will cause errors.
Use the function parfeval to call the processing in the parallel and pass in all the data the function needs to work. Ideally your function will not read nor write to the file.
The functions will then compute the values on parallel processes. You can then fetch the results on the main thread and update your files.
More information is available in the documentation on how to use parfeval.
Answers (1)
Rohith Nomula
on 7 Jul 2020
Use parfor - parallel for loop in MATLAB
Then use a cell array to store all the values assigned in the looping state
% declare variables outside parallel for loop
parfor i=1:fileLength
% your condition
CD8T{i} = Data(i,3);
IgM{i} = Data(i,4);
IgG{i} = Data(i,5);
end
This works well for optimization
You can use it in both for loops
1 Comment
Hamideh Hayati
on 7 Jul 2020
Edited: Hamideh Hayati
on 7 Jul 2020
Categories
Find more on Loops and Conditional Statements in Help Center and File Exchange
on 6 Jul 2020
on 8 Jul 2020
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
