Clear Filters
Clear Filters

About the efficiency of parallel computing

6 views (last 30 days)
Halil Eyyuboglu
Halil Eyyuboglu on 1 Oct 2018
Edited: Stephen23 on 3 Oct 2018
Hello ; We have a parallel setup in our university on a Dell computer which has 56 cores (all within the same machine). When we start the Admin Center, we see the number of cores as 56, but the count for number of workers is 112.
We have run different m files (both simple test and our actual jobs) with this configuration mainly using parfor loops.
The reduction in computation time is around 20 times.
Is this reasonable ?
We were expecting (taking into account the number of core) a computation time reduction around fifty times.
Your kind comments would be appreciated.
Halil Eyyuboglu

Answers (1)

OCDER on 1 Oct 2018
Read this:
It depends on the communication overhead. If you have to send and collect data from workers frequently, you'll get less than Num_of_Workers reduction in computation time. In some cases (for small jobs), parallel computing is actually slower than single-core jobs. The best way to set this up is to ensure each worker has a good chunk of processing to do and only when it's done, send data once back to the main terminal.
Note that if each worker is trying to read/write data to hard drive, then this data accessing speed could be the limiting factor and even having 10000 workers may not help. To speed things up, first find the bottleneck in your code using normal for-loops and by using the profiler.
Stephen23 on 3 Oct 2018
Halil Eyyuboglu's "Answer" moved here and formatted correctly:
Thanks for the kind reply. Yes you are right, the coordination of the workers to send data and collect results would simetimes overshadow the computation time. But I think my case is quite straight forward. To clarify the point I have included below the piece of code I use for test. As far as I understand here, there is almost no communication cost, no need for coordination between workers. Yet, although I have 56 cores, I get only a reduction of 20 times compared with the case of replacing parfor by for (on the same machine). As I said in my previous correspondence, my expectation is at least 50 times. The same applies to my actual m file I run.
I would be quite happy, if I can receive some comments to the following points
1) When I run the admin center, for 56 cores, 112 workers are assingned, can I change to 56 workers, how ? Would this bring any further reductions in computation time ?
2) Is there something wrong with the test code I have given below. Or rather can you suggest a simple test in order to clearly see the benefit of many workers and the (linear) relationship between the number of workers and reduction in computation time.
I would appreciate your kind reply
clear;clc;clf reset;close all
Nmax = 100;tic
for io = 1:10
parfor im = 1:Nmax
for im1 = 1:Nmax
zm = cos(rand([Nmax,Nmax,21]));
Stephen23 on 3 Oct 2018
Edited: Stephen23 on 3 Oct 2018
"As far as I understand here, there is almost no communication cost"
All parallel computations have an overhead, which is not just due to communication, but to actually setting up the parallel pool, splitting up the tasks, and combing the data afterwards. This is not a trivially small time, and cannot be disregarded.
"Is this reasonable ?"
Probably, it depends on your calculation, which you told us nothing about.
" As I said in my previous correspondence, my expectation is at least 50 times"
Then your expectation does not reflect the reality of how parallel computing actually works. Search this forum to know more: this topic has been discussed many times before:
This is also explained in the documentation:

Sign in to comment.


Find more on Cluster Configuration in Help Center and File Exchange




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!