About the efficiency of parallel computing

5 views (last 30 days)
Halil Eyyuboglu
Halil Eyyuboglu on 1 Oct 2018
Edited: Stephen23 on 3 Oct 2018
Hello ; We have a parallel setup in our university on a Dell computer which has 56 cores (all within the same machine). When we start the Admin Center, we see the number of cores as 56, but the count for number of workers is 112.
We have run different m files (both simple test and our actual jobs) with this configuration mainly using parfor loops.
The reduction in computation time is around 20 times.
Is this reasonable ?
We were expecting (taking into account the number of core) a computation time reduction around fifty times.
Your kind comments would be appreciated.
Halil Eyyuboglu

Answers (1)

OCDER on 1 Oct 2018
Read this:
It depends on the communication overhead. If you have to send and collect data from workers frequently, you'll get less than Num_of_Workers reduction in computation time. In some cases (for small jobs), parallel computing is actually slower than single-core jobs. The best way to set this up is to ensure each worker has a good chunk of processing to do and only when it's done, send data once back to the main terminal.
Note that if each worker is trying to read/write data to hard drive, then this data accessing speed could be the limiting factor and even having 10000 workers may not help. To speed things up, first find the bottleneck in your code using normal for-loops and by using the profiler.
Stephen23 on 3 Oct 2018
Edited: Stephen23 on 3 Oct 2018
"As far as I understand here, there is almost no communication cost"
All parallel computations have an overhead, which is not just due to communication, but to actually setting up the parallel pool, splitting up the tasks, and combing the data afterwards. This is not a trivially small time, and cannot be disregarded.
"Is this reasonable ?"
Probably, it depends on your calculation, which you told us nothing about.
" As I said in my previous correspondence, my expectation is at least 50 times"
Then your expectation does not reflect the reality of how parallel computing actually works. Search this forum to know more: this topic has been discussed many times before:
This is also explained in the documentation:

Sign in to comment.


Find more on Cluster Configuration in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!