Just Curious, when I run parallel computing with k cores, why there are k matlab background processes in the task manager on my machine?
4 views (last 30 days)
Show older comments
I'm recently running some simulations using matlab parallel computing with 100 cores(distributed across 10 nodes) on a computing cluster of my university. However, I received warnings about consuming too much CPUs on the node where I ran my file(after logging in). I checked the task manager and found that there are 100 matlabs launched as background processes, each one consuming an equal amount of CPU on this node(that's why total CPU consumption quickly blow up). Once I shut down 'parpool', those background processes disappeared. Just curious about what is going on there, why does matlab start a background process for each parallel on my machine? What are these background processes for?
0 Comments
Accepted Answer
Matt J
on 2 Nov 2017
That's how parallel labs in the Parallel Computing Toolbox work. Essentially, they each run their own separate MATLAB session.
2 Comments
Matt J
on 3 Nov 2017
Edited: Matt J
on 3 Nov 2017
I think the 100 background processes are inevitable, since they are running your 100 parallel jobs. The question is whether your entire parpool is running on your host node. You probably want to distribute some workers to other nodes. I suspect you need to dig into cluster profiles to do that,
More Answers (0)
See Also
Categories
Find more on Clusters and Clouds in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!