Do worker nodes in a parallel pool need direct connectivity to the client?

2 views (last 30 days)
I'm trying to start a parallel pool on a HPC cluster using a windows client. I've installed the third-party SLURM tool add-on and configured a cluster. All validation checks pass except the last one for parpool:
it says:
Error Report: Failed to initialize the interactive session.
Caused by:
Error using parallel.internal.pool.AbstractInteractiveClient>iThrowIfBadParallelJobStatus (line 423)
The interactive communicating job errored with the following message: MatlabPoolPeerInstance{fLabIndex=1, fNumberOfLabs=2, fUuid=0c40d093-9791-4944-8fe5-30feb46c98d5} was unable to find the host for workstation:27370 due to a JVM UnknownHostException: null
"workstation" being the hostname of my local machine.
It looks like the worker nodes are trying to reach my local machine, which of course is not possible, as they are isolated. I feel they should be maybe contacting the ClusterHost to which MATLAB connets via SSH and from which the jobs are submitted?

Accepted Answer

Raymond Norris
Raymond Norris on 10 Mar 2022
Our scheduler, MJS, supports interactive pools running from your desktop machine because the traffic is routed via the scheduler on the cluster. For other schedulers, like Slurm, we initiate the server socket on the desktop machine. As noted it's less likely this will work (since it's probably not reachable), but you have a couple of options.
  • For starters, if your desktop machine is reachable via IP address, you can configure it as such
pctconfig('hostname','a.b.c.d');
This requires being called each time you start MATLAB (can be placed in your startup.m file) and before you call any parallel code (e.g., parpool).
  • If you can't resolve your public IP address, if you VPN into the university/company network, try using the IP address provided by your VPN.
  2 Comments
Raymond Norris
Raymond Norris on 11 Mar 2022
Great to hear. Can you tell us what step(s) you took? Was it just changing the hostname or did you need to VPN in as well?

Sign in to comment.

More Answers (0)

Categories

Find more on Third-Party Cluster Configuration in Help Center and File Exchange

Products


Release

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!