How to create a cluster profile without GUI ?

7 views (last 30 days)
I'm trying to run matlab codes on a HPC cluster with MATLAB Distributed Computing Server (Version 6.11 R2017b) installed using PBS Pro.
I can only submit non-interactive jobs, which can lunch matlab and execute codes automatically, so I created a PBS Pro cluster profile without validation (all properties are default) on my NB, with Parallel Computing Toolbox (Version 6.12 R2018a) installed only, then I uploaded the exported setting file, and added the importing command into codes.
this is an example I submit.
the .sh file submiting job:
#!/bin/bash
#PBS -P TRI107382
#PBS -N 0
#PBS -l select=2:ncpus=2
#PBS -l walltime=00:30:00
#PBS -q ctest
cd $PBS_O_WORKDIR
/pkg/matlab/bin/matlab -nodisplay -r test
test.m:
parallel.importProfile('PBSProProfile')
parallel.defaultClusterProfile('PBSProProfile1');
c = parcluster
c.ResourceTemplate='-l select=2:ncpus=2 -P TRI107382'; %I use two nodes with two cpu cores in each.
set(c, 'NumWorkers', 4);
j = createJob(c);
for i = 1:3
createTask(j,@test_f,1,{i});
end
submit(j);
wait(j);
j.State
out = fetchOutputs(j)
test_f.m:
function y=test_f(x)
y=x;
end
Then, I got output messages and error messages as follows:
output:
ans =
'PBSProProfile1_Import_10'
[Warning: The PBSPro cluster type is being used with a possibly incompatible
version of PBS. The use of job arrays has been disabled. The version of PBS
detected was: "14.2.4.20171012010902".]
[> In parallel.cluster.PBSPro>iCheckPBSProVersion (line 114)
In parallel.cluster.PBSPro (line 63)
In parcluster (line 54)
In test (line 3)]
c =
PBSPro Cluster
Properties:
Profile: PBSProProfile1
Modified: false
Host: srvc1
NumWorkers: Inf
NumThreads: 1
JobStorageLocation: /home/u7/gsblovedyt0
ClusterMatlabRoot: /pkg/matlab
OperatingSystem: unix
SubmitArguments:
ResourceTemplate: -l select=^N^
RcpCommand: rcp
RshCommand: pbs_remsh
CommunicatingJobWrapper: unix
RequiresMathWorksHostedLicensing: false
Associated Jobs:
Number Pending: 3
Number Queued: 2
Number Running: 0
Number Finished: 4
ans =
'failed'
error:
Error using parallel.Job/fetchOutputs (line 1087)
Outputs can only be fetched if the job is in State 'finished'.
Error in test (line 16)
out = fetchOutputs(j)
I think the reason is that because of the difference of PBS Pro's version, the job array is disable, so the job is failed. Therefore I'm seeking a way of creating a PBS Pro cluster profile on the HPC with command-line interface so that the version may be correct. At the login node of HPC, for trial running, I can launch matlab in command-line mode, so I can interactively execute matlab commands, but I don't know how to generate cluster profiles in this way.

Accepted Answer

Jason Ross
Jason Ross on 30 May 2018
Have you tried using the PBS Pro Support Package? You can get it through the Add Ons menu -- just search on "pbs". After you install it, there's a wizard that will help you build the profile. It will take care of connecting to the submit node and submitting the job.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!