Reference

class plus_slurm.JobCluster(required_ram='2G', request_cpus=1, jobs_dir='jobs', request_time=10, inc_jobsdir=True, qos=None, python_bin=None, working_directory=None, exclude_nodes=None, max_jobs_per_jobcluster=1000, append_to_path=None, extra_slurm_args=None, export='NONE')

This is the main class, the controller of plus_slurm. It collects all the jobs and takes care of submitting them to the cluster. It also contains information about how much RAM the jobs need, how many CPUs are requested etc.

Parameters:

• required_ram (str, float, int, optional) – The amount of RAM required to run one Job in megabytes. A string like “2G” or “200M” will be converted accordingly.

• request_cpus (int, optional) – The number of CPUs requested

• qos (str, optional) – The quality of service to use.

• request_time (int, optional) – Maximum duration of a job in minutes.

• jobs_dir (str, optional) – Folder to put all the jobs in. This one needs to be on the shared filesystem (so somewhere under /mnt/obob)

• inc_jobsdir (str, optional) – If this is set to True (default), jobs_dir is the parent folder for all the jobs folders. Each time a job is submitted, a new folder is created in the jobs_dir folder that contains all the necessary files and a folder called “log” containing the log files. If jobs_dir is set to False, the respective files are put directly under jobs_dir. In this case, jobs_dir must either be empty or not exist at all to avoid any side effects.

• python_bin (str, optional) – The path to the python interpreter that should run the jobs. If you do not set it, it gets chosen automatically. If the python interpreter you are using when submitting the jobs is on /mnt/obob/ that one will be used. If the interpreter you are using is not on /mnt/obob/ the default one at /mnt/obob/obob_mne will be used.

• working_directory (str, optional) – The working directory when the jobs run.

• exclude_nodes (str, optional) – Comma separated list of nodes to exclude.

• max_jobs_per_jobcluster (int, optional) – Slurm only allows a certain number of jobs per array job (1000 by default). If the number of jobs to be submitted is higher that this number, jobs will be split in more than one array job.

• append_to_path (str, optional) – Path to append to the python module search path.

• extra_slurm_args (list, optional) – Extra arguments for slurm sbatch. These get appended as is with the #SBATCH prefix.

• export (str, optional) – What environment variables to export to the job. Look here for options: https://slurm.schedmd.com/sbatch.html#OPT_export

add_job(job, *args, **kwargs)

Add one job to the JobCluster. All further arguments will be passed on to the Job.

Parameters:

• job (child of plus_slurm.Job) – The job class to be added.

• *args – Variable length argument list.

• **kwargs – Arbitrary keyword arguments.

run_local()

Runs the added jobs locally.

submit(do_submit=True)

Runs the added jobs on the cluster.

Parameters:

do_submit (bool, optional) – Set this to false to not actually submit but prepare all files.

class plus_slurm.ApptainerJobCluster(*args, apptainer_image=None, mounts=('/mnt',), mount_slurm_folders=True, apptainer_args='', prepull_apptainer_image=True, **kwargs)

Bases: JobCluster

If you use this class instead of JobCluster, the jobs will run in a Apptainer Container.

You need to supply the same parameters as when instantiating JobCluster. Additionally, you need to at least set the Apptainer Image you want to use.

Parameters:

• apptainer_image (str) – Set this to a apptainer image to have the jobs execute in it. Can be a link to a local file or to some online repository.

• mounts (tuple, list, optional) – What to mount in the container.

• mount_slurm_folders (bool) – If true, mount /etc/slurm and /var/run/munge so you can issue slurm commands from within the job.

• apptainer_args (str, optional) – Additional arguments to supply to apptainer

• prepull_apptainer_image (bool, optional) – Pull the apptainer image before running the jobs no avoid 100s of jobs pulling the same image at the same time.

submit(do_submit=True)

Runs the added jobs on the cluster.

Parameters:

do_submit (bool, optional) – Set this to false to not actually submit but prepare all files.

class plus_slurm.Job(*args, **kwargs)

Abstract class for Jobs. This means, in order to define you own jobs, they need to be a subclass of this one.

You must implement (i.e. define in your subclass) the run() method. The run method can take as many arguments as you like. Only the types of arguments are restricted because they need to be saved to disk. In general, strings, numbers, lists and dictionaries are fine.

You can implement shall_run(). This can be used to see whether some output file already exists and restrict job submission to missing files.

run(*args, **kwargs)

Implement this method to do the job.

shall_run(*args, **kwargs)

This is an optional method. It gets called with the same arguments as the run() method, before the job is submitted. If it returns True, the job is submitted, if it returns False, it is not.

class plus_slurm.AutomaticFilenameJob(*args, **kwargs)

Bases: Job, ABC

Abstract class for Jobs providing automatic filename generation.

In order for this to work, you need to:

1. Set base_data_folder and job_data_folder as a

   class attribute.

2. If you use shall_run(), you need to do the super call.

This class then automatically creates the filename for each job using all the keyword arguments supplied.

Please take a look at Automatically generating filenames for your jobs for detailed examples.

Variables:

• base_data_folder (str or pathlib.Path) – The base folder for the data. Is normally set once for all jobs of a project.

• job_data_folder (str or pathlib.Path) – The folder where the data for this job should be saved.

• exclude_kwargs_from_filename (list) – Normally, all keyword arguments are used to build the filename. if you want to exclude some of them, put the key in the list here.

• include_hash_in_fname (bool) – Include a hash of all arguments in the filename. This is helpful if you excluded some keyword arguments from filename creation but still need to get distinct filename.

• run_only_when_not_existing (bool) – If true, this job will only run if the file does not already exist.

• create_folder (bool) – If true, calling folders are created automatically

• data_file_suffix (str, optional) – The extension of the file. Defaults to .dat

classmethod get_full_data_folder()

Return the data folder for this job (i.e. base_data_folder plus job_data_folder).

shall_run(*args, **kwargs)

This is an optional method. It gets called with the same arguments as the run() method, before the job is submitted. If it returns True, the job is submitted, if it returns False, it is not.

property full_output_path

The full path to the output file.

Type:

pathlib.Path

property output_filename

The filename for this subject.

Type:

str

property output_folder

The output folder for this subject.

Type:

pathlib.Path

class plus_slurm.PermuteArgument(args)

This is a container for to-be-permuted arguments. See the example in the introductions for details.