Batch Class Reference

Inheritance diagram for Batch:

Collaboration diagram for Batch:

Public Member Functions
def	__init__ (self, *backend_args=None)
def	can_submit (self, *args, **kwargs)
def	submit (self, job, check_can_submit=True, jobs_per_check=100)
def	get_batch_submit_script_path (self, job)
def	submit (self, job)
def	get_submit_script_path (self, job)

Public Attributes
	global_job_limit
	The active job limit. More...
	sleep_between_submission_checks
	Seconds we wait before checking if we can submit a list of jobs. More...
	backend_args
	The backend args that will be applied to jobs unless the job specifies them itself.

Static Public Attributes
list	submission_cmds = []
	Shell command to submit a script, should be implemented in the derived class.
int	default_global_job_limit = 1000
	Default global limit on the total number of submitted/running jobs that the user can have. More...
int	default_sleep_between_submission_checks = 30
	Default time betweeon re-checking if the active jobs is below the global job limit.
string	submit_script = "submit.sh"
	Default submission script name.
string	exit_code_file = "__BACKEND_CMD_EXIT_STATUS__"
	Default exit code file name.
dictionary	default_backend_args = {}
	Default backend_args.

Private Member Functions
def	_add_batch_directives (self, job, file)
def	_make_submit_file (self, job, submit_file_path)
def	_submit_to_batch (cls, cmd)
def	_ (self, job, check_can_submit=True, jobs_per_check=100)
def	_ (self, job, check_can_submit=True, jobs_per_check=100)
def	_ (self, jobs, check_can_submit=True, jobs_per_check=100)
def	_create_job_result (cls, job, batch_output)
def	_create_cmd (self, job)
def	_add_wrapper_script_setup (self, job, batch_file)
def	_add_wrapper_script_teardown (self, job, batch_file)
def	_create_parent_job_result (cls, parent)

Static Private Member Functions
def	_add_setup (job, batch_file)

Detailed Description

Abstract Base backend for submitting to a local batch system. Batch system specific commands should be implemented
in a derived class. Do not use this class directly!

Definition at line 1104 of file backends.py.

Constructor & Destructor Documentation

__init__()

def __init__	(		self,
		*	backend_args = None
	)

Init method for Batch Backend. Does some basic default setup.

Reimplemented from Backend.

Reimplemented in LSF, and PBS.

Definition at line 1127 of file backends.py.

Member Function Documentation

_() [1/3]

def _ (   self,   job,   check_can_submit = True,   jobs_per_check = 100  )

private

Submit method of Batch backend for a `SubJob`. Should take `SubJob` object, create needed directories,
create batch script, and send it off with the batch submission command.
It should apply the correct options (default and user requested).

Should set a Result object as an attribute of the job.

Definition at line 1179 of file backends.py.

_() [2/3]

def _ (   self,   job,   check_can_submit = True,   jobs_per_check = 100  )

private

Submit method of Batch backend. Should take job object, create needed directories, create batch script,
and send it off with the batch submission command, applying the correct options (default and user requested.)

Should set a Result object as an attribute of the job.

Definition at line 1215 of file backends.py.

_() [3/3]

def _ (   self,   jobs,   check_can_submit = True,   jobs_per_check = 100  )

private

Submit method of Batch Backend that takes a list of jobs instead of just one and submits each one.

Definition at line 1266 of file backends.py.

_add_batch_directives()

def _add_batch_directives (   self,   job,   file  )

private

Should be implemented in a derived class to write a batch submission script to the job.working_dir.
You should think about where the stdout/err should go, and set the queue name.

Reimplemented in HTCondor, LSF, and PBS.

Definition at line 1139 of file backends.py.

_add_setup()

def _add_setup (   job,   batch_file  )

staticprivateinherited

Adds setup lines to the shell script file.

Definition at line 777 of file backends.py.

_add_wrapper_script_setup()

def _add_wrapper_script_setup (   self,   job,   batch_file  )

privateinherited

Adds lines to the submitted script that help with job monitoring/setup. Mostly here so that we can insert
`trap` statements for Ctrl-C situations.

Definition at line 784 of file backends.py.

_add_wrapper_script_teardown()

def _add_wrapper_script_teardown (   self,   job,   batch_file  )

privateinherited

Adds lines to the submitted script that help with job monitoring/teardown. Mostly here so that we can insert
an exit code of the job cmd being written out to a file. Which means that we can know if the command was
successful or not even if the backend server/monitoring database purges the data about our job i.e. If PBS
removes job information too quickly we may never know if a job succeeded or failed without some kind of exit
file.

Definition at line 809 of file backends.py.

_create_cmd()

def _create_cmd (   self,   job  )

private

Reimplemented in HTCondor, LSF, and PBS.

Definition at line 1315 of file backends.py.

_create_job_result()

def _create_job_result (   cls,   job,   batch_output  )

private

Reimplemented in HTCondor, LSF, and PBS.

Definition at line 1310 of file backends.py.

_create_parent_job_result()

def _create_parent_job_result (   cls,   parent  )

privateinherited

We want to be able to call `ready()` on the top level `Job.result`. So this method needs to exist
so that a Job.result object actually exists. It will be mostly empty and simply updates subjob
statuses and allows the use of ready().

Reimplemented in HTCondor, LSF, PBS, and Local.

Definition at line 822 of file backends.py.

_make_submit_file()

def _make_submit_file (   self,   job,   submit_file_path  )

private

Useful for the HTCondor backend where a submit is needed instead of batch
directives pasted directly into the submission script. It should be overwritten
if needed.

Reimplemented in HTCondor.

Definition at line 1147 of file backends.py.

_submit_to_batch()

def _submit_to_batch (   cls,   cmd  )

private

Do the actual batch submission command and collect the output to find out the job id for later monitoring.

Reimplemented in HTCondor, LSF, and PBS.

Definition at line 1156 of file backends.py.

can_submit()

def can_submit	(		self,
		*	args,
		**	kwargs
	)

Should be implemented in a derived class to check that submitting the next job(s) shouldn't fail.
This is initially meant to make sure that we don't go over the global limits of jobs (submitted + running).

Returns:
    bool: If the job submission can continue based on the current situation.

Definition at line 1161 of file backends.py.

get_batch_submit_script_path()

def get_batch_submit_script_path	(		self,
			job
	)

Construct the Path object of the script file that we will submit using the batch command.
For most batch backends this is the same script as the bash script we submit.
But for some they require a separate submission file that describes the job.
To implement that you can implement this function in the Backend class.

Reimplemented in HTCondor.

Definition at line 1299 of file backends.py.

get_submit_script_path()

def get_submit_script_path (   self,   job  )

inherited

Construct the Path object of the bash script file that we will submit. It will contain
the actual job command, wrapper commands, setup commands, and any batch directives

Definition at line 830 of file backends.py.

submit() [1/2]

def submit (   self,   job  )

inherited

Base method for submitting collection jobs to the backend type. This MUST be
implemented for a correctly written backend class deriving from Backend().

Reimplemented in Local.

Definition at line 770 of file backends.py.

submit() [2/2]

def submit	(		self,
			job,
			check_can_submit = True,
			jobs_per_check = 100
	)

Definition at line 1172 of file backends.py.

Member Data Documentation

default_global_job_limit

int default_global_job_limit = 1000

static

Default global limit on the total number of submitted/running jobs that the user can have.

This limit will not affect the total number of jobs that are eventually submitted. But the jobs won't actually be submitted until this limit can be respected i.e. until the number of total jobs in the Batch system goes down. Since we actually submit in chunks of N jobs, before checking this limit value again, this value needs to be a little lower than the real batch system limit. Otherwise you could accidentally go over during the N job submission if other processes are checking and submitting concurrently. This is quite common for the first submission of jobs from parallel calibrations.

Note that if there are other jobs already submitted for your account, then these will count towards this limit.

Definition at line 1123 of file backends.py.

global_job_limit

global_job_limit

The active job limit.

Init method for Batch Backend. Does some basic default setup.

This is 'global' because we want to prevent us accidentally submitting too many jobs from all current and previous submission scripts.

Definition at line 1134 of file backends.py.

sleep_between_submission_checks

sleep_between_submission_checks

Seconds we wait before checking if we can submit a list of jobs.

Only relevant once we hit the global limit of active jobs, which is a lot usually.

Definition at line 1137 of file backends.py.

---

The documentation for this class was generated from the following file:

• calibration/scripts/caf/backends.py