Backend Class Reference

Inheritance diagram for Backend:

Public Member Functions
	__init__ (self, *, backend_args=None)
	submit (self, job)
	get_submit_script_path (self, job)

Public Attributes
dict	backend_args = {**self.default_backend_args, **backend_args}
	The backend args that will be applied to jobs unless the job specifies them itself.

Static Public Attributes
str	submit_script = "submit.sh"
	Default submission script name.
str	exit_code_file = "__BACKEND_CMD_EXIT_STATUS__"
	Default exit code file name.
dict	default_backend_args = {}
	Default backend_args.

Protected Member Functions
	_add_wrapper_script_setup (self, job, batch_file)
	_add_wrapper_script_teardown (self, job, batch_file)
	_create_parent_job_result (cls, parent)

Static Protected Member Functions
	_add_setup (job, batch_file)

Detailed Description

Abstract base class for a valid backend.
Classes derived from this will implement their own submission of basf2 jobs
to whatever backend they describe.
Some common methods/attributes go into this base class.

For backend_args the priority from lowest to highest is:

backend.default_backend_args -> backend.backend_args -> job.backend_args

Definition at line 773 of file backends.py.

Constructor & Destructor Documentation

__init__()

__init__	(		self,
		*	,
			backend_args = None )

Definition at line 791 of file backends.py.

    def __init__(self, *, backend_args=None):
        """
        """
        if backend_args is None:
            backend_args = {}
        
        self.backend_args = {**self.default_backend_args, **backend_args}
 

Member Function Documentation

_add_setup()

_add_setup ( job, batch_file )

staticprotected

Adds setup lines to the shell script file.

Definition at line 807 of file backends.py.

    def _add_setup(job, batch_file):
        """
        Adds setup lines to the shell script file.
        """
        for line in job.setup_cmds:
            print(line, file=batch_file)
 

_add_wrapper_script_setup()

_add_wrapper_script_setup ( self, job, batch_file )

protected

Adds lines to the submitted script that help with job monitoring/setup. Mostly here so that we can insert
`trap` statements for Ctrl-C situations.

Definition at line 814 of file backends.py.

    def _add_wrapper_script_setup(self, job, batch_file):
        """
        Adds lines to the submitted script that help with job monitoring/setup. Mostly here so that we can insert
        `trap` statements for Ctrl-C situations.
        """
        start_wrapper = f"""# ---
# trap ctrl-c and call ctrl_c()
trap '(ctrl_c 130)' SIGINT
trap '(ctrl_c 143)' SIGTERM
 
function write_exit_code() {{
  echo "Writing $1 to exit status file"
  echo "$1" > {self.exit_code_file}
  exit $1
}}
 
function ctrl_c() {{
  trap '' SIGINT SIGTERM
  echo "** Trapped Ctrl-C **"
  echo "$1" > {self.exit_code_file}
  exit $1
}}
# ---"""
        print(start_wrapper, file=batch_file)
 

_add_wrapper_script_teardown()

_add_wrapper_script_teardown ( self, job, batch_file )

protected

Adds lines to the submitted script that help with job monitoring/teardown. Mostly here so that we can insert
an exit code of the job cmd being written out to a file. Which means that we can know if the command was
successful or not even if the backend server/monitoring database purges the data about our job i.e. If PBS
removes job information too quickly we may never know if a job succeeded or failed without some kind of exit
file.

Definition at line 839 of file backends.py.

    def _add_wrapper_script_teardown(self, job, batch_file):
        """
        Adds lines to the submitted script that help with job monitoring/teardown. Mostly here so that we can insert
        an exit code of the job cmd being written out to a file. Which means that we can know if the command was
        successful or not even if the backend server/monitoring database purges the data about our job i.e. If PBS
        removes job information too quickly we may never know if a job succeeded or failed without some kind of exit
        file.
        """
        end_wrapper = """# ---
write_exit_code $?"""
        print(end_wrapper, file=batch_file)
 

_create_parent_job_result()

_create_parent_job_result ( cls, parent )

protected

We want to be able to call `ready()` on the top level `Job.result`. So this method needs to exist
so that a Job.result object actually exists. It will be mostly empty and simply updates subjob
statuses and allows the use of ready().

Reimplemented in HTCondor, Local, LSF, and PBS.

Definition at line 852 of file backends.py.

    def _create_parent_job_result(cls, parent):
        """
        We want to be able to call `ready()` on the top level `Job.result`. So this method needs to exist
        so that a Job.result object actually exists. It will be mostly empty and simply updates subjob
        statuses and allows the use of ready().
        """
        raise NotImplementedError
 

get_submit_script_path()

get_submit_script_path	(		self,
			job )

Construct the Path object of the bash script file that we will submit. It will contain
the actual job command, wrapper commands, setup commands, and any batch directives

Definition at line 860 of file backends.py.

    def get_submit_script_path(self, job):
        """
        Construct the Path object of the bash script file that we will submit. It will contain
        the actual job command, wrapper commands, setup commands, and any batch directives
        """
        return Path(job.working_dir, self.submit_script)
 
 

submit()

submit	(		self,
			job )

Base method for submitting collection jobs to the backend type. This MUST be
implemented for a correctly written backend class deriving from Backend().

Reimplemented in Batch, and Local.

Definition at line 800 of file backends.py.

    def submit(self, job):
        """
        Base method for submitting collection jobs to the backend type. This MUST be
        implemented for a correctly written backend class deriving from Backend().
        """
 

Member Data Documentation

backend_args

dict backend_args = {**self.default_backend_args, **backend_args}

The backend args that will be applied to jobs unless the job specifies them itself.

Definition at line 797 of file backends.py.

default_backend_args

dict default_backend_args = {}

static

Default backend_args.

Definition at line 789 of file backends.py.

exit_code_file

str exit_code_file = "__BACKEND_CMD_EXIT_STATUS__"

static

Default exit code file name.

Definition at line 787 of file backends.py.

submit_script

submit_script = "submit.sh"

static

Default submission script name.

Definition at line 785 of file backends.py.

---

The documentation for this class was generated from the following file:

• calibration/scripts/caf/backends.py