Inheritance diagram for SubJob:

Collaboration diagram for SubJob:

Public Member Functions
def	__init__ (self, job, subjob_id, input_files=None)

def	output_dir (self)

def	working_dir (self)

def	name (self)

def	status (self)

def	status (self, status)

def	subjobs (self)

def	job_dict (self)

def	__getattr__ (self, attribute)

def	__repr__ (self)

def	input_sandbox_files (self)

def	input_sandbox_files (self, value)

def	working_dir (self, value)

def	output_dir (self, value)

def	input_files (self)

def	input_files (self, value)

def	ready (self)

def	update_status (self)

def	create_subjob (self, i, input_files=None, args=None)

def	max_subjobs (self)

def	max_subjobs (self, value)

def	max_files_per_subjob (self)

def	max_files_per_subjob (self, value)

def	dump_to_json (self, file_path)

def	from_json (cls, file_path)

def	dump_input_data (self)

def	copy_input_sandbox_files_to_working_dir (self)

def	check_input_data_files (self)

def	full_command (self)

def	append_current_basf2_setup_cmds (self)

Public Attributes
	id
	Id of Subjob.

	parent
	Job() instance of parent to this SubJob.

	input_files
	Input files specific to this subjob.

	result
	The result object of this SubJob. More...

	args
	Arguments specific to this SubJob.

	name
	Job object's name. More...

	splitter
	The `SubjobSplitter` used to create subjobs if necessary.

	input_sandbox_files
	Files to be copied directly into the working directory (`pathlib.Path`). More...

	working_dir
	Working directory of the job (`pathlib.Path`). More...

	output_dir
	Output directory (`pathlib.Path`), where we will download our output_files to. More...

	output_patterns
	Files that we produce during the job and want to be returned. More...

	cmd
	Command and arguments as a list that wil be run by the job on the backend.

	setup_cmds
	Bash commands to run before the main self.cmd (mainly used for batch system setup)

	backend_args
	Config dictionary for the backend to use when submitting the job. More...

	subjobs
	dict of subjobs assigned to this job

	status
	Not a real attribute, it's a property.

Static Public Attributes
dictionary	statuses = {"init": 0, "submitted": 1, "running": 2, "failed": 3, "completed": 4}
	Allowed Job status dictionary. More...

list	exit_statuses = ["failed", "completed"]
	Job statuses that correspond to the Job being finished (successfully or not)

Private Member Functions
def	_get_overall_status_from_subjobs (self)

Private Attributes
	_status
	Status of the subjob.

	_output_dir

	_working_dir

	_input_sandbox_files

	_input_files

Detailed Description

This mini-class simply holds basic information about which subjob you are
and a reference to the parent Job object to be able to access the main data there.
Rather than replicating all of the parent job's configuration again.

Definition at line 632 of file backends.py.

Constructor & Destructor Documentation

◆ init()

def __init__	(	self,
		job,
		subjob_id,
		input_files = `None`
	)

Definition at line 639 of file backends.py.

Member Function Documentation

◆ getattr()

def __getattr__	(	self,
		attribute
	)

Since a SubJob uses attributes from the parent Job, everything simply accesses the Job attributes
unless otherwise specified.

Definition at line 714 of file backends.py.

◆ repr()

def __repr__ ( self )

Reimplemented from Job.

Definition at line 721 of file backends.py.

◆ append_current_basf2_setup_cmds()

def append_current_basf2_setup_cmds ( self )

inherited

This adds simple setup commands like ``source /path/to/tools/b2setup`` to your `Job`.
It should detect if you are using a local release or CVMFS and append the correct commands
so that the job will have the same basf2 release environment. It should also detect
if a local release is not compiled with the ``opt`` option.

Note that this *doesn't mean that every environment variable is inherited* from the submitting
process environment.

Definition at line 606 of file backends.py.

◆ check_input_data_files()

def check_input_data_files ( self )

inherited

Check the input files and make sure that there aren't any duplicates.
Also check if the files actually exist if possible.

Definition at line 564 of file backends.py.

◆ copy_input_sandbox_files_to_working_dir()

def copy_input_sandbox_files_to_working_dir ( self )

inherited

Get all of the requested files for the input sandbox and copy them to the working directory.
Files like the submit.sh or input_data.json are not part of this process.

Definition at line 553 of file backends.py.

◆ create_subjob()

def create_subjob	(	self,
		i,
		input_files = `None`,
		args = `None`
	)

inherited

Creates a subjob Job object that references that parent Job.
Returns the SubJob object at the end.

Definition at line 411 of file backends.py.

◆ dump_input_data()

def dump_input_data ( self )

inherited

Dumps the `Job.input_files` attribute to a JSON file. input_files should be a list of
string URI objects.

Definition at line 545 of file backends.py.

◆ dump_to_json()

def dump_to_json	(	self,
		file_path
	)

inherited

Dumps the Job object configuration to a JSON file so that it can be read in again later.

Parameters:
  file_path(`Path`): The filepath we'll dump to

Definition at line 508 of file backends.py.

◆ full_command()

def full_command ( self )

inherited

Returns:
    str: The full command that this job will run including any arguments.

Definition at line 594 of file backends.py.

◆ job_dict()

def job_dict ( self )

Returns:
    dict: A JSON serialisable representation of the `SubJob`. `Path` objects are converted to
    `string` via ``Path.as_posix()``. Since Subjobs inherit most of the parent job's config
    we only output the input files and arguments that are specific to this subjob and no other details.

Reimplemented from Job.

Definition at line 701 of file backends.py.

◆ name()

def name ( self )

Getter for name of SubJob. Accesses the parent Job name to infer this.

Definition at line 670 of file backends.py.

◆ output_dir()

def output_dir ( self )

Getter for output_dir of SubJob. Accesses the parent Job output_dir to infer this.

Reimplemented from Job.

Definition at line 659 of file backends.py.

◆ ready()

def ready ( self )

inherited

Returns whether or not the Job has finished. If the job has subjobs then it will return true when they are all finished.
It will return False as soon as it hits the first failure. Meaning that you cannot guarantee that all subjobs will have
their status updated when calling this method. Instead use :py:meth:`update_status` to update all statuses if necessary.

Definition at line 388 of file backends.py.

◆ status() [1/2]

def status ( self )

Returns the status of this SubJob.

Reimplemented from Job.

Definition at line 675 of file backends.py.

◆ status() [2/2]

def status	(	self,
		status
	)

Sets the status of this Job.

Reimplemented from Job.

Definition at line 682 of file backends.py.

◆ subjobs()

def subjobs ( self )

A subjob cannot have subjobs. Always return empty list.

Definition at line 694 of file backends.py.

◆ update_status()

def update_status ( self )

inherited

Calls :py:meth:`update_status` on the job's result. The result object should update all of the subjobs (if there are any)
in the best way for the type of result object/backend.

Definition at line 400 of file backends.py.

◆ working_dir()

def working_dir ( self )

Getter for working_dir of SubJob. Accesses the parent Job working_dir to infer this.

Reimplemented from Job.

Definition at line 665 of file backends.py.

Member Data Documentation

◆ backend_args

backend_args

inherited

Config dictionary for the backend to use when submitting the job.

Saves us from having multiple attributes that may or may not be used.

Definition at line 359 of file backends.py.

◆ input_sandbox_files

input_sandbox_files

inherited

Files to be copied directly into the working directory (pathlib.Path).

Not the input root files, those should be in Job.input_files.

Definition at line 342 of file backends.py.

◆ name

name

inherited

Job object's name.

Only descriptive, not necessarily unique.

Definition at line 335 of file backends.py.

◆ output_dir

output_dir

inherited

Output directory (pathlib.Path), where we will download our output_files to.

Default is '.'

Definition at line 346 of file backends.py.

◆ output_patterns

output_patterns

inherited

Files that we produce during the job and want to be returned.

Can use wildcard (*)

Definition at line 348 of file backends.py.

◆ result

result

The result object of this SubJob.

Only filled once it is is submitted to a backend since the backend creates a special result class depending on its type.

Definition at line 652 of file backends.py.

◆ statuses

dictionary statuses = {"init": 0, "submitted": 1, "running": 2, "failed": 3, "completed": 4}

staticinherited

Allowed Job status dictionary.

The key is the status name and the value is its level. The lowest level out of all subjobs is the one that is the overal status of the overall job.

Definition at line 326 of file backends.py.

◆ working_dir

working_dir

inherited

Working directory of the job (pathlib.Path).

Default is '.', mostly used in Local() backend

Definition at line 344 of file backends.py.

The documentation for this class was generated from the following file:

calibration/scripts/caf/backends.py

Public Member Functions

Public Attributes

Static Public Attributes

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ __getattr__()

◆ __repr__()

◆ append_current_basf2_setup_cmds()

◆ check_input_data_files()

◆ copy_input_sandbox_files_to_working_dir()

◆ create_subjob()

◆ dump_input_data()

◆ dump_to_json()

◆ full_command()

◆ job_dict()

◆ name()

◆ output_dir()

◆ ready()

◆ status() [1/2]

◆ status() [2/2]

◆ subjobs()

◆ update_status()

◆ working_dir()

Member Data Documentation

◆ backend_args

◆ input_sandbox_files

◆ name

◆ output_dir

◆ output_patterns

◆ result

◆ statuses

◆ working_dir

◆ init()

◆ getattr()

◆ repr()