Inheritance diagram for Cluster:

Collaboration diagram for Cluster:

Public Member Functions
def	__init__ (self)
	The default constructor. More...

def	adjust_path (self, str path)
	This method can be used if path names are different on submission and execution hosts. More...

def	available (self)
	The cluster should always be available to accept new jobs. More...

def	execute (self, Script job, options="", dry=False, tag="current")
	Takes a Script object and a string with options and runs it on the cluster, either with ROOT or with basf2, depending on the file type. More...

def	is_job_finished (self, Script job)
	Checks whether the '.done'-file has been created for a job. More...

str	createDoneFileName (self, Script job)
	Generate the file name used for the done output.

def	prepareSubmission (self, Script job, options, tag)
	Setup output folders and create the wrapping shell script. More...

def	checkDoneFile (self, job)
	Checks whether the '.done'-file has been created for a job. More...

def	terminate (self, Script job)
	Terminate running job.

def	adjust_path (self, path)
	This method can be used if path names are different on submission and execution hosts. More...

Static Public Member Functions
def	is_supported ()

def	name ()

def	description ()

Public Attributes
	native_spec
	The command to submit a job. More...

	requirement_vmem
	required vmem by the job in GB, required on DESY NAF, otherwise jobs get killed due to memory consumption

	requirement_storage
	the storage IO in GB which can be performed by each job. More...

	queuename
	Queue best suitable for execution at DESY NAF.

	path
	The default constructor. More...

	logger
	Contains a reference to the logger-object from validate_basf2 Set up the logging functionality for the 'cluster execution'-Class, so we can log to validate_basf2.py's log what is going on in .execute and .is_finished.

	tools
	Path to the basf2 tools and central/local release.

	b2setup
	The command for b2setup (and b2code-option)

	clusterlog
	The file object to which all cluster messages will be written.

Detailed Description

A class that provides the controls for running jobs on a (remote)
Sun Grid Engine cluster. It provides two methods:
- is_job_finished(job): Returns True or False, depending on whether the job
    has finished execution
- execute(job): Takes a job and executes it by sending it to the cluster

Definition at line 19 of file clustercontroldrmaa.py.

Constructor & Destructor Documentation

◆ init()

def __init__ ( self )

The default constructor.

Holds the current working directory, which is also the location of the shellscripts that are being sent to the cluster.
Initializes a logger which writes to validate_basf2.py's log.
Finds the revision of basf2 that will be set up on the cluster.

Reimplemented from ClusterBase.

Definition at line 62 of file clustercontroldrmaa.py.

     def __init__(self):
         """!
         The default constructor.
         - Holds the current working directory, which is also the location of
           the shellscripts that are being sent to the cluster.
         - Initializes a logger which writes to validate_basf2.py's log.
         - Finds the revision of basf2 that will be set up on the cluster.
         """
  
         
         self.native_spec = (
             "-l h_vmem={requirement_vmem}G,h_fsize={"
             "requirement_storage}G "
             "-q {queuename} -V"
         )
  
         
         self.requirement_vmem = 4
  
         
         self.requirement_storage = 50
  
         
         self.queuename = "short.q"
  
         # call the base constructor, which will setup the batch cluster
         # common stuff
         super().__init__()
  

Member Function Documentation

◆ adjust_path() [1/2]

def adjust_path	(	self,
		path
	)

inherited

This method can be used if path names are different on submission and execution hosts.

Parameters

path	The past that needs to be adjusted

Returns: : The adjusted path

Definition at line 177 of file clustercontrolbase.py.

◆ adjust_path() [2/2]

def adjust_path	(		self,
		str	path
	)

This method can be used if path names are different on submission and execution hosts.

Parameters

path	The past that needs to be adjusted

Returns: : The adjusted path

Definition at line 96 of file clustercontroldrmaa.py.

◆ available()

def available ( self )

The cluster should always be available to accept new jobs.

Returns: : Will always return True if the function can be called

Definition at line 107 of file clustercontroldrmaa.py.

◆ checkDoneFile()

def checkDoneFile	(	self,
		job
	)

inherited

Checks whether the '.done'-file has been created for a job.

If so, it returns True, else it returns False in the first part of the tuple. Also deletes the .done-File it if exists. The second entry in the tuple will be the exit code read from the done file

Definition at line 139 of file clustercontrolbase.py.

◆ description()

def description ( )

static

Returns description of this job control

Definition at line 56 of file clustercontroldrmaa.py.

◆ execute()

def execute	(		self,
		Script	job,
			options = `""`,
			dry = `False`,
			tag = `"current"`
	)

Takes a Script object and a string with options and runs it on the cluster, either with ROOT or with basf2, depending on the file type.

Parameters

job	The steering file object that should be executed
options	Options that will be given to the basf2 command
dry	Whether to perform a dry run or not
tag	The folder within the results directory

Returns: : None

Definition at line 115 of file clustercontroldrmaa.py.

◆ is_job_finished()

def is_job_finished	(		self,
		Script	job
	)

Checks whether the '.done'-file has been created for a job.

If so, it returns True, else it returns False. Also deletes the .done-File once it has returned True.

Parameters

job	The job of which we want to know if it finished

Returns: : (True if the job has finished, exit code). If we can't find the exit code in the '.done'-file, the returncode will be -654. If the job is not finished, the exit code is returned as 0.

Definition at line 164 of file clustercontroldrmaa.py.

◆ is_supported()

def is_supported ( )

static

Check if qsub is available

Definition at line 29 of file clustercontroldrmaa.py.

◆ name()

def name ( void )

static

Returns name of this job contol

Definition at line 49 of file clustercontroldrmaa.py.

◆ prepareSubmission()

def prepareSubmission	(		self,
		Script	job,
			options,
			tag
	)

inherited

Setup output folders and create the wrapping shell script.

Will return the full file name of the generated wrapper script.

Definition at line 82 of file clustercontrolbase.py.

Member Data Documentation

◆ native_spec

native_spec

The command to submit a job.

'LOGFILE' will be replaced by the actual log file name

Definition at line 73 of file clustercontroldrmaa.py.

◆ path

path

inherited

The default constructor.

The path, where the help files are being created Maybe there should be a special subfolder for them?

Definition at line 33 of file clustercontrolbase.py.

◆ requirement_storage

requirement_storage

the storage IO in GB which can be performed by each job.

By default, this is 3GB at DESY which is to small for some validation scripts

Definition at line 86 of file clustercontroldrmaa.py.

The documentation for this class was generated from the following file:

validation/scripts/clustercontroldrmaa.py

Public Member Functions

Static Public Member Functions

Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ adjust_path() [1/2]

◆ adjust_path() [2/2]

◆ available()

◆ checkDoneFile()

◆ description()

◆ execute()

◆ is_job_finished()

◆ is_supported()

◆ name()

◆ prepareSubmission()

Member Data Documentation

◆ native_spec

◆ path

◆ requirement_storage

◆ init()