Cluster Class Reference

Public Member Functions
def	__init__ (self)
	The default constructor. More...
def	adjust_path (self, path)
	This method can be used if path names are different on submission and execution hosts. More...
def	available (self)
	The cluster should always be available to accept new jobs. More...
def	execute (self, Script job, options="", dry=False, tag="current")
	Takes a Script object and a string with options and runs it on the cluster, either with ROOT or with basf2, depending on the file type. More...
Tuple[bool, int]	is_job_finished (self, Script job)
	Checks whether the '.done'-file has been created for a job. More...
def	terminate (self, Script job)
	Terminate a running job.

Static Public Member Functions
def	is_supported ()
def	name ()
def	description ()

Public Attributes
	path
	The path, where the help files are being created Maybe there should be a special subfolder for them?
	logger
	Contains a reference to the logger-object from validate_basf2 Set up the logging functionality for the 'cluster execution'-Class, so we can log to validate_basf2.py's log what is going on in .execute and .is_finished.
	tools
	Path to the basf2 tools and central/local release.
	b2setup
	The command for b2setup (and b2code-option)

Private Member Functions
None	_cleanup (self, Script job)
str	_get_tmp_name (self, Script job)

Detailed Description

A class that provides the controls for running jobs on a (remote)
cluster. It provides two methods:
- is_job_finished(job): Returns True or False, depending on whether the job
    has finished execution
- execute(job): Takes a job and executes it by sending it to the cluster

Definition at line 25 of file clustercontrol.py.

Constructor & Destructor Documentation

__init__()

def __init__

(

self

)

The default constructor.

• Holds the current working directory, which is also the location of the shellscripts that are being sent to the cluster.
• Initializes a logger which writes to validate_basf2.py's log.
• Finds the revision of basf2 that will be set up on the cluster.

Definition at line 55 of file clustercontrol.py.

    def __init__(self):
        """!
        The default constructor.
        - Holds the current working directory, which is also the location of
          the shellscripts that are being sent to the cluster.
        - Initializes a logger which writes to validate_basf2.py's log.
        - Finds the revision of basf2 that will be set up on the cluster.
        """
 
        
        self.path = os.getcwd()
 
        
        self.logger = logging.getLogger("validate_basf2")
 
        # We need to set up the same environment on the cluster like on the
        # local machine. The information can be extracted from $BELLE2_TOOLS,
        # $BELLE2_RELEASE_DIR and $BELLE2_LOCAL_DIR
 
        
        self.tools = self.adjust_path(os.environ["BELLE2_TOOLS"])
        belle2_release_dir = os.environ.get("BELLE2_RELEASE_DIR", None)
        belle2_local_dir = os.environ.get("BELLE2_LOCAL_DIR", None)
 
        
        self.b2setup = "b2setup"
        if belle2_release_dir is not None:
            self.b2setup += " " + belle2_release_dir.split("/")[-1]
        if belle2_local_dir is not None:
            self.b2setup = (
                "MY_BELLE2_DIR="
                + self.adjust_path(belle2_local_dir)
                + " "
                + self.b2setup
            )
        if os.environ.get("BELLE2_OPTION") != "debug":
            self.b2setup += "; b2code-option " + os.environ.get("BELLE2_OPTION")
 
        # Write to log which revision we are using
        self.logger.debug(f"Setting up the following release: {self.b2setup}")
 
        # Define the folder in which the log of the cluster messages will be
        # stored
        clusterlog_dir = "./html/logs/__general__/"
        if not os.path.exists(clusterlog_dir):
            os.makedirs(clusterlog_dir)
 

Member Function Documentation

_cleanup()

None _cleanup (   self, Script  job  )

private

 Clean up after job has finished. 

Definition at line 237 of file clustercontrol.py.

_get_tmp_name()

str _get_tmp_name (   self, Script  job  )

private

 Name of temporary file used for job submission. 

Definition at line 242 of file clustercontrol.py.

adjust_path()

def adjust_path	(		self,
			path
	)

This method can be used if path names are different on submission and execution hosts.

Parameters

path	The past that needs to be adjusted

Returns

: The adjusted path

Definition at line 107 of file clustercontrol.py.

available()

def available

(

self

)

The cluster should always be available to accept new jobs.

Returns

: Will always return True if the function can be called

Definition at line 118 of file clustercontrol.py.

description()

def description ( )

static

Returns description of this job control

Definition at line 49 of file clustercontrol.py.

execute()

def execute	(		self,
		Script	job,
			options = "",
			dry = False,
			tag = "current"
	)

Takes a Script object and a string with options and runs it on the cluster, either with ROOT or with basf2, depending on the file type.

Parameters

job	The steering file object that should be executed
options	Options that will be given to the basf2 command
dry	Whether to perform a dry run or not
tag	The folder within the results directory

Returns

: None

Definition at line 126 of file clustercontrol.py.

is_job_finished()

Tuple[bool, int] is_job_finished	(		self,
		Script	job
	)

Checks whether the '.done'-file has been created for a job.

If so, it returns True, else it returns False. Also deletes the .done-File once it has returned True.

Parameters

job	The job of which we want to know if it finished

Returns

: (True if the job has finished, exit code). If we can't find the exit code in the '.done'-file, the returncode will be -654. If the job is not finished, the exit code is returned as 0.

Definition at line 246 of file clustercontrol.py.

is_supported()

def is_supported ( )

static

Check if the bsub command is available

Definition at line 35 of file clustercontrol.py.

name()

def name ( void  )

static

Returns name of this job contol

Definition at line 42 of file clustercontrol.py.

---

The documentation for this class was generated from the following file:

• validation/scripts/clustercontrol.py