Inheritance diagram for PBS.PBSResult:

Public Member Functions
	__init__ (self, job, job_id)

	update_status (self)

	ready (self)

	get_exit_code_from_file (self)

Public Attributes
	job_id = job_id
	job id given by PBS

	job = job
	Job object for result.

	time_to_wait_for_exit_code_file = timedelta(minutes=5)
	After our first attempt to view the exit code file once the job is 'finished', how long should we wait for it to exist before timing out?

	exit_code_file_initial_time = None
	Time we started waiting for the exit code file to appear.

Static Public Attributes
dict	backend_code_to_status
	PBS statuses mapped to Job statuses.

Protected Member Functions
	_update_result_status (self, qstat_output)

	_get_status_from_output (self, output)

Protected Attributes
bool	_is_ready = False
	Quicker way to know if it's ready once it has already been found.

Detailed Description

Simple class to help monitor status of jobs submitted by `PBS` Backend.

You pass in a `Job` object (or `SubJob`) and job id from a qsub command.
When you call the `ready` method it runs bjobs to see whether or not the job has finished.

Definition at line 1425 of file backends.py.

Constructor & Destructor Documentation

◆ init()

__init__	(	self,
		job,
		job_id )

Pass in the job object and the job id to allow the result to do monitoring and perform
post processing of the job.

Definition at line 1445 of file backends.py.

        def __init__(self, job, job_id):
            """
            Pass in the job object and the job id to allow the result to do monitoring and perform
            post processing of the job.
            """
            super().__init__(job)
            
            self.job_id = job_id
 

Member Function Documentation

◆ _get_status_from_output()

_get_status_from_output	(		self,
			output )

protected

Get status from output

Definition at line 1504 of file backends.py.

        def _get_status_from_output(self, output):
            """
            Get status from output
            """
            for job_info in output["JOBS"]:
                if job_info["Job_Id"] == self.job_id:
                    return job_info["job_state"]
            else:
                raise KeyError
 
    def can_submit(self, njobs=1):

◆ _update_result_status()

_update_result_status	(		self,
			qstat_output )

protected

Parameters:
        qstat_output (dict): The JSON output of a previous call to qstat which we can reuse to find the
        status of this job. Obviously you should only be passing a JSON dict that contains the 'Job_Id' and
        'job_state' information, otherwise it is useless.

Definition at line 1467 of file backends.py.

        def _update_result_status(self, qstat_output):
            """
            Parameters:
                    qstat_output (dict): The JSON output of a previous call to qstat which we can reuse to find the
                    status of this job. Obviously you should only be passing a JSON dict that contains the 'Job_Id' and
                    'job_state' information, otherwise it is useless.
 
            """
            try:
                backend_status = self._get_status_from_output(qstat_output)
            except KeyError:
                # If this happens then maybe the job id wasn't in the qstat_output argument because it finished.
                # Instead of failing immediately we try looking for the exit code file and then fail if it still isn't there.
                B2DEBUG(29, f"Checking of the exit code from file for {self.job}")
                try:
                    exit_code = self.get_exit_code_from_file()
                except FileNotFoundError:
                    waiting_time = datetime.now() - self.exit_code_file_initial_time
                    if self.time_to_wait_for_exit_code_file > waiting_time:
                        B2ERROR(f"Exit code file for {self.job} missing and we can't wait longer. Setting exit code to 1.")
                        exit_code = 1
                    else:
                        B2WARNING(f"Exit code file for {self.job} missing, will wait longer.")
                        return
                if exit_code:
                    backend_status = "E"
                else:
                    backend_status = "C"
 
            try:
                new_job_status = self.backend_code_to_status[backend_status]
            except KeyError as err:
                raise BackendError(f"Unidentified backend status found for {self.job}: {backend_status}") from err
 
            if new_job_status != self.job.status:
                self.job.status = new_job_status
 

◆ get_exit_code_from_file()

get_exit_code_from_file ( self )

inherited

Read the exit code file to discover the exit status of the job command. Useful fallback if the job is no longer
known to the job database (batch system purged it for example). Since some backends may take time to download
the output files of the job back to the working directory we use a time limit on how long to wait.

Definition at line 909 of file backends.py.

    def get_exit_code_from_file(self):
        """
        Read the exit code file to discover the exit status of the job command. Useful fallback if the job is no longer
        known to the job database (batch system purged it for example). Since some backends may take time to download
        the output files of the job back to the working directory we use a time limit on how long to wait.
        """
        if not self.exit_code_file_initial_time:
            self.exit_code_file_initial_time = datetime.now()
        exit_code_path = Path(self.job.working_dir, Backend.exit_code_file)
        with open(exit_code_path) as f:
            exit_code = int(f.read().strip())
            B2DEBUG(29, f"Exit code from file for {self.job} was {exit_code}")
            return exit_code
 
 

◆ ready()

ready ( self )

inherited

Returns whether or not this job result is known to be ready. Doesn't actually change the job status. Just changes
the 'readiness' based on the known job status.

Definition at line 888 of file backends.py.

    def ready(self):
        """
        Returns whether or not this job result is known to be ready. Doesn't actually change the job status. Just changes
        the 'readiness' based on the known job status.
        """
        B2DEBUG(29, f"Calling {self.job}.result.ready()")
        if self._is_ready:
            return True
        elif self.job.status in self.job.exit_statuses:
            self._is_ready = True
            return True
        else:
            return False
 

◆ update_status()

update_status ( self )

Update the job's (or subjobs') status by calling qstat.

Reimplemented from Result.

Definition at line 1454 of file backends.py.

        def update_status(self):
            """
            Update the job's (or subjobs') status by calling qstat.
            """
            B2DEBUG(29, f"Calling {self.job}.result.update_status()")
            # Get all jobs info and reuse it for each status update to minimise tie spent on this updating.
            qstat_output = PBS.qstat()
            if self.job.subjobs:
                for subjob in self.job.subjobs.values():
                    subjob.result._update_result_status(qstat_output)
            else:
                self._update_result_status(qstat_output)
 

Member Data Documentation

◆ _is_ready

bool _is_ready = False

protectedinherited

Quicker way to know if it's ready once it has already been found.

Saves a lot of calls to batch system commands.

Definition at line 881 of file backends.py.

◆ backend_code_to_status

dict backend_code_to_status

static

Initial value:

=  {"R": "running",
                                  "C": "completed",
                                  "FINISHED": "completed",
                                  "E": "failed",
                                  "H": "submitted",
                                  "Q": "submitted",
                                  "T": "submitted",
                                  "W": "submitted",
                                  "H": "submitted"
                                  }

PBS statuses mapped to Job statuses.

Definition at line 1434 of file backends.py.

◆ exit_code_file_initial_time

exit_code_file_initial_time = None

inherited

Time we started waiting for the exit code file to appear.

Definition at line 886 of file backends.py.

◆ job

job = job

inherited

Job object for result.

Definition at line 879 of file backends.py.

◆ job_id

job_id = job_id

job id given by PBS

Definition at line 1452 of file backends.py.

◆ time_to_wait_for_exit_code_file

time_to_wait_for_exit_code_file = timedelta(minutes=5)

inherited

After our first attempt to view the exit code file once the job is 'finished', how long should we wait for it to exist before timing out?

Definition at line 884 of file backends.py.

The documentation for this class was generated from the following file:

calibration/scripts/caf/backends.py

Public Member Functions

Public Attributes

Static Public Attributes

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ _get_status_from_output()

◆ _update_result_status()

◆ get_exit_code_from_file()

◆ ready()

◆ update_status()

Member Data Documentation

◆ _is_ready

◆ backend_code_to_status

◆ exit_code_file_initial_time

◆ job

◆ job_id

◆ time_to_wait_for_exit_code_file

◆ init()