Belle II Software development
SubjobSplitter Class Reference
Inheritance diagram for SubjobSplitter:
ArgumentsSplitter MaxFilesSplitter MaxSubjobsSplitter

Public Member Functions

 __init__ (self, *, arguments_generator=None)
 
 create_subjobs (self, job)
 
 assign_arguments (self, job)
 
 __repr__ (self)
 

Public Attributes

 arguments_generator = arguments_generator
 The ArgumentsGenerator used when creating subjobs.
 

Detailed Description

Abstract base class. This class handles the logic of creating subjobs for a `Job` object.
The `create_subjobs` function should be implemented and used to construct
the subjobs of the parent job object.

Parameters:
    arguments_generator (ArgumentsGenerator): Used to construct the generator function that will yield the argument
        tuple for each `SubJob`. The splitter will iterate through the generator each time `create_subjobs` is
        called. The `SubJob` will be sent into the generator with ``send(subjob)`` so that the generator can decide what
        arguments to return.

Definition at line 152 of file backends.py.

Constructor & Destructor Documentation

◆ __init__()

__init__ ( self,
* ,
arguments_generator = None )
Derived classes should call `super` to run this.

Definition at line 165 of file backends.py.

165 def __init__(self, *, arguments_generator=None):
166 """
167 Derived classes should call `super` to run this.
168 """
169
170 self.arguments_generator = arguments_generator
171

Member Function Documentation

◆ __repr__()

__repr__ ( self)
 

Definition at line 204 of file backends.py.

204 def __repr__(self):
205 """
206 """
207 return f"{self.__class__.__name__}"
208
209

◆ assign_arguments()

assign_arguments ( self,
job )
Use the `arguments_generator` (if one exists) to assign the argument tuples to the
subjobs.

Definition at line 178 of file backends.py.

178 def assign_arguments(self, job):
179 """
180 Use the `arguments_generator` (if one exists) to assign the argument tuples to the
181 subjobs.
182 """
183 if self.arguments_generator:
184 arg_gen = self.arguments_generator.generator
185 generating = True
186 for subjob in sorted(job.subjobs.values(), key=lambda sj: sj.id):
187 if generating:
188 try:
189 args = arg_gen.send(subjob)
190 except StopIteration:
191 B2ERROR(f"StopIteration called when getting args for {subjob}, "
192 "setting all subsequent subjobs to have empty argument tuples.")
193 args = tuple() # If our generator finishes before the subjobs we use empty argument tuples
194 generating = False
195 else:
196 args = tuple()
197 B2DEBUG(29, f"Arguments for {subjob}: {args}")
198 subjob.args = args
199 return
200
201 B2INFO(f"No ArgumentsGenerator assigned to the {self} so subjobs of {job} "
202 "won't automatically have arguments assigned.")
203

◆ create_subjobs()

create_subjobs ( self,
job )
Implement this method in derived classes to generate the `SubJob` objects.

Reimplemented in ArgumentsSplitter, MaxFilesSplitter, and MaxSubjobsSplitter.

Definition at line 173 of file backends.py.

173 def create_subjobs(self, job):
174 """
175 Implement this method in derived classes to generate the `SubJob` objects.
176 """
177

Member Data Documentation

◆ arguments_generator

arguments_generator = arguments_generator

The ArgumentsGenerator used when creating subjobs.

Definition at line 170 of file backends.py.


The documentation for this class was generated from the following file: