Inheritance diagram for PickleHarvestingModule:

Public Member Functions
	barn (self)

	refine (self, crops)

	id (self)

	initialize (self)

	event (self)

	terminate (self)

	gather (self)

	prepare (self)

	peel (self, crop)

	pick (self, crop)

Static Public Member Functions
	create_crop_part_collection ()

	iter_store_obj (store_obj)

Public Attributes
	foreach = foreach
	Name of the StoreArray or iterable StoreObjPtr that contains the objects to be harvested.

	output_file_name = output_file_name
	Name of the ROOT output file to be generated.

	title = title or self.name()
	Name of this harvest.

	contact = contact
	Contact email address to be displayed on the validation page.

int	expert_level = self.default_expert_level if expert_level is None else expert_level
	Integer expert level that controls to detail of plots to be generated.

list	refiners = []
	A list of additional refiner instances to be executed on top of the refiner methods that are members of this class.

	show_results = show_results
	Switch to show the result ROOT file in a TBrowser on terminate.

	stash = self.barn()
	stash of the harvested crops (start with those in the barn)

	crops
	the dictionaries from peel

	raw_crops = raw_crops
	the dictionaries from peel as a numpy.array of doubles

Static Public Attributes
int	default_expert_level = 1
	The default value of expert_level if not specified explicitly by the caller.

Detailed Description

Overloaded harvester, that stores its data into a pandas data frame instead of a numpy array,
because they are more flexible when it comes to changing columns and value types to be stored.

Definition at line 22 of file variable_modules.py.

Member Function Documentation

◆ barn()

barn ( self )

Coroutine that receives the dictionaries of names and values from peel and store them into a pandas df.

Reimplemented from HarvestingModule.

Definition at line 28 of file variable_modules.py.

    def barn(self):
        """Coroutine that receives the dictionaries of names and values from peel and store them into a pandas df."""
        crop = (yield)
        crops = []
 
        if isinstance(crop, numbers.Number):
            try:
                while True:
                    crops.append({"value": crop})
                    # next crop
                    crop = (yield)
            except GeneratorExit:
                pass
 
        elif isinstance(crop, collections.abc.MutableMapping):
            try:
                while True:
                    crops.append(crop)
                    crop = (yield)
            except GeneratorExit:
                pass
 
        else:
            msg = f"Unrecognised crop {crop} of type {type(crop)}"
            raise ValueError(msg)
 
        
        self.crops = crops
 

◆ create_crop_part_collection()

create_crop_part_collection ( )

staticinherited

Create the storing objects for the crop values

Currently a numpy.array of doubles is used to store all values in memory.

Definition at line 279 of file harvesting.py.

    def create_crop_part_collection():
        """Create the storing objects for the crop values
 
        Currently a numpy.array of doubles is used to store all values in memory.
        """
        return array.array("d")
 

◆ event()

event ( self )

inherited

Event method of the module

* Does invoke the prepare method before the iteration starts.
* In each event fetch the StoreArray / iterable StoreObjPtr,
* Iterate through all instances
* Feed each instance to the pick method to decide it the instance is relevant
* Forward it to the peel method that should generated a dictionary of values
* Store each dictionary of values

Definition at line 239 of file harvesting.py.

    def event(self):
        """Event method of the module
 
        * Does invoke the prepare method before the iteration starts.
        * In each event fetch the StoreArray / iterable StoreObjPtr,
        * Iterate through all instances
        * Feed each instance to the pick method to decide it the instance is relevant
        * Forward it to the peel method that should generated a dictionary of values
        * Store each dictionary of values
        """
        self.prepare()
        stash = self.stash.send
        pick = self.pick
        peel = self.peel
        for crop in self.gather():
            if pick(crop):
                crop = peel(crop)
                if isinstance(crop, types.GeneratorType):
                    many_crops = crop
                    for crop in many_crops:
                        stash(crop)
                else:
                    stash(crop)
 

◆ gather()

gather ( self )

inherited

Iterator that yield the instances form the StoreArray / iterable StoreObj.

Yields
------
Object instances from the StoreArray, iterable StoreObj or the StoreObj itself
in case it is not iterable.

Definition at line 329 of file harvesting.py.

    def gather(self):
        """Iterator that yield the instances form the StoreArray / iterable StoreObj.
 
        Yields
        ------
        Object instances from the StoreArray, iterable StoreObj or the StoreObj itself
        in case it is not iterable.
        """
 
        registered_store_arrays = Belle2.PyStoreArray.list()
        registered_store_objs = Belle2.PyStoreObj.list()
 
        foreach = self.foreach
        foreach_is_store_obj = foreach in registered_store_objs
        foreach_is_store_array = foreach in registered_store_arrays
 
        if foreach is not None:
            if foreach_is_store_array:
                store_array = Belle2.PyStoreArray(self.foreach)
                yield from store_array
 
            elif foreach_is_store_obj:
                store_obj = Belle2.PyStoreObj(self.foreach)
                try:
                    yield from self.iter_store_obj(store_obj)
                except TypeError:
                    # Cannot iter the store object. Yield it instead.
                    yield store_obj.obj()
 
            else:
                msg = f"Name {self.foreach} does not refer to a valid object on the data store"
                raise KeyError(msg)
        else:
            yield None
 

◆ id()

id ( self )

inherited

Working around that name() is a method.

Exposing the name as a property using a different name

Definition at line 224 of file harvesting.py.

    def id(self):
        """Working around that name() is a method.
 
        Exposing the name as a property using a different name
        """
        return self.name()
 

◆ initialize()

initialize ( self )

inherited

Initialisation method of the module.

Prepares the receiver stash of objects to be harvestered.

Reimplemented in ClusterFilterValidationModule, ElossHarvestingModule, LegendreBinningValidationModule, SegmentFitValidationModule, Saving1stMVAData, Saving2ndMVAData, SegmentPairCreationValidationModule, EventwiseTrackingValidationModule, MCSideTrackingValidationModule, and PRSideTrackingValidationModule.

Definition at line 231 of file harvesting.py.

    def initialize(self):
        """Initialisation method of the module.
 
        Prepares the receiver stash of objects to be harvestered.
        """
        
        self.stash = self.barn()
 

◆ iter_store_obj()

iter_store_obj ( store_obj )

staticinherited

Obtain a iterator from a StoreObj

Repeatedly calls iter(store_obj) or store_obj.__iter__()
until the final iterator returns itself

Returns
-------
iterator of the StoreObj

Definition at line 443 of file harvesting.py.

    def iter_store_obj(store_obj):
        """Obtain a iterator from a StoreObj
 
        Repeatedly calls iter(store_obj) or store_obj.__iter__()
        until the final iterator returns itself
 
        Returns
        -------
        iterator of the StoreObj
        """
        iterable = store_obj.obj()
        last_iterable = None
        while iterable is not last_iterable:
            if hasattr(iterable, "__iter__"):
                iterable, last_iterable = iterable.__iter__(), iterable
            else:
                iterable, last_iterable = iter(iterable), iterable
        return iterable
 
 

◆ peel()

peel	(		self,
			crop )

inherited

Unpack the the instances and return and dictionary of names to values or
a generator of those dictionaries to be saved.

Returns
-------
dict(str -> float)
    Unpacked names and values
or

Yields
------
dict(str -> float)
    Unpacked names and values

Reimplemented in VxdCdcMergerHarvesterMCSide, VxdCdcMergerHarvesterPRSide, ClusterFilterValidationModule, FitValidationModule, VxdCdcPartFinderHarvester, MCParticleHarvester, MCTrajectoryHarvester, ReconstructionPositionHarvester, SeedsAnalyser, SegmentFakeRatesModule, SegmentFinderParameterExtractorModule, WrongRLInfoCounter, ElossHarvestingModule, LegendreBinningValidationModule, SegmentFitValidationModule, Saving1stMVAData, Saving2ndMVAData, SegmentPairCreationValidationModule, SummarizeTriggerResults, SummarizeTriggerVariables, EventwiseTrackingValidationModule, MCSideTrackingValidationModule, PRSideTrackingValidationModule, EventInfoHarvester, HitInfoHarvester, TrackInfoHarvester, and VxdCdcMergerHarvester.

Definition at line 371 of file harvesting.py.

    def peel(self, crop):
        """Unpack the the instances and return and dictionary of names to values or
        a generator of those dictionaries to be saved.
 
        Returns
        -------
        dict(str -> float)
            Unpacked names and values
        or
 
        Yields
        ------
        dict(str -> float)
            Unpacked names and values
 
        """
        return {"name": np.nan}
 

◆ pick()

pick	(		self,
			crop )

inherited

Indicate whether the instance should be forwarded to the peeling

Returns
-------
bool : Indicator if the instance is valuable in the current harvest.

Reimplemented in VxdCdcMergerHarvesterMCSide, ClusterFilterValidationModule, MCParticleHarvester, ElossHarvestingModule, LegendreBinningValidationModule, SegmentFitValidationModule, Saving1stMVAData, Saving2ndMVAData, SegmentPairCreationValidationModule, EventwiseTrackingValidationModule, MCSideTrackingValidationModule, PRSideTrackingValidationModule, and VxdCdcMergerHarvester.

Definition at line 389 of file harvesting.py.

    def pick(self, crop):
        """Indicate whether the instance should be forwarded to the peeling
 
        Returns
        -------
        bool : Indicator if the instance is valuable in the current harvest.
        """
        return True
 

◆ prepare()

prepare ( self )

inherited

Default implementation of prepare.

Can be overridden by subclasses.

Reimplemented in ClusterFilterValidationModule, FitValidationModule, SegmentFakeRatesModule, ElossHarvestingModule, LegendreBinningValidationModule, SegmentFitValidationModule, Saving1stMVAData, Saving2ndMVAData, SegmentPairCreationValidationModule, MCSideTrackingValidationModule, and PRSideTrackingValidationModule.

Definition at line 364 of file harvesting.py.

    def prepare(self):
        """Default implementation of prepare.
 
        Can be overridden by subclasses.
        """
        return
 

◆ refine()

refine	(		self,
			crops )

Receive the gathered crops and saves them into a ROOT file.

Reimplemented from HarvestingModule.

Definition at line 57 of file variable_modules.py.

    def refine(self, crops):
        """Receive the gathered crops and saves them into a ROOT file."""
        with open(self.output_file_name, "wb") as f:
            pickle.dump(crops, f)
 
 

◆ terminate()

terminate ( self )

inherited

Termination method of the module.

Finalize the collected crops.
Start the refinement.

Reimplemented in ClusterFilterValidationModule.

Definition at line 263 of file harvesting.py.

    def terminate(self):
        """Termination method of the module.
 
        Finalize the collected crops.
        Start the refinement.
        """
 
        self.stash.close()
        del self.stash
 
        try:
            self.refine(self.crops)
        except AttributeError:
            pass
 

Member Data Documentation

◆ contact

contact = contact

inherited

Contact email address to be displayed on the validation page.

Definition at line 211 of file harvesting.py.

◆ crops

crops

inherited

the dictionaries from peel

Definition at line 274 of file harvesting.py.

◆ default_expert_level

int default_expert_level = 1

staticinherited

The default value of expert_level if not specified explicitly by the caller.

Definition at line 156 of file harvesting.py.

◆ expert_level

int expert_level = self.default_expert_level if expert_level is None else expert_level

inherited

Integer expert level that controls to detail of plots to be generated.

Definition at line 214 of file harvesting.py.

◆ foreach

foreach = foreach

inherited

Name of the StoreArray or iterable StoreObjPtr that contains the objects to be harvested.

Definition at line 196 of file harvesting.py.

◆ output_file_name

output_file_name = output_file_name

inherited

Name of the ROOT output file to be generated.

Definition at line 199 of file harvesting.py.

◆ raw_crops

raw_crops = raw_crops

inherited

the dictionaries from peel as a numpy.array of doubles

Definition at line 325 of file harvesting.py.

◆ refiners

list refiners = []

inherited

A list of additional refiner instances to be executed on top of the refiner methods that are members of this class.

Definition at line 218 of file harvesting.py.

◆ show_results

show_results = show_results

inherited

Switch to show the result ROOT file in a TBrowser on terminate.

Definition at line 221 of file harvesting.py.

◆ stash

stash = self.barn()

inherited

stash of the harvested crops (start with those in the barn)

Definition at line 237 of file harvesting.py.

◆ title

title = title or self.name()

inherited

Name of this harvest.

Title particle of this harvest

Definition at line 208 of file harvesting.py.

The documentation for this class was generated from the following file:

hlt/softwaretrigger/scripts/softwaretrigger/variable_modules.py

Public Member Functions

Static Public Member Functions

Public Attributes

Static Public Attributes

Detailed Description

Member Function Documentation

◆ barn()

◆ create_crop_part_collection()

◆ event()

◆ gather()

◆ id()

◆ initialize()

◆ iter_store_obj()

◆ peel()

◆ pick()

◆ prepare()

◆ refine()

◆ terminate()

Member Data Documentation

◆ contact

◆ crops

◆ default_expert_level

◆ expert_level

◆ foreach

◆ output_file_name

◆ raw_crops

◆ refiners

◆ show_results

◆ stash

◆ title