RecoTrackQEDataCollectionTask Class Reference

Inheritance diagram for RecoTrackQEDataCollectionTask:

Public Member Functions
def	get_records_file_name (self, n_events=None, random_seed=None, recotrack_option=None)
	Filename of the recorded/collected data for the final QE MVA training.
def	get_input_files (self, n_events=None, random_seed=None)
def	requires (self)
def	output (self)
def	create_path (self)

Static Public Attributes
b2luigi	n_events = b2luigi.IntParameter()
	Number of events to generate.
b2luigi	experiment_number = b2luigi.IntParameter()
	Experiment number of the conditions database, e.g.
b2luigi	random_seed = b2luigi.Parameter()
	Random basf2 seed used by the GenerateSimTask.
b2luigi	cdc_training_target = b2luigi.Parameter()
	Feature/variable to use as truth label for the CDC track quality estimator.
b2luigi	recotrack_option
	RecoTrack option, use string that is additive: deleteCDCQI0XY (= deletes CDCTracks with CDC-QI below 0.XY), useCDC (= uses trained CDC stored in datafiles/), useVXD (uses trained VXD stored in datafiles/), noVXD (= doesn't use the VXD MVA at all)
b2luigi	fast_bdt_option
	Hyperparameter option of the FastBDT algorithm.
str	queue = 'l'
	specify queue.

Detailed Description

Collect variables/features from the reco track reconstruction including the
fit and write them to a ROOT file.

These variables are to be used as labelled training data for the MVA
classifier which is the MVA track quality estimator.  The collected
variables include the classifier outputs from the VXD and CDC quality
estimators, namely the CDC and VXD quality indicators, combined with fit,
merger, timing, energy loss information etc.  This task requires the
subdetector quality estimators to be trained.

Definition at line 911 of file combined_quality_estimator_teacher.py.

Member Function Documentation

create_path()

def create_path

(

self

)

Create basf2 reconstruction path that should mirror the default path
from ``add_tracking_reconstruction()``, but with modules for the VXD QE
and CDC QE application and for collection of variables for the reco
track quality estimator.

Definition at line 1038 of file combined_quality_estimator_teacher.py.

    def create_path(self):
        """
        Create basf2 reconstruction path that should mirror the default path
        from ``add_tracking_reconstruction()``, but with modules for the VXD QE
        and CDC QE application and for collection of variables for the reco
        track quality estimator.
        """
        path = basf2.create_path()
        inputFileNames = self.get_input_files()
        path.add_module(
            "RootInput",
            inputFileNames=inputFileNames,
        )
        path.add_module("Gearbox")
 
        # First add tracking reconstruction with default quality estimation modules
        mvaCDC = True
        mvaVXD = True
        if 'noCDC' in self.recotrack_option:
            mvaCDC = False
        if 'noVXD' in self.recotrack_option:
            mvaVXD = False
        if 'DATA' in self.random_seed:
            from rawdata import add_unpackers
            add_unpackers(path)
        tracking.add_tracking_reconstruction(path, add_cdcTrack_QI=mvaCDC, add_vxdTrack_QI=mvaVXD, add_recoTrack_QI=True)
 
        # if data shall be processed check if newly trained mva files are available. Otherwise use default ones (CDB payloads):
        # if useCDC/VXD is specified, use the identifier lying in datafiles/ Otherwise, replace weightfile identifiers from defaults
        # (CDB payloads) to new weightfiles created by this b2luigi script
        if ('DATA' in self.random_seed or 'useCDC' in self.recotrack_option) and 'noCDC' not in self.recotrack_option:
            cdc_identifier = 'datafiles/' + \
                             CDCQETeacherTask.get_weightfile_xml_identifier(CDCQETeacherTask, fast_bdt_option=self.fast_bdt_option)
            if os.path.exists(cdc_identifier):
                replace_cdc_qi = True
            elif 'useCDC' in self.recotrack_option:
                raise ValueError(f"CDC QI Identifier not found: {cdc_identifier}")
            else:
                replace_cdc_qi = False
        elif 'noCDC' in self.recotrack_option:
            replace_cdc_qi = False
        else:
            cdc_identifier = self.get_input_file_names(
                CDCQETeacherTask.get_weightfile_xml_identifier(
                    CDCQETeacherTask, fast_bdt_option=self.fast_bdt_option))[0]
            replace_cdc_qi = True
        if ('DATA' in self.random_seed or 'useVXD' in self.recotrack_option) and 'noVXD' not in self.recotrack_option:
            vxd_identifier = 'datafiles/' + \
                VXDQETeacherTask.get_weightfile_xml_identifier(VXDQETeacherTask, fast_bdt_option=self.fast_bdt_option)
            if os.path.exists(vxd_identifier):
                replace_vxd_qi = True
            elif 'useVXD' in self.recotrack_option:
                raise ValueError(f"VXD QI Identifier not found: {vxd_identifier}")
            else:
                replace_vxd_qi = False
        elif 'noVXD' in self.recotrack_option:
            replace_vxd_qi = False
        else:
            vxd_identifier = self.get_input_file_names(
                VXDQETeacherTask.get_weightfile_xml_identifier(
                    VXDQETeacherTask, fast_bdt_option=self.fast_bdt_option))[0]
            replace_vxd_qi = True
 
        cdc_qe_mva_filter_parameters = None
        # if tracks below a certain CDC QI index shall be deleted online, this needs to be specified in the filter parameters.
        # this is also possible in case of the default (CBD) payloads.
        if 'deleteCDCQI' in self.recotrack_option:
            cut_index = self.recotrack_option.find('deleteCDCQI') + len('deleteCDCQI')
            cut = int(self.recotrack_option[cut_index:cut_index+3])/100.
            if replace_cdc_qi:
                cdc_qe_mva_filter_parameters = {
                    "identifier": cdc_identifier, "cut": cut}
            else:
                cdc_qe_mva_filter_parameters = {
                    "cut": cut}
        elif replace_cdc_qi:
            cdc_qe_mva_filter_parameters = {
                "identifier": cdc_identifier}
        if cdc_qe_mva_filter_parameters is not None:
            # if no cut is specified, the default value is at zero and nothing is deleted.
            basf2.set_module_parameters(
                path,
                name="TFCDC_TrackQualityEstimator",
                filterParameters=cdc_qe_mva_filter_parameters,
                deleteTracks=True,
                resetTakenFlag=True
                )
        if replace_vxd_qi:
            basf2.set_module_parameters(
                path,
                name="VXDQualityEstimatorMVA",
                WeightFileIdentifier=vxd_identifier)
 
        # Replace final quality estimator module by training data collector module
        track_qe_module_name = "TrackQualityEstimatorMVA"
        module_found = False
        new_path = basf2.create_path()
        for module in path.modules():
            if module.name() != track_qe_module_name:
                if not module.name == 'TrackCreator':
                    new_path.add_module(module)
            else:
                # the TrackCreator needs to be conducted before the Collector such that
                # MDSTTracks are related to RecoTracks and d0 and z0 can be read out
                new_path.add_module(
                    'TrackCreator',
                    pdgCodes=[
                        211,
                        321,
                        2212],
                    recoTrackColName='RecoTracks',
                    trackColName='MDSTTracks')  # , useClosestHitToIP=True, useBFieldAtHit=True)
                new_path.add_module(
                    "TrackQETrainingDataCollector",
                    TrainingDataOutputName=self.get_output_file_name(self.get_records_file_name()),
                    collectEventFeatures=True,
                    SVDPlusCDCStandaloneRecoTracksStoreArrayName="SVDPlusCDCStandaloneRecoTracks",
                )
                module_found = True
        if not module_found:
            raise KeyError(f"No module {track_qe_module_name} found in path")
        path = new_path
        return path
 
 

get_input_files()

def get_input_files	(		self,
			n_events = None,
			random_seed = None
	)

Get input file names depending on the use case: If they already exist, search in
the corresponding folders, for data check the specified list and if they are created
in the same run, check for the task that produced them.

Definition at line 973 of file combined_quality_estimator_teacher.py.

    def get_input_files(self, n_events=None, random_seed=None):
        """
        Get input file names depending on the use case: If they already exist, search in
        the corresponding folders, for data check the specified list and if they are created
        in the same run, check for the task that produced them.
        """
        if n_events is None:
            n_events = self.n_events
        if random_seed is None:
            random_seed = self.random_seed
        if "USESIM" in random_seed:
            if 'USESIMBB' in random_seed:
                random_seed = 'BBBAR_' + random_seed.split("_", 1)[1]
            elif 'USESIMEE' in random_seed:
                random_seed = 'BHABHA_' + random_seed.split("_", 1)[1]
            return ['datafiles/' + GenerateSimTask.output_file_name(GenerateSimTask,
                                                                    n_events=n_events, random_seed=random_seed)]
        elif "DATA" in random_seed:
            return MasterTask.datafiles
        else:
            return self.get_input_file_names(GenerateSimTask.output_file_name(
                GenerateSimTask, n_events=n_events, random_seed=random_seed))
 

get_records_file_name()

def get_records_file_name	(		self,
			n_events = None,
			random_seed = None,
			recotrack_option = None
	)

Filename of the recorded/collected data for the final QE MVA training.

Create output file name depending on number of events and production
mode that is specified in the random_seed string.

Definition at line 951 of file combined_quality_estimator_teacher.py.

    def get_records_file_name(self, n_events=None, random_seed=None, recotrack_option=None):
        """
        Create output file name depending on number of events and production
        mode that is specified in the random_seed string.
        """
        if n_events is None:
            n_events = self.n_events
        if random_seed is None:
            random_seed = self.random_seed
        if recotrack_option is None:
            recotrack_option = self.recotrack_option
        if 'rec' not in random_seed:
            random_seed += '_rec'
        if 'DATA' in random_seed:
            return 'qe_records_DATA_rec.root'
        else:
            if 'USESIMBB' in random_seed:
                random_seed = 'BBBAR_' + random_seed.split("_", 1)[1]
            elif 'USESIMEE' in random_seed:
                random_seed = 'BHABHA_' + random_seed.split("_", 1)[1]
            return 'qe_records_N' + str(n_events) + '_' + random_seed + '_' + recotrack_option + '.root'
 

output()

def output

(

self

)

Generate list of output files that the task should produce.
The task is considered finished if and only if the outputs all exist.

Definition at line 1031 of file combined_quality_estimator_teacher.py.

    def output(self):
        """
        Generate list of output files that the task should produce.
        The task is considered finished if and only if the outputs all exist.
        """
        yield self.add_to_output(self.get_records_file_name())
 

requires()

def requires

(

self

)

Generate list of luigi Tasks that this Task depends on.

Definition at line 996 of file combined_quality_estimator_teacher.py.

    def requires(self):
        """
        Generate list of luigi Tasks that this Task depends on.
        """
        if "USESIM" in self.random_seed or "DATA" in self.random_seed:
            for filename in self.get_input_files():
                yield CheckExistingFile(
                    filename=filename,
                    )
        else:
            yield SplitNMergeSimTask(
                bkgfiles_dir=MasterTask.bkgfiles_by_exp[self.experiment_number],
                random_seed=self.random_seed,
                n_events=self.n_events,
                experiment_number=self.experiment_number,
                )
        if "DATA" not in self.random_seed:
            if 'useCDC' not in self.recotrack_option and 'noCDC' not in self.recotrack_option:
                yield CDCQETeacherTask(
                    n_events_training=MasterTask.n_events_training,
                    experiment_number=self.experiment_number,
                    training_target=self.cdc_training_target,
                    process_type=self.random_seed.split("_", 1)[0],
                    exclude_variables=MasterTask.exclude_variables_cdc,
                    fast_bdt_option=self.fast_bdt_option,
                    )
            if 'useVXD' not in self.recotrack_option and 'noVXD' not in self.recotrack_option:
                yield VXDQETeacherTask(
                    n_events_training=MasterTask.n_events_training,
                    experiment_number=self.experiment_number,
                    process_type=self.random_seed.split("_", 1)[0],
                    exclude_variables=MasterTask.exclude_variables_vxd,
                    fast_bdt_option=self.fast_bdt_option,
                    )
 

Member Data Documentation

cdc_training_target

b2luigi cdc_training_target = b2luigi.Parameter()

static

Feature/variable to use as truth label for the CDC track quality estimator.

Definition at line 932 of file combined_quality_estimator_teacher.py.

experiment_number

b2luigi experiment_number = b2luigi.IntParameter()

static

Experiment number of the conditions database, e.g.

defines simulation geometry

Definition at line 927 of file combined_quality_estimator_teacher.py.

fast_bdt_option

b2luigi fast_bdt_option

static

Initial value:

=  b2luigi.ListParameter(
        
 
 
    )

Hyperparameter option of the FastBDT algorithm.

default are the FastBDT default values.

Definition at line 942 of file combined_quality_estimator_teacher.py.

n_events

b2luigi n_events = b2luigi.IntParameter()

static

Number of events to generate.

Definition at line 925 of file combined_quality_estimator_teacher.py.

queue

str queue = 'l'

static

specify queue.

E.g. choose between 'l' (long), 's' (short) or 'sx' (short, extra ram)

Definition at line 948 of file combined_quality_estimator_teacher.py.

random_seed

b2luigi random_seed = b2luigi.Parameter()

static

Random basf2 seed used by the GenerateSimTask.

It is further used to read of the production process to preserve clearness in the b2luigi output.

Definition at line 930 of file combined_quality_estimator_teacher.py.

recotrack_option

b2luigi recotrack_option

static

Initial value:

=  b2luigi.Parameter(
        
 
 
    )

RecoTrack option, use string that is additive: deleteCDCQI0XY (= deletes CDCTracks with CDC-QI below 0.XY), useCDC (= uses trained CDC stored in datafiles/), useVXD (uses trained VXD stored in datafiles/), noVXD (= doesn't use the VXD MVA at all)

Definition at line 936 of file combined_quality_estimator_teacher.py.

---

The documentation for this class was generated from the following file:

• tracking/scripts/tracking/train/combined_quality_estimator_teacher.py