Belle II Software light-2406-ragdoll
RootInputModule Class Reference

Module to read TTree data from file into the data store. More...

#include <RootInputModule.h>

Inheritance diagram for RootInputModule:
Collaboration diagram for RootInputModule:

Classes

struct  ReadStats
 for collecting statistics over multiple files. More...
 

Public Types

enum  EModulePropFlags {
  c_Input = 1 ,
  c_Output = 2 ,
  c_ParallelProcessingCertified = 4 ,
  c_HistogramManager = 8 ,
  c_InternalSerializer = 16 ,
  c_TerminateInAllProcesses = 32 ,
  c_DontCollectStatistics = 64
}
 Each module can be tagged with property flags, which indicate certain features of the module. More...
 
typedef ModuleCondition::EAfterConditionPath EAfterConditionPath
 Forward the EAfterConditionPath definition from the ModuleCondition.
 

Public Member Functions

 RootInputModule ()
 Constructor.
 
virtual ~RootInputModule ()
 Destructor.
 
virtual void initialize () override
 Initialize the Module.
 
virtual void event () override
 Running over all events.
 
virtual void terminate () override
 Is called at the end of your Module.
 
virtual std::vector< std::string > getFileNames (bool outputFiles=false) override
 Get list of input files, taking -i command line overrides into account.
 
virtual void beginRun ()
 Called when entering a new run.
 
virtual void endRun ()
 This method is called if the current run ends.
 
const std::string & getName () const
 Returns the name of the module.
 
const std::string & getType () const
 Returns the type of the module (i.e.
 
const std::string & getPackage () const
 Returns the package this module is in.
 
const std::string & getDescription () const
 Returns the description of the module.
 
void setName (const std::string &name)
 Set the name of the module.
 
void setPropertyFlags (unsigned int propertyFlags)
 Sets the flags for the module properties.
 
LogConfiggetLogConfig ()
 Returns the log system configuration.
 
void setLogConfig (const LogConfig &logConfig)
 Set the log system configuration.
 
void setLogLevel (int logLevel)
 Configure the log level.
 
void setDebugLevel (int debugLevel)
 Configure the debug messaging level.
 
void setAbortLevel (int abortLevel)
 Configure the abort log level.
 
void setLogInfo (int logLevel, unsigned int logInfo)
 Configure the printed log information for the given level.
 
void if_value (const std::string &expression, const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
 Add a condition to the module.
 
void if_false (const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
 A simplified version to add a condition to the module.
 
void if_true (const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
 A simplified version to set the condition of the module.
 
bool hasCondition () const
 Returns true if at least one condition was set for the module.
 
const ModuleConditiongetCondition () const
 Return a pointer to the first condition (or nullptr, if none was set)
 
const std::vector< ModuleCondition > & getAllConditions () const
 Return all set conditions for this module.
 
bool evalCondition () const
 If at least one condition was set, it is evaluated and true returned if at least one condition returns true.
 
std::shared_ptr< PathgetConditionPath () const
 Returns the path of the last true condition (if there is at least one, else reaturn a null pointer).
 
Module::EAfterConditionPath getAfterConditionPath () const
 What to do after the conditional path is finished.
 
std::vector< std::shared_ptr< Path > > getAllConditionPaths () const
 Return all condition paths currently set (no matter if the condition is true or not).
 
bool hasProperties (unsigned int propertyFlags) const
 Returns true if all specified property flags are available in this module.
 
bool hasUnsetForcedParams () const
 Returns true and prints error message if the module has unset parameters which the user has to set in the steering file.
 
const ModuleParamListgetParamList () const
 Return module param list.
 
template<typename T >
ModuleParam< T > & getParam (const std::string &name) const
 Returns a reference to a parameter.
 
bool hasReturnValue () const
 Return true if this module has a valid return value set.
 
int getReturnValue () const
 Return the return value set by this module.
 
std::shared_ptr< PathElementclone () const override
 Create an independent copy of this module.
 
std::shared_ptr< boost::python::list > getParamInfoListPython () const
 Returns a python list of all parameters.
 

Static Public Member Functions

static void exposePythonAPI ()
 Exposes methods of the Module class to Python.
 

Protected Member Functions

virtual void def_initialize ()
 Wrappers to make the methods without "def_" prefix callable from Python.
 
virtual void def_beginRun ()
 Wrapper method for the virtual function beginRun() that has the implementation to be used in a call from Python.
 
virtual void def_event ()
 Wrapper method for the virtual function event() that has the implementation to be used in a call from Python.
 
virtual void def_endRun ()
 This method can receive that the current run ends as a call from the Python side.
 
virtual void def_terminate ()
 Wrapper method for the virtual function terminate() that has the implementation to be used in a call from Python.
 
void setDescription (const std::string &description)
 Sets the description of the module.
 
void setType (const std::string &type)
 Set the module type.
 
template<typename T >
void addParam (const std::string &name, T &paramVariable, const std::string &description, const T &defaultValue)
 Adds a new parameter to the module.
 
template<typename T >
void addParam (const std::string &name, T &paramVariable, const std::string &description)
 Adds a new enforced parameter to the module.
 
void setReturnValue (int value)
 Sets the return value for this module as integer.
 
void setReturnValue (bool value)
 Sets the return value for this module as bool.
 
void setParamList (const ModuleParamList &params)
 Replace existing parameter list.
 

Private Types

typedef std::vector< DataStore::StoreEntry * > StoreEntries
 Vector of entries in the data store.
 

Private Member Functions

void readTree ()
 Actually performs the reading from the tree.
 
bool connectBranches (TTree *tree, DataStore::EDurability durability, StoreEntries *storeEntries)
 Connect branches of the given tree to the data store.
 
bool createParentStoreEntries ()
 Connect the parent trees and fill m_parentStoreEntries.
 
bool readParentTrees ()
 Read data of the current event from the parents.
 
void readPersistentEntry (long fileEntry)
 Loads given entry from persistent tree.
 
void entryNotFound (const std::string &entryOrigin, const std::string &name, bool fileChanged=true)
 Check if we warn the user or abort after an entry was missing after changing files.
 
void addEventListForIndexFile (const std::string &parentLfn)
 For index files, this creates TEventList/TEntryListArray to enable better cache use.
 
void realDataWorkaround (FileMetaData &metaData)
 Correct isMC flag for raw data recorded before experiment 8 run 2364.
 
std::list< ModulePtrgetModules () const override
 no submodules, return empty list
 
std::string getPathString () const override
 return the module name.
 
void setParamPython (const std::string &name, const boost::python::object &pyObj)
 Implements a method for setting boost::python objects.
 
void setParamPythonDict (const boost::python::dict &dictionary)
 Implements a method for reading the parameter values from a boost::python dictionary.
 

Private Attributes

std::string m_inputFileName
 File to read from.
 
std::vector< std::string > m_inputFileNames
 Files to read from.
 
std::vector< std::string > m_entrySequences
 The number sequences (e.g.
 
bool m_ignoreCommandLineOverride
 Ignore filename override from command line.
 
std::vector< std::string > m_branchNames [DataStore::c_NDurabilityTypes]
 Array for names of branches, that shall be written out.
 
std::vector< std::string > m_excludeBranchNames [DataStore::c_NDurabilityTypes]
 Array for names of branches that should NOT be written out.
 
unsigned int m_skipNEvents
 Can be set from steering file to skip the first N events.
 
int m_parentLevel
 Level of parent files to be read.
 
bool m_collectStatistics
 Collect statistics on amount of data read and print statistics (seperate for input & parent files) after processing.
 
std::vector< int > m_skipToEvent
 experiment, run, event number of first event to load
 
long m_nextEntry
 Next entry to be read in event tree.
 
long m_lastPersistentEntry
 last entry to be in persistent tree.
 
std::string m_lastParentFileLFN
 last parent file LFN seen.
 
TChain * m_tree
 TTree for event input.
 
TChain * m_persistent
 TTree for persistent input.
 
std::set< std::string > m_connectedBranches [DataStore::c_NDurabilityTypes]
 Already connected branches.
 
StoreEntries m_storeEntries
 Vector of DataStore entries of event durability that we are supposed to read in.
 
StoreEntries m_persistentStoreEntries
 Vector of DataStore entries of persistent durability that we are supposed to read in.
 
std::vector< StoreEntriesm_parentStoreEntries
 The parent DataStore entries per level.
 
std::map< std::string, TTree * > m_parentTrees
 Map of file LFNs to trees.
 
ReadStats m_readStats
 some statistics for all files read so far.
 
int m_cacheSize {0}
 Input ROOT File Cache size in MB, <0 means default.
 
bool m_discardErrorEvents {true}
 Discard events that have an error flag != 0.
 
unsigned int m_discardErrorMask {EventMetaData::c_HLTDiscard}
 Don't issue a warning when discarding events if the error flag consists exclusively of flags in this mask.
 
bool m_processingAllEvents {true}
 Set to true if we process the input files completely: No skip events or sequences or -n parameters.
 
bool m_isSecondaryInput {false}
 When using a second RootInputModule in an independent path [usually if you are using add_independent_merge_path(...)] this has to be set to true.
 
std::string m_name
 The name of the module, saved as a string (user-modifiable)
 
std::string m_type
 The type of the module, saved as a string.
 
std::string m_package
 Package this module is found in (may be empty).
 
std::string m_description
 The description of the module.
 
unsigned int m_propertyFlags
 The properties of the module as bitwise or (with |) of EModulePropFlags.
 
LogConfig m_logConfig
 The log system configuration of the module.
 
ModuleParamList m_moduleParamList
 List storing and managing all parameter of the module.
 
bool m_hasReturnValue
 True, if the return value is set.
 
int m_returnValue
 The return value.
 
std::vector< ModuleConditionm_conditions
 Module condition, only non-null if set.
 

Detailed Description

Module to read TTree data from file into the data store.

For more information consult the basf2 Software Portal confluence page. You can specify different TTrees for different durabilities, and specify if only specific branches should be read (branchNames), or if some should be excluded (excludeBranchNames).

The module supports reading from multiple files using TChain, entries will be read in the order the files are specified.

See also
DataStore::EDurability

Definition at line 42 of file RootInputModule.h.

Member Typedef Documentation

◆ EAfterConditionPath

Forward the EAfterConditionPath definition from the ModuleCondition.

Definition at line 88 of file Module.h.

◆ StoreEntries

typedef std::vector<DataStore::StoreEntry*> StoreEntries
private

Vector of entries in the data store.

Definition at line 79 of file RootInputModule.h.

Member Enumeration Documentation

◆ EModulePropFlags

enum EModulePropFlags
inherited

Each module can be tagged with property flags, which indicate certain features of the module.

Enumerator
c_Input 

This module is an input module (reads data).

c_Output 

This module is an output module (writes data).

c_ParallelProcessingCertified 

This module can be run in parallel processing mode safely (All I/O must be done through the data store, in particular, the module must not write any files.)

c_HistogramManager 

This module is used to manage histograms accumulated by other modules.

c_InternalSerializer 

This module is an internal serializer/deserializer for parallel processing.

c_TerminateInAllProcesses 

When using parallel processing, call this module's terminate() function in all processes().

This will also ensure that there is exactly one process (single-core if no parallel modules found) or at least one input, one main and one output process.

c_DontCollectStatistics 

No statistics is collected for this module.

Definition at line 77 of file Module.h.

77 {
78 c_Input = 1,
79 c_Output = 2,
85 };
@ c_HistogramManager
This module is used to manage histograms accumulated by other modules.
Definition: Module.h:81
@ c_Input
This module is an input module (reads data).
Definition: Module.h:78
@ c_DontCollectStatistics
No statistics is collected for this module.
Definition: Module.h:84
@ c_ParallelProcessingCertified
This module can be run in parallel processing mode safely (All I/O must be done through the data stor...
Definition: Module.h:80
@ c_InternalSerializer
This module is an internal serializer/deserializer for parallel processing.
Definition: Module.h:82
@ c_Output
This module is an output module (writes data).
Definition: Module.h:79
@ c_TerminateInAllProcesses
When using parallel processing, call this module's terminate() function in all processes().
Definition: Module.h:83

Constructor & Destructor Documentation

◆ RootInputModule()

Constructor.

Definition at line 39 of file RootInputModule.cc.

39 : Module(), m_nextEntry(0), m_lastPersistentEntry(-1), m_tree(nullptr), m_persistent(nullptr)
40{
41 //Set module properties
42 setDescription("Reads objects/arrays from one or more .root files saved by the RootOutput module and makes them available through the DataStore. Files do not necessarily have to be local, http:// and root:// (for files in xrootd) URLs are supported as well.");
44
45 //Parameter definition
46 vector<string> emptyvector;
47 addParam("inputFileName", m_inputFileName,
48 "Input file name. For multiple files, use inputFileNames or wildcards instead. Can be overridden using the -i argument to basf2.",
49 string(""));
50 addParam("inputFileNames", m_inputFileNames,
51 "List of input files. You may use shell-like expansions to specify multiple files, e.g. 'somePrefix_*.root' or 'file_[a,b]_[1-15].root'. Can be overridden using the -i argument to basf2.",
52 emptyvector);
53 addParam("entrySequences", m_entrySequences,
54 "The number sequences (e.g. 23:42,101) defining the entries which are processed for each inputFileName."
55 "Must be specified exactly once for each file to be opened."
56 "The first event has the entry number 0.", emptyvector);
57 addParam("ignoreCommandLineOverride", m_ignoreCommandLineOverride,
58 "Ignore override of file name via command line argument -i.", false);
59
60 addParam("skipNEvents", m_skipNEvents, "Skip this number of events before starting.", 0u);
61 addParam("skipToEvent", m_skipToEvent, "Skip events until the event with "
62 "the specified (experiment, run, event number) occurs. This parameter "
63 "is useful for debugging to start with a specific event.", m_skipToEvent);
64
66 "Names of event durability branches to be read. Empty means all branches. (EventMetaData is always read)", emptyvector);
68 "Names of persistent durability branches to be read. Empty means all branches. (FileMetaData is always read)", emptyvector);
69
71 "Names of event durability branches NOT to be read. Takes precedence over branchNames.", emptyvector);
72 vector<string> excludePersistent({"ProcessStatistics"});
74 "Names of persistent durability branches NOT to be read. Takes precedence over branchNamesPersistent.", excludePersistent);
75
76 addParam("parentLevel", m_parentLevel,
77 "Number of generations of parent files (files used as input when creating a file) to be read. This can be useful if a file is missing some information available in its parent. See https://confluence.desy.de/display/BI/Software+ParentFiles for details.",
78 0);
79
80 addParam("collectStatistics", m_collectStatistics,
81 "Collect statistics on amount of data read and print statistics (seperate for input & parent files) after processing. Data is collected from TFile using GetBytesRead(), GetBytesReadExtra(), GetReadCalls()",
82 false);
83 addParam("cacheSize", m_cacheSize,
84 "file cache size in Mbytes. If negative, use root default", 0);
85
86 addParam("discardErrorEvents", m_discardErrorEvents,
87 "Discard events with an error flag != 0", m_discardErrorEvents);
88 addParam("silentErrrorDiscardMask", m_discardErrorMask,
89 "Bitmask of error flags to silently discard without raising a WARNING. Should be a combination of the ErrorFlags defined "
90 "in the EventMetaData. No Warning will be issued when discarding an event if the error flag consists exclusively of flags "
91 "present in this mask", m_discardErrorMask);
92
93 addParam("isSecondaryInput", m_isSecondaryInput,
94 "When using a second RootInputModule in an independent path [usually if you are using add_independent_merge_path(...)] "
95 "this has to be set to true",
96 false);
97}
void setDescription(const std::string &description)
Sets the description of the module.
Definition: Module.cc:214
void setPropertyFlags(unsigned int propertyFlags)
Sets the flags for the module properties.
Definition: Module.cc:208
Module()
Constructor.
Definition: Module.cc:30
bool m_discardErrorEvents
Discard events that have an error flag != 0.
unsigned int m_discardErrorMask
Don't issue a warning when discarding events if the error flag consists exclusively of flags in this ...
bool m_isSecondaryInput
When using a second RootInputModule in an independent path [usually if you are using add_independent_...
bool m_collectStatistics
Collect statistics on amount of data read and print statistics (seperate for input & parent files) af...
long m_lastPersistentEntry
last entry to be in persistent tree.
TChain * m_tree
TTree for event input.
int m_parentLevel
Level of parent files to be read.
std::string m_inputFileName
File to read from.
TChain * m_persistent
TTree for persistent input.
std::vector< std::string > m_inputFileNames
Files to read from.
std::vector< std::string > m_excludeBranchNames[DataStore::c_NDurabilityTypes]
Array for names of branches that should NOT be written out.
int m_cacheSize
Input ROOT File Cache size in MB, <0 means default.
long m_nextEntry
Next entry to be read in event tree.
bool m_ignoreCommandLineOverride
Ignore filename override from command line.
std::vector< int > m_skipToEvent
experiment, run, event number of first event to load
unsigned int m_skipNEvents
Can be set from steering file to skip the first N events.
std::vector< std::string > m_branchNames[DataStore::c_NDurabilityTypes]
Array for names of branches, that shall be written out.
std::vector< std::string > m_entrySequences
The number sequences (e.g.
void addParam(const std::string &name, T &paramVariable, const std::string &description, const T &defaultValue)
Adds a new parameter to the module.
Definition: Module.h:560
const std::string c_SteerExcludeBranchNames[]
Steering parameter names for m_excludeBranchNames.
const std::string c_SteerBranchNames[]
Steering parameter names for m_branchNames.

Member Function Documentation

◆ addEventListForIndexFile()

void addEventListForIndexFile ( const std::string &  parentLfn)
private

For index files, this creates TEventList/TEntryListArray to enable better cache use.

Definition at line 700 of file RootInputModule.cc.

701{
702 //is this really an index file? (=only EventMetaData stored)
703 if (!(m_parentLevel > 0 and m_storeEntries.size() == 1))
704 return;
705 //did we handle the current parent file already?
706 if (parentLfn == m_lastParentFileLFN)
707 return;
708 m_lastParentFileLFN = parentLfn;
709
710 B2INFO("Index file detected, scanning to generate event list.");
711 TTree* tree = m_parentTrees.at(parentLfn);
712
713 //both types of list work, TEventList seems to result in slightly less data being read.
714 auto* elist = new TEventList("parent_entrylist");
715 //TEntryListArray* elist = new TEntryListArray();
716
717 TBranch* branch = m_tree->GetBranch("EventMetaData");
718 auto* address = branch->GetAddress();
719 EventMetaData* eventMetaData = nullptr;
720 branch->SetAddress(&eventMetaData);
721 long nEntries = m_tree->GetEntries();
722 for (long i = m_nextEntry; i < nEntries; i++) {
723 branch->GetEntry(i);
724 int experiment = eventMetaData->getExperiment();
725 int run = eventMetaData->getRun();
726 unsigned int event = eventMetaData->getEvent();
727 const std::string& newParentLfn = eventMetaData->getParentLfn();
728
729 if (parentLfn != newParentLfn) {
730 //parent file changed, stopping for now
731 break;
732 }
733 long entry = RootIOUtilities::getEntryNumberWithEvtRunExp(tree, event, run, experiment);
734 elist->Enter(entry);
735 }
736 branch->SetAddress(address);
737
738 if (tree) {
739 tree->SetEventList(elist);
740 //tree->SetEntryList(elist);
741 }
742}
Store event, run, and experiment numbers.
Definition: EventMetaData.h:33
const std::string & getParentLfn() const
Return LFN of the current parent file, or an empty string if not set.
int getRun() const
Run Getter.
unsigned int getEvent() const
Event Getter.
int getExperiment() const
Experiment Getter.
std::map< std::string, TTree * > m_parentTrees
Map of file LFNs to trees.
StoreEntries m_storeEntries
Vector of DataStore entries of event durability that we are supposed to read in.
virtual void event() override
Running over all events.
std::string m_lastParentFileLFN
last parent file LFN seen.
long getEntryNumberWithEvtRunExp(TTree *tree, long event, long run, long experiment)
return entry number with given (event, run, experiment) from tree.

◆ beginRun()

◆ clone()

std::shared_ptr< PathElement > clone ( ) const
overridevirtualinherited

Create an independent copy of this module.

Note that parameters are shared, so changing them on a cloned module will also affect the original module.

Implements PathElement.

Definition at line 179 of file Module.cc.

180{
182 newModule->m_moduleParamList.setParameters(getParamList());
183 newModule->setName(getName());
184 newModule->m_package = m_package;
185 newModule->m_propertyFlags = m_propertyFlags;
186 newModule->m_logConfig = m_logConfig;
187 newModule->m_conditions = m_conditions;
188
189 return newModule;
190}
std::shared_ptr< Module > registerModule(const std::string &moduleName, std::string sharedLibPath="") noexcept(false)
Creates an instance of a module and registers it to the ModuleManager.
static ModuleManager & Instance()
Exception is thrown if the requested module could not be created by the ModuleManager.
const ModuleParamList & getParamList() const
Return module param list.
Definition: Module.h:363
const std::string & getName() const
Returns the name of the module.
Definition: Module.h:187
const std::string & getType() const
Returns the type of the module (i.e.
Definition: Module.cc:41
unsigned int m_propertyFlags
The properties of the module as bitwise or (with |) of EModulePropFlags.
Definition: Module.h:512
LogConfig m_logConfig
The log system configuration of the module.
Definition: Module.h:514
std::vector< ModuleCondition > m_conditions
Module condition, only non-null if set.
Definition: Module.h:521
std::string m_package
Package this module is found in (may be empty).
Definition: Module.h:510
std::shared_ptr< Module > ModulePtr
Defines a pointer to a module object as a boost shared pointer.
Definition: Module.h:43

◆ connectBranches()

bool connectBranches ( TTree *  tree,
DataStore::EDurability  durability,
StoreEntries storeEntries 
)
private

Connect branches of the given tree to the data store.

Parameters
treeThe tree to be connected.
durabilityThe data store durability level.
storeEntriesThe store entries to which the branches are connected will be added to this vector.
Returns
True if the branches could be connected successfully.

Definition at line 518 of file RootInputModule.cc.

519{
520 B2DEBUG(30, "File changed, loading persistent data.");
522 //Go over the branchlist and connect the branches with DataStore entries
523 const TObjArray* branchesObjArray = tree->GetListOfBranches();
524 if (!branchesObjArray) {
525 B2FATAL("Tree '" << tree->GetName() << "' doesn't contain any branches!");
526 }
527 std::vector<TBranch*> branches;
528 set<string> branchList;
529 for (int jj = 0; jj < branchesObjArray->GetEntriesFast(); jj++) {
530 auto* branch = static_cast<TBranch*>(branchesObjArray->At(jj));
531 if (!branch) continue;
532 branchList.insert(branch->GetName());
533 branches.emplace_back(branch);
534 // start with all branches disabled and only enable what we read
535 setBranchStatus(branch, false);
536 }
537 //skip branches the user doesn't want
538 branchList = filterBranches(branchList, m_branchNames[durability], m_excludeBranchNames[durability], durability, true);
539 for (TBranch* branch : branches) {
540 const std::string branchName = branch->GetName();
541 //skip already connected branches
542 if (m_connectedBranches[durability].find(branchName) != m_connectedBranches[durability].end())
543 continue;
544
545 if ((branchList.count(branchName) == 0) and
546 ((branchName != "FileMetaData") || (tree != m_persistent)) and
547 ((branchName != "EventMetaData") || (tree != m_tree))) {
548 continue;
549 }
550 auto found = setBranchStatus(branch, true);
551 B2DEBUG(32, "Enabling branch" << LogVar("branchName", branchName)
552 << LogVar("children found", found));
553
554 //Get information about the object in the branch
555 TObject* objectPtr = nullptr;
556 branch->SetAddress(&objectPtr);
557 branch->GetEntry();
558 bool array = (string(branch->GetClassName()) == "TClonesArray");
559 TClass* objClass = nullptr;
560 if (array)
561 objClass = (static_cast<TClonesArray*>(objectPtr))->GetClass();
562 else
563 objClass = objectPtr->IsA();
564 delete objectPtr;
565
566 //Create a DataStore entry and connect the branch address to it
567 if (!DataStore::Instance().registerEntry(branchName, durability, objClass, array, DataStore::c_WriteOut)) {
568 B2FATAL("Cannot connect branch to datastore" << LogVar("branchName", branchName));
569 continue;
570 }
571 DataStore::StoreEntry& entry = (map.find(branchName))->second;
572 tree->SetBranchAddress(branch->GetName(), &(entry.object));
573 if (storeEntries) storeEntries->push_back(&entry);
574
575 //Keep track of already connected branches
576 m_connectedBranches[durability].insert(branchName);
577 }
578
579 return true;
580}
@ c_WriteOut
Object/array should be saved by output modules.
Definition: DataStore.h:70
StoreEntryMap & getStoreEntryMap(EDurability durability)
Get a reference to the object/array map.
Definition: DataStore.h:325
static DataStore & Instance()
Instance of singleton Store.
Definition: DataStore.cc:54
bool registerEntry(const std::string &name, EDurability durability, TClass *objClass, bool array, EStoreFlags storeFlags)
Register an entry in the DataStore map.
Definition: DataStore.cc:190
std::map< std::string, StoreEntry > StoreEntryMap
Map for StoreEntries.
Definition: DataStore.h:87
std::set< std::string > m_connectedBranches[DataStore::c_NDurabilityTypes]
Already connected branches.
Class to store variables with their name which were sent to the logging service.
std::set< std::string > filterBranches(const std::set< std::string > &branchesToFilter, const std::vector< std::string > &branches, const std::vector< std::string > &excludeBranches, int durability, bool quiet=false)
Given a list of input branches and lists of branches to include/exclude, returns a list of branches t...
size_t setBranchStatus(TBranch *branch, bool process)
Set Branch to be read or not.
Wraps a stored array/object, stored under unique (name, durability) key.
Definition: StoreEntry.h:22
TObject * object
The pointer to the actual object.
Definition: StoreEntry.h:48

◆ createParentStoreEntries()

bool createParentStoreEntries ( )
private

Connect the parent trees and fill m_parentStoreEntries.

Definition at line 583 of file RootInputModule.cc.

584{
585 // get the experiment/run/event number and parentLfn of the first entry
586 assert(m_tree);
587 TBranch* branch = m_tree->GetBranch("EventMetaData");
588 char* address = branch->GetAddress();
589 EventMetaData* eventMetaData = nullptr;
590 branch->SetAddress(&eventMetaData);
591 branch->GetEntry(0);
592 int experiment = eventMetaData->getExperiment();
593 int run = eventMetaData->getRun();
594 unsigned int event = eventMetaData->getEvent();
595 std::string parentLfn = eventMetaData->getParentLfn();
596 branch->SetAddress(address);
597
598 // loop over parents and get their metadata
599 for (int level = 0; level < m_parentLevel; level++) {
600 // open the parent file
601 TDirectory* dir = gDirectory;
602 const std::string parentPfn = FileCatalog::Instance().getPhysicalFileName(parentLfn);
603 TFile* file = TFile::Open(parentPfn.c_str(), "READ");
604 dir->cd();
605 if (!file || !file->IsOpen()) {
606 B2ERROR("Couldn't open parent file. Maybe you need to create a file catalog using b2file-catalog-add?"
607 << LogVar("LFN", parentLfn) << LogVar("PFN", parentPfn));
608 return false;
609 }
610
611 // get the event tree and connect its branches
612 auto* tree = dynamic_cast<TTree*>(file->Get(c_treeNames[DataStore::c_Event].c_str()));
613 if (!tree) {
614 B2ERROR("No tree " << c_treeNames[DataStore::c_Event] << " found in " << parentPfn);
615 return false;
616 }
617 if (int(m_parentStoreEntries.size()) <= level) m_parentStoreEntries.resize(level + 1);
619 m_parentTrees.insert(std::make_pair(parentLfn, tree));
620
621 // get the persistent tree and read its branches
622 auto* persistent = dynamic_cast<TTree*>(file->Get(c_treeNames[DataStore::c_Persistent].c_str()));
623 if (!persistent) {
624 B2ERROR("No tree " << c_treeNames[DataStore::c_Persistent] << " found in " << parentPfn);
625 return false;
626 }
627 connectBranches(persistent, DataStore::c_Persistent, nullptr);
628
629 // get parent LFN of parent
630 EventMetaData* metaData = nullptr;
631 tree->SetBranchAddress("EventMetaData", &metaData);
632 long entry = RootIOUtilities::getEntryNumberWithEvtRunExp(tree, event, run, experiment);
633 tree->GetBranch("EventMetaData")->GetEntry(entry);
634 parentLfn = metaData->getParentLfn();
635 }
636
637 return true;
638}
@ c_Persistent
Object is available during entire execution time.
Definition: DataStore.h:60
@ c_Event
Different object in each event, all objects/arrays are invalidated after event() function has been ca...
Definition: DataStore.h:59
static FileCatalog & Instance()
Static method to get a reference to the FileCatalog instance.
Definition: FileCatalog.cc:23
virtual std::string getPhysicalFileName(const std::string &lfn)
Get the physical file name for the LFN.
Definition: FileCatalog.cc:180
std::vector< StoreEntries > m_parentStoreEntries
The parent DataStore entries per level.
bool connectBranches(TTree *tree, DataStore::EDurability durability, StoreEntries *storeEntries)
Connect branches of the given tree to the data store.
const std::string c_treeNames[]
Names of trees.

◆ def_beginRun()

virtual void def_beginRun ( )
inlineprotectedvirtualinherited

Wrapper method for the virtual function beginRun() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 426 of file Module.h.

426{ beginRun(); }
virtual void beginRun()
Called when entering a new run.
Definition: Module.h:147

◆ def_endRun()

virtual void def_endRun ( )
inlineprotectedvirtualinherited

This method can receive that the current run ends as a call from the Python side.

For regular C++-Modules that forwards the call to the regular endRun() method.

Reimplemented in PyModule.

Definition at line 439 of file Module.h.

439{ endRun(); }
virtual void endRun()
This method is called if the current run ends.
Definition: Module.h:166

◆ def_event()

virtual void def_event ( )
inlineprotectedvirtualinherited

Wrapper method for the virtual function event() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 432 of file Module.h.

432{ event(); }
virtual void event()
This method is the core of the module.
Definition: Module.h:157

◆ def_initialize()

virtual void def_initialize ( )
inlineprotectedvirtualinherited

Wrappers to make the methods without "def_" prefix callable from Python.

Overridden in PyModule. Wrapper method for the virtual function initialize() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 420 of file Module.h.

420{ initialize(); }
virtual void initialize()
Initialize the Module.
Definition: Module.h:109

◆ def_terminate()

virtual void def_terminate ( )
inlineprotectedvirtualinherited

Wrapper method for the virtual function terminate() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 445 of file Module.h.

445{ terminate(); }
virtual void terminate()
This method is called at the end of the event processing.
Definition: Module.h:176

◆ endRun()

virtual void endRun ( )
inlinevirtualinherited

◆ entryNotFound()

void entryNotFound ( const std::string &  entryOrigin,
const std::string &  name,
bool  fileChanged = true 
)
private

Check if we warn the user or abort after an entry was missing after changing files.

Definition at line 744 of file RootInputModule.cc.

745{
746 if (name == "ProcessStatistics" or DataStore::Instance().getDependencyMap().isUsedAs(name, DependencyMap::c_Input)) {
747 B2FATAL(entryOrigin << " in " << m_tree->GetFile()->GetName() << " does not contain required object " << name << ", aborting.");
748 } else if (fileChanged) {
749 B2WARNING(entryOrigin << " in " << m_tree->GetFile()->GetName() << " does not contain object " << name <<
750 " that was present in a previous entry.");
751 }
752}
@ c_Input
required input.
Definition: DependencyMap.h:33

◆ evalCondition()

bool evalCondition ( ) const
inherited

If at least one condition was set, it is evaluated and true returned if at least one condition returns true.

If no condition or result value was defined, the method returns false. Otherwise, the condition is evaluated and true returned, if at least one condition returns true. To speed up the evaluation, the condition strings were already parsed in the method if_value().

Returns
True if at least one condition and return value exists and at least one condition expression was evaluated to true.

Definition at line 96 of file Module.cc.

97{
98 if (m_conditions.empty()) return false;
99
100 //okay, a condition was set for this Module...
101 if (!m_hasReturnValue) {
102 B2FATAL("A condition was set for '" << getName() << "', but the module did not set a return value!");
103 }
104
105 for (const auto& condition : m_conditions) {
106 if (condition.evaluate(m_returnValue)) {
107 return true;
108 }
109 }
110 return false;
111}
int m_returnValue
The return value.
Definition: Module.h:519
bool m_hasReturnValue
True, if the return value is set.
Definition: Module.h:518

◆ event()

void event ( )
overridevirtual

Running over all events.

Reimplemented from Module.

Definition at line 345 of file RootInputModule.cc.

346{
347 if (!m_tree)
348 return;
349
350 while (true) {
352 if (nextEntry >= 0 && nextEntry < InputController::numEntries(m_isSecondaryInput)) {
353 // don't show this message if we are doing event merging, as it will pop up twice for every event
355 B2INFO("RootInput: will read entry " << nextEntry << " next.");
356 }
357 m_nextEntry = nextEntry;
362 if (entry >= 0) {
363 const long chainentry = m_tree->GetChainEntryNumber(entry);
364 B2INFO("RootInput: will read entry " << chainentry << " (entry " << entry << " in current file) next.");
365 m_nextEntry = chainentry;
366 } else {
367 B2ERROR("Couldn't find entry (" << InputController::getNextEvent() << ", " << InputController::getNextRun() << ", " <<
368 InputController::getNextExperiment() << ") in file! Loading entry " << m_nextEntry << " instead.");
369 }
370 }
372
373 readTree();
374 m_nextEntry++;
375
376 // check for events with errors
377 unsigned int errorFlag = 0;
378 if (m_discardErrorEvents && (m_nextEntry >= 0)) {
379 const StoreObjPtr<EventMetaData> eventMetaData;
380 errorFlag = eventMetaData->getErrorFlag();
381 if (errorFlag != 0) {
382 if (errorFlag & ~m_discardErrorMask) {
383 B2WARNING("Discarding corrupted event" << LogVar("errorFlag", errorFlag) << LogVar("experiment", eventMetaData->getExperiment())
384 << LogVar("run", eventMetaData->getRun()) << LogVar("event", eventMetaData->getEvent()));
385 }
386 // make sure this event is not used if it's the last one in the file
387 eventMetaData->setEndOfData();
388 }
389 }
390 if (errorFlag == 0) break;
391 }
392}
static long getNextRun()
Return run number set via setNextEntry().
static long getNextExperiment()
Return experiment number set via setNextEntry().
static bool getEventMerging()
Get if we are merging events from two paths.
static void eventLoaded(long entry, bool independentPath=false)
Indicate that an event (in the given entry) was loaded and reset all members related to the next entr...
static long numEntries(bool independentPath=false)
Returns total number of entries in the event tree.
static long getNextEntry(bool independentPath=false)
Return entry number set via setNextEntry().
static long getNextEvent()
Return event number set via setNextEntry().
void readTree()
Actually performs the reading from the tree.
Type-safe access to single objects in the data store.
Definition: StoreObjPtr.h:96

◆ exposePythonAPI()

void exposePythonAPI ( )
staticinherited

Exposes methods of the Module class to Python.

Definition at line 325 of file Module.cc.

326{
327 // to avoid confusion between std::arg and boost::python::arg we want a shorthand namespace as well
328 namespace bp = boost::python;
329
330 docstring_options options(true, true, false); //userdef, py sigs, c++ sigs
331
332 void (Module::*setReturnValueInt)(int) = &Module::setReturnValue;
333
334 enum_<Module::EAfterConditionPath>("AfterConditionPath",
335 R"(Determines execution behaviour after a conditional path has been executed:
336
337.. attribute:: END
338
339 End processing of this path after the conditional path. (this is the default for if_value() etc.)
340
341.. attribute:: CONTINUE
342
343 After the conditional path, resume execution after this module.)")
344 .value("END", Module::EAfterConditionPath::c_End)
345 .value("CONTINUE", Module::EAfterConditionPath::c_Continue)
346 ;
347
348 /* Do not change the names of >, <, ... we use them to serialize conditional pathes */
349 enum_<Belle2::ModuleCondition::EConditionOperators>("ConditionOperator")
356 ;
357
358 enum_<Module::EModulePropFlags>("ModulePropFlags",
359 R"(Flags to indicate certain low-level features of modules, see :func:`Module.set_property_flags()`, :func:`Module.has_properties()`. Most useful flags are:
360
361.. attribute:: PARALLELPROCESSINGCERTIFIED
362
363 This module can be run in parallel processing mode safely (All I/O must be done through the data store, in particular, the module must not write any files.)
364
365.. attribute:: HISTOGRAMMANAGER
366
367 This module is used to manage histograms accumulated by other modules
368
369.. attribute:: TERMINATEINALLPROCESSES
370
371 When using parallel processing, call this module's terminate() function in all processes. This will also ensure that there is exactly one process (single-core if no parallel modules found) or at least one input, one main and one output process.
372)")
373 .value("INPUT", Module::EModulePropFlags::c_Input)
374 .value("OUTPUT", Module::EModulePropFlags::c_Output)
375 .value("PARALLELPROCESSINGCERTIFIED", Module::EModulePropFlags::c_ParallelProcessingCertified)
376 .value("HISTOGRAMMANAGER", Module::EModulePropFlags::c_HistogramManager)
377 .value("INTERNALSERIALIZER", Module::EModulePropFlags::c_InternalSerializer)
378 .value("TERMINATEINALLPROCESSES", Module::EModulePropFlags::c_TerminateInAllProcesses)
379 ;
380
381 //Python class definition
382 class_<Module, PyModule> module("Module", R"(
383Base class for Modules.
384
385A module is the smallest building block of the framework.
386A typical event processing chain consists of a Path containing
387modules. By inheriting from this base class, various types of
388modules can be created. To use a module, please refer to
389:func:`Path.add_module()`. A list of modules is available by running
390``basf2 -m`` or ``basf2 -m package``, detailed information on parameters is
391given by e.g. ``basf2 -m RootInput``.
392
393The 'Module Development' section in the manual provides detailed information
394on how to create modules, setting parameters, or using return values/conditions:
395https://confluence.desy.de/display/BI/Software+Basf2manual#Module_Development
396
397)");
398 module
399 .def("__str__", &Module::getPathString)
400 .def("name", &Module::getName, return_value_policy<copy_const_reference>(),
401 "Returns the name of the module. Can be changed via :func:`set_name() <Module.set_name()>`, use :func:`type() <Module.type()>` for identifying a particular module class.")
402 .def("type", &Module::getType, return_value_policy<copy_const_reference>(),
403 "Returns the type of the module (i.e. class name minus 'Module')")
404 .def("set_name", &Module::setName, args("name"), R"(
405Set custom name, e.g. to distinguish multiple modules of the same type.
406
407>>> path.add_module('EventInfoSetter')
408>>> ro = path.add_module('RootOutput', branchNames=['EventMetaData'])
409>>> ro.set_name('RootOutput_metadata_only')
410>>> print(path)
411[EventInfoSetter -> RootOutput_metadata_only]
412
413)")
414 .def("description", &Module::getDescription, return_value_policy<copy_const_reference>(),
415 "Returns the description of this module.")
416 .def("package", &Module::getPackage, return_value_policy<copy_const_reference>(),
417 "Returns the package this module belongs to.")
418 .def("available_params", &_getParamInfoListPython,
419 "Return list of all module parameters as `ModuleParamInfo` instances")
420 .def("has_properties", &Module::hasProperties, (bp::arg("properties")),
421 R"DOCSTRING(Allows to check if the module has the given properties out of `ModulePropFlags` set.
422
423>>> if module.has_properties(ModulePropFlags.PARALLELPROCESSINGCERTIFIED):
424>>> ...
425
426Parameters:
427 properties (int): bitmask of `ModulePropFlags` to check for.
428)DOCSTRING")
429 .def("set_property_flags", &Module::setPropertyFlags, args("property_mask"),
430 "Set module properties in the form of an OR combination of `ModulePropFlags`.");
431 {
432 // python signature is too crowded, make ourselves
433 docstring_options subOptions(true, false, false); //userdef, py sigs, c++ sigs
434 module
435 .def("if_value", &Module::if_value,
436 (bp::arg("expression"), bp::arg("condition_path"), bp::arg("after_condition_path")= Module::EAfterConditionPath::c_End),
437 R"DOCSTRING(if_value(expression, condition_path, after_condition_path=AfterConditionPath.END)
438
439Sets a conditional sub path which will be executed after this
440module if the return value set in the module passes the given ``expression``.
441
442Modules can define a return value (int or bool) using ``setReturnValue()``,
443which can be used in the steering file to split the Path based on this value, for example
444
445>>> module_with_condition.if_value("<1", another_path)
446
447In case the return value of the ``module_with_condition`` for a given event is
448less than 1, the execution will be diverted into ``another_path`` for this event.
449
450You could for example set a special return value if an error occurs, and divert
451the execution into a path containing :b2:mod:`RootOutput` if it is found;
452saving only the data producing/produced by the error.
453
454After a conditional path has executed, basf2 will by default stop processing
455the path for this event. This behaviour can be changed by setting the
456``after_condition_path`` argument.
457
458Parameters:
459 expression (str): Expression to determine if the conditional path should be executed.
460 This should be one of the comparison operators ``<``, ``>``, ``<=``,
461 ``>=``, ``==``, or ``!=`` followed by a numerical value for the return value
462 condition_path (Path): path to execute in case the expression is fulfilled
463 after_condition_path (AfterConditionPath): What to do once the ``condition_path`` has been executed.
464)DOCSTRING")
465 .def("if_false", &Module::if_false,
466 (bp::arg("condition_path"), bp::arg("after_condition_path")= Module::EAfterConditionPath::c_End),
467 R"DOC(if_false(condition_path, after_condition_path=AfterConditionPath.END)
468
469Sets a conditional sub path which will be executed after this module if
470the return value of the module evaluates to False. This is equivalent to
471calling `if_value` with ``expression=\"<1\"``)DOC")
472 .def("if_true", &Module::if_true,
473 (bp::arg("condition_path"), bp::arg("after_condition_path")= Module::EAfterConditionPath::c_End),
474 R"DOC(if_true(condition_path, after_condition_path=AfterConditionPath.END)
475
476Sets a conditional sub path which will be executed after this module if
477the return value of the module evaluates to True. It is equivalent to
478calling `if_value` with ``expression=\">=1\"``)DOC");
479 }
480 module
481 .def("has_condition", &Module::hasCondition,
482 "Return true if a conditional path has been set for this module "
483 "using `if_value`, `if_true` or `if_false`")
484 .def("get_all_condition_paths", &_getAllConditionPathsPython,
485 "Return a list of all conditional paths set for this module using "
486 "`if_value`, `if_true` or `if_false`")
487 .def("get_all_conditions", &_getAllConditionsPython,
488 "Return a list of all conditional path expressions set for this module using "
489 "`if_value`, `if_true` or `if_false`")
490 .add_property("logging", make_function(&Module::getLogConfig, return_value_policy<reference_existing_object>()),
@ c_GE
Greater or equal than: ">=".
@ c_SE
Smaller or equal than: "<=".
@ c_GT
Greater than: ">"
@ c_NE
Not equal: "!=".
@ c_EQ
Equal: "=" or "=="
@ c_ST
Smaller than: "<"
Base class for Modules.
Definition: Module.h:72
LogConfig & getLogConfig()
Returns the log system configuration.
Definition: Module.h:225
void if_value(const std::string &expression, const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
Add a condition to the module.
Definition: Module.cc:79
void if_true(const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
A simplified version to set the condition of the module.
Definition: Module.cc:90
void setReturnValue(int value)
Sets the return value for this module as integer.
Definition: Module.cc:220
void setLogConfig(const LogConfig &logConfig)
Set the log system configuration.
Definition: Module.h:230
const std::string & getDescription() const
Returns the description of the module.
Definition: Module.h:202
void if_false(const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
A simplified version to add a condition to the module.
Definition: Module.cc:85
bool hasCondition() const
Returns true if at least one condition was set for the module.
Definition: Module.h:311
const std::string & getPackage() const
Returns the package this module is in.
Definition: Module.h:197
void setName(const std::string &name)
Set the name of the module.
Definition: Module.h:214
bool hasProperties(unsigned int propertyFlags) const
Returns true if all specified property flags are available in this module.
Definition: Module.cc:160
std::string getPathString() const override
return the module name.
Definition: Module.cc:192

◆ getAfterConditionPath()

Module::EAfterConditionPath getAfterConditionPath ( ) const
inherited

What to do after the conditional path is finished.

(defaults to c_End if no condition is set)

Definition at line 133 of file Module.cc.

134{
135 if (m_conditions.empty()) return EAfterConditionPath::c_End;
136
137 //okay, a condition was set for this Module...
138 if (!m_hasReturnValue) {
139 B2FATAL("A condition was set for '" << getName() << "', but the module did not set a return value!");
140 }
141
142 for (const auto& condition : m_conditions) {
143 if (condition.evaluate(m_returnValue)) {
144 return condition.getAfterConditionPath();
145 }
146 }
147
148 return EAfterConditionPath::c_End;
149}

◆ getAllConditionPaths()

std::vector< std::shared_ptr< Path > > getAllConditionPaths ( ) const
inherited

Return all condition paths currently set (no matter if the condition is true or not).

Definition at line 150 of file Module.cc.

151{
152 std::vector<std::shared_ptr<Path>> allConditionPaths;
153 for (const auto& condition : m_conditions) {
154 allConditionPaths.push_back(condition.getPath());
155 }
156
157 return allConditionPaths;
158}

◆ getAllConditions()

const std::vector< ModuleCondition > & getAllConditions ( ) const
inlineinherited

Return all set conditions for this module.

Definition at line 324 of file Module.h.

325 {
326 return m_conditions;
327 }

◆ getCondition()

const ModuleCondition * getCondition ( ) const
inlineinherited

Return a pointer to the first condition (or nullptr, if none was set)

Definition at line 314 of file Module.h.

315 {
316 if (m_conditions.empty()) {
317 return nullptr;
318 } else {
319 return &m_conditions.front();
320 }
321 }

◆ getConditionPath()

std::shared_ptr< Path > getConditionPath ( ) const
inherited

Returns the path of the last true condition (if there is at least one, else reaturn a null pointer).


Definition at line 113 of file Module.cc.

114{
115 PathPtr p;
116 if (m_conditions.empty()) return p;
117
118 //okay, a condition was set for this Module...
119 if (!m_hasReturnValue) {
120 B2FATAL("A condition was set for '" << getName() << "', but the module did not set a return value!");
121 }
122
123 for (const auto& condition : m_conditions) {
124 if (condition.evaluate(m_returnValue)) {
125 return condition.getPath();
126 }
127 }
128
129 // if none of the conditions were true, return a null pointer.
130 return p;
131}
std::shared_ptr< Path > PathPtr
Defines a pointer to a path object as a boost shared pointer.
Definition: Path.h:35

◆ getDescription()

const std::string & getDescription ( ) const
inlineinherited

Returns the description of the module.

Definition at line 202 of file Module.h.

202{return m_description;}
std::string m_description
The description of the module.
Definition: Module.h:511

◆ getFileNames()

virtual std::vector< std::string > getFileNames ( bool  outputFiles = false)
inlineoverridevirtual

Get list of input files, taking -i command line overrides into account.

Reimplemented from Module.

Definition at line 61 of file RootInputModule.h.

62 {
63 B2ASSERT("RootInput is not an output module", !outputFiles);
64 std::vector<std::string> inputFiles = !m_isSecondaryInput ? Environment::Instance().getInputFilesOverride() :
66 if (!m_ignoreCommandLineOverride and !inputFiles.empty()) {
67 return inputFiles;
68 }
69 inputFiles = m_inputFileNames;
70 if (!m_inputFileName.empty())
71 inputFiles.push_back(m_inputFileName);
72 return inputFiles;
73 }
const std::vector< std::string > & getInputFilesOverride() const
Return overriden input file names, or empty vector if none were set.
Definition: Environment.h:115
static Environment & Instance()
Static method to get a reference to the Environment instance.
Definition: Environment.cc:28
const std::vector< std::string > & getSecondaryInputFilesOverride() const
Return overriden secondary input file names, or empty vector if none were set.
Definition: Environment.h:121

◆ getLogConfig()

LogConfig & getLogConfig ( )
inlineinherited

Returns the log system configuration.

Definition at line 225 of file Module.h.

225{return m_logConfig;}

◆ getModules()

std::list< ModulePtr > getModules ( ) const
inlineoverrideprivatevirtualinherited

no submodules, return empty list

Implements PathElement.

Definition at line 506 of file Module.h.

506{ return std::list<ModulePtr>(); }

◆ getName()

const std::string & getName ( ) const
inlineinherited

Returns the name of the module.

This can be changed via e.g. set_name() in the steering file to give more useful names if there is more than one module of the same type.

For identifying the type of a module, using getType() (or type() in Python) is recommended.

Definition at line 187 of file Module.h.

187{return m_name;}
std::string m_name
The name of the module, saved as a string (user-modifiable)
Definition: Module.h:508

◆ getPackage()

const std::string & getPackage ( ) const
inlineinherited

Returns the package this module is in.

Definition at line 197 of file Module.h.

197{return m_package;}

◆ getParamInfoListPython()

std::shared_ptr< boost::python::list > getParamInfoListPython ( ) const
inherited

Returns a python list of all parameters.

Each item in the list consists of the name of the parameter, a string describing its type, a python list of all default values and the description of the parameter.

Returns
A python list containing the parameters of this parameter list.

Definition at line 279 of file Module.cc.

280{
282}
std::shared_ptr< boost::python::list > getParamInfoListPython() const
Returns a python list of all parameters.
ModuleParamList m_moduleParamList
List storing and managing all parameter of the module.
Definition: Module.h:516

◆ getParamList()

const ModuleParamList & getParamList ( ) const
inlineinherited

Return module param list.

Definition at line 363 of file Module.h.

363{ return m_moduleParamList; }

◆ getPathString()

std::string getPathString ( ) const
overrideprivatevirtualinherited

return the module name.

Implements PathElement.

Definition at line 192 of file Module.cc.

193{
194
195 std::string output = getName();
196
197 for (const auto& condition : m_conditions) {
198 output += condition.getString();
199 }
200
201 return output;
202}

◆ getReturnValue()

int getReturnValue ( ) const
inlineinherited

Return the return value set by this module.

This value is only meaningful if hasReturnValue() is true

Definition at line 381 of file Module.h.

381{ return m_returnValue; }

◆ getType()

const std::string & getType ( ) const
inherited

Returns the type of the module (i.e.

class name minus 'Module')

Definition at line 41 of file Module.cc.

42{
43 if (m_type.empty())
44 B2FATAL("Module type not set for " << getName());
45 return m_type;
46}
std::string m_type
The type of the module, saved as a string.
Definition: Module.h:509

◆ hasCondition()

bool hasCondition ( ) const
inlineinherited

Returns true if at least one condition was set for the module.

Definition at line 311 of file Module.h.

311{ return not m_conditions.empty(); };

◆ hasProperties()

bool hasProperties ( unsigned int  propertyFlags) const
inherited

Returns true if all specified property flags are available in this module.

Parameters
propertyFlagsOred EModulePropFlags which should be compared with the module flags.

Definition at line 160 of file Module.cc.

161{
162 return (propertyFlags & m_propertyFlags) == propertyFlags;
163}

◆ hasReturnValue()

bool hasReturnValue ( ) const
inlineinherited

Return true if this module has a valid return value set.

Definition at line 378 of file Module.h.

378{ return m_hasReturnValue; }

◆ hasUnsetForcedParams()

bool hasUnsetForcedParams ( ) const
inherited

Returns true and prints error message if the module has unset parameters which the user has to set in the steering file.

Definition at line 166 of file Module.cc.

167{
169 std::string allMissing = "";
170 for (const auto& s : missing)
171 allMissing += s + " ";
172 if (!missing.empty())
173 B2ERROR("The following required parameters of Module '" << getName() << "' were not specified: " << allMissing <<
174 "\nPlease add them to your steering file.");
175 return !missing.empty();
176}
std::vector< std::string > getUnsetForcedParams() const
Returns list of unset parameters (if they are required to have a value.

◆ if_false()

void if_false ( const std::shared_ptr< Path > &  path,
EAfterConditionPath  afterConditionPath = EAfterConditionPath::c_End 
)
inherited

A simplified version to add a condition to the module.

Please note that successive calls of this function will add more than one condition to the module. If more than one condition results in true, only the last of them will be used.

Please be careful: Avoid creating cyclic paths, e.g. by linking a condition to a path which is processed before the path where this module is located in.

It is equivalent to the if_value() method, using the expression "<1". This method is meant to be used together with the setReturnValue(bool value) method.

Parameters
pathShared pointer to the Path which will be executed if the return value is false.
afterConditionPathWhat to do after executing 'path'.

Definition at line 85 of file Module.cc.

86{
87 if_value("<1", path, afterConditionPath);
88}

◆ if_true()

void if_true ( const std::shared_ptr< Path > &  path,
EAfterConditionPath  afterConditionPath = EAfterConditionPath::c_End 
)
inherited

A simplified version to set the condition of the module.

Please note that successive calls of this function will add more than one condition to the module. If more than one condition results in true, only the last of them will be used.

Please be careful: Avoid creating cyclic paths, e.g. by linking a condition to a path which is processed before the path where this module is located in.

It is equivalent to the if_value() method, using the expression ">=1". This method is meant to be used together with the setReturnValue(bool value) method.

Parameters
pathShared pointer to the Path which will be executed if the return value is true.
afterConditionPathWhat to do after executing 'path'.

Definition at line 90 of file Module.cc.

91{
92 if_value(">=1", path, afterConditionPath);
93}

◆ if_value()

void if_value ( const std::string &  expression,
const std::shared_ptr< Path > &  path,
EAfterConditionPath  afterConditionPath = EAfterConditionPath::c_End 
)
inherited

Add a condition to the module.

Please note that successive calls of this function will add more than one condition to the module. If more than one condition results in true, only the last of them will be used.

See https://confluence.desy.de/display/BI/Software+ModCondTut or ModuleCondition for a description of the syntax.

Please be careful: Avoid creating cyclic paths, e.g. by linking a condition to a path which is processed before the path where this module is located in.

Parameters
expressionThe expression of the condition.
pathShared pointer to the Path which will be executed if the condition is evaluated to true.
afterConditionPathWhat to do after executing 'path'.

Definition at line 79 of file Module.cc.

80{
81 m_conditions.emplace_back(expression, path, afterConditionPath);
82}

◆ initialize()

void initialize ( )
overridevirtual

Initialize the Module.

Reimplemented from Module.

Definition at line 101 of file RootInputModule.cc.

102{
103 unsigned int skipNEventsOverride = Environment::Instance().getSkipEventsOverride();
104 if (skipNEventsOverride != 0)
105 m_skipNEvents = skipNEventsOverride;
106
107 auto entrySequencesOverride = Environment::Instance().getEntrySequencesOverride();
108 if (entrySequencesOverride.size() > 0)
109 m_entrySequences = entrySequencesOverride;
110
114
115 const vector<string>& inputFiles = getFileNames();
116 if (inputFiles.empty()) {
117 B2FATAL("You have to set either the 'inputFileName' or the 'inputFileNames' parameter, or start basf2 with the '-i MyFile.root' option.");
118 }
119 if (!m_inputFileName.empty() && !m_inputFileNames.empty()) {
120 B2FATAL("Cannot use both 'inputFileName' and 'inputFileNames' parameters!");
121 }
123 if (m_inputFileNames.empty()) {
124 B2FATAL("No valid files specified!");
125 }
126
127 if (m_entrySequences.size() > 0 and m_inputFileNames.size() != m_entrySequences.size()) {
128 B2FATAL("Number of provided filenames does not match the number of given entrySequences parameters: len(inputFileNames) = "
129 << m_inputFileNames.size() << " len(entrySequences) = " << m_entrySequences.size());
130 }
131
132 m_inputFileName = "";
133 // we'll only use m_inputFileNames from now on
134
135 // so let's create the chain objects ...
136 m_persistent = new TChain(c_treeNames[DataStore::c_Persistent].c_str());
137 m_tree = new TChain(c_treeNames[DataStore::c_Event].c_str());
138
139 // time for sanity checks. The problem is that this needs to read a few bytes
140 // from every input file so for jobs with large amount of input files this
141 // will not be efficient.
142 // TODO: We might want to create a different input module which will not
143 // check anything and require manual input like the number of events in
144 // each file and the global tags to use. That would be more efficient
145 // but also less safe
146
147 // list of required branches. We keep this empty for now and only fill
148 // it after we checked the first file to make sure all other files have the
149 // same branches available.
150 std::set<std::string> requiredEventBranches;
151 std::set<std::string> requiredPersistentBranches;
152 // Event metadata from all files, keep it around for sanity checks and globaltag replay
153 std::vector<FileMetaData> fileMetaData;
154 // and if so, what is the sum
155 std::result_of<decltype(&FileMetaData::getMcEvents)(FileMetaData)>::type sumInputMCEvents{0};
156
157 // scope for local variables
158 {
159 // temporarily disable some root warnings
160 auto rootWarningGuard = ScopeGuard::guardValue(gErrorIgnoreLevel, kWarning + 1);
161 // do all files have a consistent number of MC events? that is all positive or all zero
162 bool validInputMCEvents{true};
163 for (const string& fileName : m_inputFileNames) {
164 // read metadata and create sum of MCEvents and global tags
165 try {
166 RootIOUtilities::RootFileInfo fileInfo(fileName);
167 FileMetaData meta = fileInfo.getFileMetaData();
168 if (meta.getNEvents() == 0) {
169 B2WARNING("File appears to be empty, skipping" << LogVar("filename", fileName));
170 continue;
171 }
172 realDataWorkaround(meta);
173 fileMetaData.push_back(meta);
174 // make sure we only look at data or MC. For the first file this is trivially true
175 if (fileMetaData.front().isMC() != meta.isMC()) {
176 throw std::runtime_error("Mixing real data and simulated data for input files is not supported");
177 }
178 // accumulate number of inputMCEvents now
179 if (validInputMCEvents) {
180 // make sure that all files have either a non-zero or zero mcevents.
181 if ((sumInputMCEvents > 0 and meta.getMcEvents() == 0)) {
182 B2WARNING("inconsistent input files: zero mcEvents, setting total number of MC events to zero" << LogVar("filename", fileName));
183 validInputMCEvents = false;
184 }
185 // So accumulate the number of MCEvents but let's be careful to not have an overflow here
186 if (__builtin_add_overflow(sumInputMCEvents, meta.getMcEvents(), &sumInputMCEvents)) {
187 B2FATAL("Number of MC events is too large and cannot be represented anymore");
188 }
189 }
190 // for the first file we don't know what branches are required but now we can determine them as we know the file can be opened
191 if (requiredEventBranches.empty()) {
192 // make sure we have event meta data
193 fileInfo.checkMissingBranches({"EventMetaData"}, false);
194 requiredEventBranches = fileInfo.getBranchNames(false);
195 // filter the branches depending on what the user selected. Note we
196 // do the same thing again in connectBranches but we leave it like
197 // that because we also want to read branches from parent files
198 // selectively and thus we need to filter the branches there anyway.
199 // Here we just do it to ensure all files we read directly (which is
200 // 99% of the use case) contain all the branches we want.
201 requiredEventBranches = RootIOUtilities::filterBranches(requiredEventBranches, m_branchNames[DataStore::c_Event],
203 // but make sure we always have EventMetaData ...
204 requiredEventBranches.emplace("EventMetaData");
205
206 // Same for persistent data ...
207 requiredPersistentBranches = fileInfo.getBranchNames(true);
208 // filter the branches depending on what the user selected
209 requiredPersistentBranches = RootIOUtilities::filterBranches(requiredPersistentBranches, m_branchNames[DataStore::c_Persistent],
211 } else {
212 // ok we already have the list ... so let's make sure following files have the same branches
213 fileInfo.checkMissingBranches(requiredEventBranches, false);
214 fileInfo.checkMissingBranches(requiredPersistentBranches, true);
215 }
216 // ok, so now we have the file, add it to the chain. We trust the amount of events from metadata here.
217 if (m_tree->AddFile(fileName.c_str(), meta.getNEvents()) == 0 || m_persistent->AddFile(fileName.c_str(), 1) == 0) {
218 throw std::runtime_error("Could not add file to TChain");
219 }
220 B2INFO("Added file " + fileName);
221 } catch (std::exception& e) {
222 B2FATAL("Could not open input file " << std::quoted(fileName) << ": " << e.what());
223 }
224 }
225 }
226
227 if (m_tree->GetNtrees() == 0) B2FATAL("No file could be opened, aborting");
228 // Set cache size TODO: find out if files are remote and use a bigger default
229 // value if at least one file is non-local
230 if (m_cacheSize >= 0) m_tree->SetCacheSize(m_cacheSize * 1024 * 1024);
231
232 // Check if the files we added to the Chain are unique,
233 // if the same file is added multiple times the TEventList used for the eventSequence feature
234 // will process each file only once with the union of both given sequences.
235 // It is not clear if the user wants this, so we raise a fatal in this situation.
236 {
237 std::set<std::string> unique_filenames;
238
239 // The following lines are directly from the ROOT documentation
240 // see TChain::AddFile
241 TObjArray* fileElements = m_tree->GetListOfFiles();
242 TIter next(fileElements);
243 TChainElement* chEl = nullptr;
244 while ((chEl = (TChainElement*)next())) {
245 if (!unique_filenames.insert(chEl->GetTitle()).second) {
246 B2WARNING("The input file '" << chEl->GetTitle() << "' was specified more than once");
247 // seems we have duplicate files so we process events more than once. Disable forwarding of MC event number
248 m_processingAllEvents = false;
249 }
250 }
251 if ((unsigned int)m_tree->GetNtrees() != unique_filenames.size() && m_entrySequences.size() > 0) {
252 B2FATAL("You specified a file multiple times, and specified a sequence of entries which should be used for each file. "
253 "Please specify each file only once if you're using the sequence feature!");
254 }
255 }
256
257 if (m_entrySequences.size() > 0) {
258 auto* elist = new TEventList("input_event_list");
259 for (unsigned int iFile = 0; iFile < m_entrySequences.size(); ++iFile) {
260 int64_t offset = m_tree->GetTreeOffset()[iFile];
261 int64_t next_offset = m_tree->GetTreeOffset()[iFile + 1];
262 // check if Sequence consists only of ':', e.g. the whole file is requested
263 if (m_entrySequences[iFile] == ":") {
264 for (int64_t global_entry = offset; global_entry < next_offset; ++global_entry)
265 elist->Enter(global_entry);
266 } else {
267 for (const auto& entry : generate_number_sequence(m_entrySequences[iFile])) {
268 int64_t global_entry = entry + offset;
269 if (global_entry >= next_offset) {
270 B2WARNING("Given sequence contains entry numbers which are out of range. "
271 "I won't add any further events to the EventList for the current file.");
272 break;
273 } else {
274 elist->Enter(global_entry);
275 }
276 }
277 }
278 }
279 m_tree->SetEventList(elist);
280 }
281
282 B2DEBUG(33, "Opened tree '" + c_treeNames[DataStore::c_Persistent] + "'" << LogVar("entries", m_persistent->GetEntriesFast()));
283 B2DEBUG(33, "Opened tree '" + c_treeNames[DataStore::c_Event] + "'" << LogVar("entries", m_tree->GetEntriesFast()));
284
287
289 delete m_tree;
290 m_tree = nullptr; //don't try to read from there
291 } else {
294 }
295
296 if (m_parentLevel > 0) {
298 } else if (m_parentLevel < 0) {
299 B2ERROR("parentLevel must be >= 0!");
300 return;
301 }
302
303 // Let's check check if we process everything
304 // * all filenames unique (already done above)
305 // * no event skipping either with skipN, entry sequences or skipToEvent
306 // * no -n or process(path, N) with N <= the number of entries in our files
307 unsigned int maxEvent = Environment::Instance().getNumberEventsOverride();
310
311 if (!m_skipToEvent.empty()) {
312 // Skipping to some specific event is also not processing all events ...
313 m_processingAllEvents = false;
314 // make sure the number of entries is exactly 3
315 if (m_skipToEvent.size() != 3) {
316 B2ERROR("skipToEvent must be a list of three values: experiment, run, event number");
317 // ignore the value
318 m_skipToEvent.clear();
319 } else {
321 }
322 if (m_nextEntry > 0) {
323 B2ERROR("You cannot supply a number of events to skip (skipNEvents) and an "
324 "event to skip to (skipToEvent) at the same time, ignoring skipNEvents");
325 //force the number of skipped events to be zero
326 m_nextEntry = 0;
327 }
328 }
329
330 // Tell the InputController which event will be processed first
331 // (important if we want to do event mixing and skip some events in the input)
332 if (m_nextEntry > 0) {
334 }
335
336 // Processing everything so forward number of MC events
338 Environment::Instance().setNumberOfMCEvents(sumInputMCEvents);
339 }
340 // And setup global tag replay ...
342}
static Configuration & getInstance()
Get a reference to the instance which will be used when the Database is initialized.
void setInputMetadata(const std::vector< FileMetaData > &inputMetadata)
To be called by input modules with the list of all input FileMetaData.
unsigned int getSkipEventsOverride() const
Get skipNEvents override, or 0 if unset.
Definition: Environment.h:89
std::vector< std::string > getEntrySequencesOverride() const
Returns the number sequences (e.g.
Definition: Environment.h:73
unsigned int getNumberEventsOverride() const
Returns number of events in run 1 for EventInfoSetter module, or 0 for no override.
Definition: Environment.h:67
void setNumberOfMCEvents(unsigned int n)
Set number of generated events (for EventInfoSetter).
Definition: Environment.h:109
Metadata information about a file.
Definition: FileMetaData.h:29
bool isMC() const
Is it generated data?.
Definition: FileMetaData.h:119
unsigned int getNEvents() const
Number of events getter.
Definition: FileMetaData.h:41
unsigned int getMcEvents() const
Number of generated events getter.
Definition: FileMetaData.h:123
static void setNextEntry(long entry, bool independentPath=false)
Set the file entry to be loaded the next time event() is called.
static void setSkippedEntries(long entries, bool independentPath=false)
set the number of entries skipped by the RootInputModule.
static void setChain(const TChain *chain, bool independentPath=false)
Set the loaded TChain (event durability).
static void setCanControlInput(bool on)
Call this function from supported input modules.
Helper class to factorize some necessary tasks when working with Belle2 output files.
Definition: RootFileInfo.h:27
StoreEntries m_persistentStoreEntries
Vector of DataStore entries of persistent durability that we are supposed to read in.
void realDataWorkaround(FileMetaData &metaData)
Correct isMC flag for raw data recorded before experiment 8 run 2364.
virtual std::vector< std::string > getFileNames(bool outputFiles=false) override
Get list of input files, taking -i command line overrides into account.
void readPersistentEntry(long fileEntry)
Loads given entry from persistent tree.
bool createParentStoreEntries()
Connect the parent trees and fill m_parentStoreEntries.
bool m_processingAllEvents
Set to true if we process the input files completely: No skip events or sequences or -n parameters.
static ScopeGuard guardValue(T &reference)
Create a ScopeGuard for a value: The content of reference will be copied and reset when the returned ...
Definition: ScopeGuard.h:76
std::set< int64_t > generate_number_sequence(const std::string &str)
Generate a sequence of numbers defined by a string.
std::vector< std::string > expandWordExpansions(const std::vector< std::string > &filenames)
Performs wildcard expansion using wordexp(), returns matches.

◆ readParentTrees()

bool readParentTrees ( )
private

Read data of the current event from the parents.

Definition at line 641 of file RootInputModule.cc.

642{
643 const StoreObjPtr<EventMetaData> eventMetaData;
644 int experiment = eventMetaData->getExperiment();
645 int run = eventMetaData->getRun();
646 unsigned int event = eventMetaData->getEvent();
647
648 std::string parentLfn = eventMetaData->getParentLfn();
649 for (int level = 0; level < m_parentLevel; level++) {
650 const std::string& parentPfn = FileCatalog::Instance().getPhysicalFileName(parentLfn);
651
652 // Open the parent file if we haven't done this already
653 TTree* tree = nullptr;
654 if (m_parentTrees.find(parentLfn) == m_parentTrees.end()) {
655 TDirectory* dir = gDirectory;
656 B2DEBUG(30, "Opening parent file" << LogVar("PFN", parentPfn));
657 TFile* file = TFile::Open(parentPfn.c_str(), "READ");
658 dir->cd();
659 if (!file || !file->IsOpen()) {
660 B2ERROR("Couldn't open parent file " << parentPfn);
661 return false;
662 }
663 tree = dynamic_cast<TTree*>(file->Get(c_treeNames[DataStore::c_Event].c_str()));
664 if (!tree) {
665 B2ERROR("No tree " << c_treeNames[DataStore::c_Event] << " found in " << parentPfn);
666 return false;
667 }
668 for (auto entry : m_parentStoreEntries[level]) {
669 tree->SetBranchAddress(entry->name.c_str(), &(entry->object));
670 }
671 m_parentTrees.insert(std::make_pair(parentLfn, tree));
672 } else {
673 tree = m_parentTrees[parentLfn];
674 }
675
676 // get entry number in parent tree
677 long entryNumber = RootIOUtilities::getEntryNumberWithEvtRunExp(tree, event, run, experiment);
678 if (entryNumber < 0) {
679 B2ERROR("No event " << experiment << "/" << run << "/" << event << " in parent file " << parentPfn);
680 return false;
681 }
682
683 // read the tree and mark the data read in the data store
684 EventMetaData* parentMetaData = nullptr;
685 tree->SetBranchAddress("EventMetaData", &parentMetaData);
686 tree->GetEntry(entryNumber);
687 for (auto entry : m_parentStoreEntries[level]) {
688 entry->ptr = entry->object;
689 }
690
691 // set the parent LFN to the next level
692 parentLfn = parentMetaData->getParentLfn();
693 }
694
695 addEventListForIndexFile(parentLfn);
696
697 return true;
698}
void addEventListForIndexFile(const std::string &parentLfn)
For index files, this creates TEventList/TEntryListArray to enable better cache use.

◆ readPersistentEntry()

void readPersistentEntry ( long  fileEntry)
private

Loads given entry from persistent tree.

Definition at line 754 of file RootInputModule.cc.

755{
756 m_lastPersistentEntry = fileEntry;
757
758 for (auto entry : m_persistentStoreEntries) {
759 bool isMergeable = entry->object->InheritsFrom(Mergeable::Class());
760 TObject* copyOfPreviousVersion = nullptr;
761 if (isMergeable) {
762 copyOfPreviousVersion = entry->object->Clone();
763 }
764 entry->resetForGetEntry();
765 //ptr stores old value (or nullptr)
766 entry->ptr = copyOfPreviousVersion;
767 }
768
769 int bytesRead = m_persistent->GetEntry(fileEntry);
770 if (bytesRead <= 0) {
771 const char* name = m_tree->GetCurrentFile() ? m_tree->GetCurrentFile()->GetName() : "<unknown>";
772 B2FATAL("Could not read 'persistent' TTree #" << fileEntry << " in file " << name);
773 }
774
775 for (auto entry : m_persistentStoreEntries) {
776 if (entry->object) {
777 bool isMergeable = entry->object->InheritsFrom(Mergeable::Class());
778 if (isMergeable) {
779 const Mergeable* oldObj = static_cast<Mergeable*>(entry->ptr);
780 auto* newObj = static_cast<Mergeable*>(entry->object);
781 newObj->merge(oldObj);
782
783 delete entry->ptr;
784 }
785 entry->ptr = entry->object;
786 } else {
787 entryNotFound("Persistent tree", entry->name);
788 entry->recoverFromNullObject();
789 entry->ptr = nullptr;
790 }
791 }
792}
Abstract base class for objects that can be merged.
Definition: Mergeable.h:31
virtual void merge(const Mergeable *other)=0
Merge object 'other' into this one.
void entryNotFound(const std::string &entryOrigin, const std::string &name, bool fileChanged=true)
Check if we warn the user or abort after an entry was missing after changing files.

◆ readTree()

void readTree ( )
private

Actually performs the reading from the tree.

Definition at line 427 of file RootInputModule.cc.

428{
429 if (!m_tree)
430 return;
431
432 //keep snapshot of TFile stats (to use if it changes)
433 ReadStats currentEventStats;
435 currentEventStats.addFromFile(m_tree->GetFile());
436 }
437
438 // Check if there are still new entries available.
439 int localEntryNumber = m_nextEntry;
440 if (m_entrySequences.size() > 0) {
441 localEntryNumber = m_tree->GetEntryNumber(localEntryNumber);
442 }
443 localEntryNumber = m_tree->LoadTree(localEntryNumber);
444
445 if (localEntryNumber == -2) {
446 m_nextEntry = -2;
447 return; //end of file
448 } else if (localEntryNumber < 0) {
449 B2FATAL("Failed to load tree, corrupt file? Check standard error for additional messages. TChain::LoadTree() returned" <<
450 LogVar("error", localEntryNumber));
451 }
452 B2DEBUG(39, "Reading file entry " << m_nextEntry);
453
454 //Make sure transient members of objects are reinitialised
455 for (auto entry : m_storeEntries) {
456 entry->resetForGetEntry();
457 }
458 for (const auto& storeEntries : m_parentStoreEntries) {
459 for (auto entry : storeEntries) {
460 entry->resetForGetEntry();
461 }
462 }
463
464 int bytesRead = m_tree->GetTree()->GetEntry(localEntryNumber);
465 if (bytesRead <= 0) {
466 B2FATAL("Could not read 'tree' entry " << m_nextEntry << " in file " << m_tree->GetCurrentFile()->GetName());
467 }
468
469 //In case someone is tempted to change this:
470 // TTree::GetCurrentFile() returns a TFile pointer to a fixed location,
471 // calling GetName() on the TFile almost works as expected, but starts with the
472 // last file in a TChain. (-> we re-read the first persistent tree with TChain,
473 // with ill results for Mergeable objects.)
474 // GetTreeNumber() also starts at the last entry before we read the first event from m_tree,
475 // so we'll save the last persistent tree loaded and only reload on changes.
477 const long treeNum = m_tree->GetTreeNumber();
478 const bool fileChanged = (m_lastPersistentEntry != treeNum);
479 if (fileChanged) {
481 m_readStats.add(currentEventStats);
482 }
483 // file changed, read the FileMetaData object from the persistent tree and update the parent file metadata
484 readPersistentEntry(treeNum);
485 B2INFO("Loading new input file"
486 << LogVar("filename", m_tree->GetFile()->GetName())
487 << LogVar("metadata LFN", fileMetaData->getLfn()));
488 }
489 realDataWorkaround(*fileMetaData);
490
491 for (auto entry : m_storeEntries) {
492 if (!entry->object) {
493 entryNotFound("Event durability tree (global entry: " + std::to_string(m_nextEntry) + ")", entry->name, fileChanged);
494 entry->recoverFromNullObject();
495 entry->ptr = nullptr;
496 } else {
497 entry->ptr = entry->object;
498 }
499 }
500
501 if (m_parentLevel > 0) {
502 if (!readParentTrees())
503 B2FATAL("Could not read data from parent file!");
504 }
505
506 // Nooow, if the object didn't exist in the event when we wrote it to File we still have it in the file but it's marked as invalid Object.
507 // So go through everything and check for the bit and invalidate as necessary
508 for (auto entry : m_storeEntries) {
509 if (entry->object->TestBit(kInvalidObject)) entry->invalidate();
510 }
511 for (const auto& storeEntries : m_parentStoreEntries) {
512 for (auto entry : storeEntries) {
513 if (entry->object->TestBit(kInvalidObject)) entry->invalidate();
514 }
515 }
516}
ReadStats m_readStats
some statistics for all files read so far.
bool readParentTrees()
Read data of the current event from the parents.
void add(const ReadStats &b)
add other stats object.

◆ realDataWorkaround()

void realDataWorkaround ( FileMetaData metaData)
private

Correct isMC flag for raw data recorded before experiment 8 run 2364.

Definition at line 794 of file RootInputModule.cc.

795{
796 if ((metaData.getSite().find("bfe0") == 0) && (metaData.getDate().compare("2019-06-30") < 0) &&
797 (metaData.getExperimentLow() > 0) && (metaData.getExperimentHigh() < 9) && (metaData.getRunLow() > 0)) {
798 metaData.declareRealData();
799 }
800}
int getRunLow() const
Lowest run number getter.
Definition: FileMetaData.h:53
void declareRealData()
Declare that this is not generated, but real data.
Definition: FileMetaData.h:294
const std::string & getDate() const
File creation date and time getter (UTC)
Definition: FileMetaData.h:95
const std::string & getSite() const
Site where the file was created getter.
Definition: FileMetaData.h:99
int getExperimentLow() const
Lowest experiment number getter.
Definition: FileMetaData.h:49
int getExperimentHigh() const
Highest experiment number getter.
Definition: FileMetaData.h:61

◆ setAbortLevel()

void setAbortLevel ( int  abortLevel)
inherited

Configure the abort log level.

Definition at line 67 of file Module.cc.

68{
69 m_logConfig.setAbortLevel(static_cast<LogConfig::ELogLevel>(abortLevel));
70}
ELogLevel
Definition of the supported log levels.
Definition: LogConfig.h:26
void setAbortLevel(ELogLevel abortLevel)
Configure the abort level.
Definition: LogConfig.h:112

◆ setDebugLevel()

void setDebugLevel ( int  debugLevel)
inherited

Configure the debug messaging level.

Definition at line 61 of file Module.cc.

62{
63 m_logConfig.setDebugLevel(debugLevel);
64}
void setDebugLevel(int debugLevel)
Configure the debug messaging level.
Definition: LogConfig.h:98

◆ setDescription()

void setDescription ( const std::string &  description)
protectedinherited

Sets the description of the module.

Parameters
descriptionA description of the module.

Definition at line 214 of file Module.cc.

215{
216 m_description = description;
217}

◆ setLogConfig()

void setLogConfig ( const LogConfig logConfig)
inlineinherited

Set the log system configuration.

Definition at line 230 of file Module.h.

230{m_logConfig = logConfig;}

◆ setLogInfo()

void setLogInfo ( int  logLevel,
unsigned int  logInfo 
)
inherited

Configure the printed log information for the given level.

Parameters
logLevelThe log level (one of LogConfig::ELogLevel)
logInfoWhat kind of info should be printed? ORed combination of LogConfig::ELogInfo flags.

Definition at line 73 of file Module.cc.

74{
75 m_logConfig.setLogInfo(static_cast<LogConfig::ELogLevel>(logLevel), logInfo);
76}
void setLogInfo(ELogLevel logLevel, unsigned int logInfo)
Configure the printed log information for the given level.
Definition: LogConfig.h:127

◆ setLogLevel()

void setLogLevel ( int  logLevel)
inherited

Configure the log level.

Definition at line 55 of file Module.cc.

56{
57 m_logConfig.setLogLevel(static_cast<LogConfig::ELogLevel>(logLevel));
58}
void setLogLevel(ELogLevel logLevel)
Configure the log level.
Definition: LogConfig.cc:25

◆ setName()

void setName ( const std::string &  name)
inlineinherited

Set the name of the module.

Note
The module name is set when using the REG_MODULE macro, but the module can be renamed before calling process() using the set_name() function in your steering file.
Parameters
nameThe name of the module

Definition at line 214 of file Module.h.

214{ m_name = name; };

◆ setParamList()

void setParamList ( const ModuleParamList params)
inlineprotectedinherited

Replace existing parameter list.

Definition at line 501 of file Module.h.

501{ m_moduleParamList = params; }

◆ setParamPython()

void setParamPython ( const std::string &  name,
const boost::python::object &  pyObj 
)
privateinherited

Implements a method for setting boost::python objects.

The method supports the following types: list, dict, int, double, string, bool The conversion of the python object to the C++ type and the final storage of the parameter value is done in the ModuleParam class.

Parameters
nameThe unique name of the parameter.
pyObjThe object which should be converted and stored as the parameter value.

Definition at line 234 of file Module.cc.

235{
236 LogSystem& logSystem = LogSystem::Instance();
237 logSystem.updateModule(&(getLogConfig()), getName());
238 try {
240 } catch (std::runtime_error& e) {
241 throw std::runtime_error("Cannot set parameter '" + name + "' for module '"
242 + m_name + "': " + e.what());
243 }
244
245 logSystem.updateModule(nullptr);
246}
Class for logging debug, info and error messages.
Definition: LogSystem.h:46
void updateModule(const LogConfig *moduleLogConfig=nullptr, const std::string &moduleName="")
Sets the log configuration to the given module log configuration and sets the module name This method...
Definition: LogSystem.h:191
static LogSystem & Instance()
Static method to get a reference to the LogSystem instance.
Definition: LogSystem.cc:31
void setParamPython(const std::string &name, const PythonObject &pyObj)
Implements a method for setting boost::python objects.

◆ setParamPythonDict()

void setParamPythonDict ( const boost::python::dict &  dictionary)
privateinherited

Implements a method for reading the parameter values from a boost::python dictionary.

The key of the dictionary has to be the name of the parameter and the value has to be of one of the supported parameter types.

Parameters
dictionaryThe python dictionary from which the parameter values are read.

Definition at line 249 of file Module.cc.

250{
251
252 LogSystem& logSystem = LogSystem::Instance();
253 logSystem.updateModule(&(getLogConfig()), getName());
254
255 boost::python::list dictKeys = dictionary.keys();
256 int nKey = boost::python::len(dictKeys);
257
258 //Loop over all keys in the dictionary
259 for (int iKey = 0; iKey < nKey; ++iKey) {
260 boost::python::object currKey = dictKeys[iKey];
261 boost::python::extract<std::string> keyProxy(currKey);
262
263 if (keyProxy.check()) {
264 const boost::python::object& currValue = dictionary[currKey];
265 setParamPython(keyProxy, currValue);
266 } else {
267 B2ERROR("Setting the module parameters from a python dictionary: invalid key in dictionary!");
268 }
269 }
270
271 logSystem.updateModule(nullptr);
272}
void setParamPython(const std::string &name, const boost::python::object &pyObj)
Implements a method for setting boost::python objects.
Definition: Module.cc:234

◆ setPropertyFlags()

void setPropertyFlags ( unsigned int  propertyFlags)
inherited

Sets the flags for the module properties.

Parameters
propertyFlagsbitwise OR of EModulePropFlags

Definition at line 208 of file Module.cc.

209{
210 m_propertyFlags = propertyFlags;
211}

◆ setReturnValue() [1/2]

void setReturnValue ( bool  value)
protectedinherited

Sets the return value for this module as bool.

The bool value is saved as an integer with the convention 1 meaning true and 0 meaning false. The value can be used in the steering file to divide the analysis chain into several paths.

Parameters
valueThe value of the return value.

Definition at line 227 of file Module.cc.

228{
229 m_hasReturnValue = true;
230 m_returnValue = value;
231}

◆ setReturnValue() [2/2]

void setReturnValue ( int  value)
protectedinherited

Sets the return value for this module as integer.

The value can be used in the steering file to divide the analysis chain into several paths.

Parameters
valueThe value of the return value.

Definition at line 220 of file Module.cc.

221{
222 m_hasReturnValue = true;
223 m_returnValue = value;
224}

◆ setType()

void setType ( const std::string &  type)
protectedinherited

Set the module type.

Only for use by internal modules (which don't use the normal REG_MODULE mechanism).

Definition at line 48 of file Module.cc.

49{
50 if (!m_type.empty())
51 B2FATAL("Trying to change module type from " << m_type << " is not allowed, the value is assumed to be fixed.");
52 m_type = type;
53}

◆ terminate()

void terminate ( )
overridevirtual

Is called at the end of your Module.

Reimplemented from Module.

Definition at line 395 of file RootInputModule.cc.

396{
397 if (m_collectStatistics and m_tree) {
398 //add stats for last file
399 m_readStats.addFromFile(m_tree->GetFile());
400 }
401 delete m_tree;
402 delete m_persistent;
403 ReadStats parentReadStats;
404 for (const auto& entry : m_parentTrees) {
405 TFile* f = entry.second->GetCurrentFile();
407 parentReadStats.addFromFile(f);
408
409 delete f;
410 }
411
413 B2INFO("Statistics for event tree: " << m_readStats.getString());
414 B2INFO("Statistics for event tree (parent files): " << parentReadStats.getString());
415 }
416
417 for (auto& branch : m_connectedBranches) {
418 branch.clear();
419 }
420 m_storeEntries.clear();
422 m_parentStoreEntries.clear();
423 m_parentTrees.clear();
424}
std::string getString() const
string suitable for printing.
void addFromFile(const TFile *f)
add current statistics from TFile object.

Member Data Documentation

◆ m_branchNames

std::vector<std::string> m_branchNames[DataStore::c_NDurabilityTypes]
private

Array for names of branches, that shall be written out.

Empty vector results in all branches being read. These vectors can be configured in the steering file.

Definition at line 133 of file RootInputModule.h.

◆ m_cacheSize

int m_cacheSize {0}
private

Input ROOT File Cache size in MB, <0 means default.

Definition at line 221 of file RootInputModule.h.

◆ m_collectStatistics

bool m_collectStatistics
private

Collect statistics on amount of data read and print statistics (seperate for input & parent files) after processing.

Definition at line 150 of file RootInputModule.h.

◆ m_conditions

std::vector<ModuleCondition> m_conditions
privateinherited

Module condition, only non-null if set.

Definition at line 521 of file Module.h.

◆ m_connectedBranches

std::set<std::string> m_connectedBranches[DataStore::c_NDurabilityTypes]
private

Already connected branches.

Definition at line 174 of file RootInputModule.h.

◆ m_description

std::string m_description
privateinherited

The description of the module.

Definition at line 511 of file Module.h.

◆ m_discardErrorEvents

bool m_discardErrorEvents {true}
private

Discard events that have an error flag != 0.

Definition at line 224 of file RootInputModule.h.

◆ m_discardErrorMask

unsigned int m_discardErrorMask {EventMetaData::c_HLTDiscard}
private

Don't issue a warning when discarding events if the error flag consists exclusively of flags in this mask.

Definition at line 226 of file RootInputModule.h.

◆ m_entrySequences

std::vector<std::string> m_entrySequences
private

The number sequences (e.g.

23:42,101) defining the entries which are processed for each inputFileName.

Definition at line 122 of file RootInputModule.h.

◆ m_excludeBranchNames

std::vector<std::string> m_excludeBranchNames[DataStore::c_NDurabilityTypes]
private

Array for names of branches that should NOT be written out.

This takes precedence over m_branchNames, so if a branch is in both m_branchNames[d] and m_excludeBranchNames[d], it is not saved.

Definition at line 140 of file RootInputModule.h.

◆ m_hasReturnValue

bool m_hasReturnValue
privateinherited

True, if the return value is set.

Definition at line 518 of file Module.h.

◆ m_ignoreCommandLineOverride

bool m_ignoreCommandLineOverride
private

Ignore filename override from command line.

Definition at line 125 of file RootInputModule.h.

◆ m_inputFileName

std::string m_inputFileName
private

File to read from.

Cannot be used together with m_inputFileNames.

Definition at line 114 of file RootInputModule.h.

◆ m_inputFileNames

std::vector<std::string> m_inputFileNames
private

Files to read from.

Definition at line 117 of file RootInputModule.h.

◆ m_isSecondaryInput

bool m_isSecondaryInput {false}
private

When using a second RootInputModule in an independent path [usually if you are using add_independent_merge_path(...)] this has to be set to true.

Definition at line 234 of file RootInputModule.h.

◆ m_lastParentFileLFN

std::string m_lastParentFileLFN
private

last parent file LFN seen.

(used by addEventListForIndexFile())

Definition at line 164 of file RootInputModule.h.

◆ m_lastPersistentEntry

long m_lastPersistentEntry
private

last entry to be in persistent tree.


Definition at line 161 of file RootInputModule.h.

◆ m_logConfig

LogConfig m_logConfig
privateinherited

The log system configuration of the module.

Definition at line 514 of file Module.h.

◆ m_moduleParamList

ModuleParamList m_moduleParamList
privateinherited

List storing and managing all parameter of the module.

Definition at line 516 of file Module.h.

◆ m_name

std::string m_name
privateinherited

The name of the module, saved as a string (user-modifiable)

Definition at line 508 of file Module.h.

◆ m_nextEntry

long m_nextEntry
private

Next entry to be read in event tree.


Definition at line 158 of file RootInputModule.h.

◆ m_package

std::string m_package
privateinherited

Package this module is found in (may be empty).

Definition at line 510 of file Module.h.

◆ m_parentLevel

int m_parentLevel
private

Level of parent files to be read.

Definition at line 147 of file RootInputModule.h.

◆ m_parentStoreEntries

std::vector<StoreEntries> m_parentStoreEntries
private

The parent DataStore entries per level.

Definition at line 182 of file RootInputModule.h.

◆ m_parentTrees

std::map<std::string, TTree*> m_parentTrees
private

Map of file LFNs to trees.

Definition at line 185 of file RootInputModule.h.

◆ m_persistent

TChain* m_persistent
private

TTree for persistent input.

Definition at line 171 of file RootInputModule.h.

◆ m_persistentStoreEntries

StoreEntries m_persistentStoreEntries
private

Vector of DataStore entries of persistent durability that we are supposed to read in.

Definition at line 179 of file RootInputModule.h.

◆ m_processingAllEvents

bool m_processingAllEvents {true}
private

Set to true if we process the input files completely: No skip events or sequences or -n parameters.

Definition at line 229 of file RootInputModule.h.

◆ m_propertyFlags

unsigned int m_propertyFlags
privateinherited

The properties of the module as bitwise or (with |) of EModulePropFlags.

Definition at line 512 of file Module.h.

◆ m_readStats

ReadStats m_readStats
private

some statistics for all files read so far.

Definition at line 218 of file RootInputModule.h.

◆ m_returnValue

int m_returnValue
privateinherited

The return value.

Definition at line 519 of file Module.h.

◆ m_skipNEvents

unsigned int m_skipNEvents
private

Can be set from steering file to skip the first N events.

Definition at line 144 of file RootInputModule.h.

◆ m_skipToEvent

std::vector<int> m_skipToEvent
private

experiment, run, event number of first event to load

Definition at line 153 of file RootInputModule.h.

◆ m_storeEntries

StoreEntries m_storeEntries
private

Vector of DataStore entries of event durability that we are supposed to read in.

Definition at line 177 of file RootInputModule.h.

◆ m_tree

TChain* m_tree
private

TTree for event input.

Definition at line 168 of file RootInputModule.h.

◆ m_type

std::string m_type
privateinherited

The type of the module, saved as a string.

Definition at line 509 of file Module.h.


The documentation for this class was generated from the following files: