SubEventModule Class Reference

Framework-internal module that implements the functionality of Path::forEach() as well as Path::doWhile(). More...

#include <SubEventModule.h>

Inheritance diagram for SubEventModule:

Public Types
enum	EModes {   c_ForEach = 0 ,   c_DoWhile = 1 }
	Define the constants for the different modes. More...
enum	EModulePropFlags {   c_Input = 1 ,   c_Output = 2 ,   c_ParallelProcessingCertified = 4 ,   c_HistogramManager = 8 ,   c_InternalSerializer = 16 ,   c_TerminateInAllProcesses = 32 ,   c_DontCollectStatistics = 64 }
	Each module can be tagged with property flags, which indicate certain features of the module. More...
typedef ModuleCondition::EAfterConditionPath	EAfterConditionPath
	Forward the EAfterConditionPath definition from the ModuleCondition.

Public Member Functions
void	initSubEvent (const std::string &objectName, const std::string &loopOver, std::shared_ptr< Path > path)
	used by Path::forEach() to actually set parameters.
void	initSubLoop (std::shared_ptr< Path > path, const std::string &condition, unsigned int maxIterations)
	ised by Path::doWhile() to actually set parameters
virtual void	initialize () override
	Initialize the Module.
virtual void	beginRun () override
	Called when entering a new run.
virtual void	endRun () override
	This method is called if the current run ends.
virtual void	event () override
	This method is the core of the module.
virtual void	terminate () override
	This method is called at the end of the event processing.
virtual std::vector< std::string >	getFileNames (bool outputFiles)
	Return a list of output filenames for this modules.
const std::string &	getName () const
	Returns the name of the module.
const std::string &	getType () const
	Returns the type of the module (i.e.
const std::string &	getPackage () const
	Returns the package this module is in.
const std::string &	getDescription () const
	Returns the description of the module.
void	setName (const std::string &name)
	Set the name of the module.
void	setPropertyFlags (unsigned int propertyFlags)
	Sets the flags for the module properties.
LogConfig &	getLogConfig ()
	Returns the log system configuration.
void	setLogConfig (const LogConfig &logConfig)
	Set the log system configuration.
void	setLogLevel (int logLevel)
	Configure the log level.
void	setDebugLevel (int debugLevel)
	Configure the debug messaging level.
void	setAbortLevel (int abortLevel)
	Configure the abort log level.
void	setLogInfo (int logLevel, unsigned int logInfo)
	Configure the printed log information for the given level.
void	if_value (const std::string &expression, const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
	Add a condition to the module.
void	if_false (const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
	A simplified version to add a condition to the module.
void	if_true (const std::shared_ptr< Path > &path, EAfterConditionPath afterConditionPath=EAfterConditionPath::c_End)
	A simplified version to set the condition of the module.
bool	hasCondition () const
	Returns true if at least one condition was set for the module.
const ModuleCondition *	getCondition () const
	Return a pointer to the first condition (or nullptr, if none was set)
const std::vector< ModuleCondition > &	getAllConditions () const
	Return all set conditions for this module.
bool	evalCondition () const
	If at least one condition was set, it is evaluated and true returned if at least one condition returns true.
std::shared_ptr< Path >	getConditionPath () const
	Returns the path of the last true condition (if there is at least one, else reaturn a null pointer).
Module::EAfterConditionPath	getAfterConditionPath () const
	What to do after the conditional path is finished.
std::vector< std::shared_ptr< Path > >	getAllConditionPaths () const
	Return all condition paths currently set (no matter if the condition is true or not).
bool	hasProperties (unsigned int propertyFlags) const
	Returns true if all specified property flags are available in this module.
bool	hasUnsetForcedParams () const
	Returns true and prints error message if the module has unset parameters which the user has to set in the steering file.
const ModuleParamList &	getParamList () const
	Return module param list.
template<typename T >
ModuleParam< T > &	getParam (const std::string &name) const
	Returns a reference to a parameter.
bool	hasReturnValue () const
	Return true if this module has a valid return value set.
int	getReturnValue () const
	Return the return value set by this module.
std::shared_ptr< PathElement >	clone () const override
	Create an independent copy of this module.
std::shared_ptr< boost::python::list >	getParamInfoListPython () const
	Returns a python list of all parameters.
void	process (const PathPtr &startPath, long maxEvent=0)
	Processes the full module chain, starting with the first module in the given path.
void	setProfileModuleName (const std::string &name)
	Set the name of the module we want to profile.

Static Public Member Functions
static void	exposePythonAPI ()
	Exposes methods of the Module class to Python.
static void	writeToStdErr (const char msg[])
	async-safe method to write something to STDERR.
static void	installSignalHandler (int sig, void(*fn)(int))
	Install a signal handler 'fn' for given signal.
static void	installMainSignalHandlers (void(*fn)(int)=nullptr)
	Install signal handler for INT, TERM and QUIT signals.

Protected Member Functions
virtual void	def_initialize ()
	Wrappers to make the methods without "def_" prefix callable from Python.
virtual void	def_beginRun ()
	Wrapper method for the virtual function beginRun() that has the implementation to be used in a call from Python.
virtual void	def_event ()
	Wrapper method for the virtual function event() that has the implementation to be used in a call from Python.
virtual void	def_endRun ()
	This method can receive that the current run ends as a call from the Python side.
virtual void	def_terminate ()
	Wrapper method for the virtual function terminate() that has the implementation to be used in a call from Python.
void	setDescription (const std::string &description)
	Sets the description of the module.
void	setType (const std::string &type)
	Set the module type.
template<typename T >
void	addParam (const std::string &name, T &paramVariable, const std::string &description, const T &defaultValue)
	Adds a new parameter to the module.
template<typename T >
void	addParam (const std::string &name, T &paramVariable, const std::string &description)
	Adds a new enforced parameter to the module.
void	setReturnValue (int value)
	Sets the return value for this module as integer.
void	setReturnValue (bool value)
	Sets the return value for this module as bool.
void	setParamList (const ModuleParamList &params)
	Replace existing parameter list.
void	processInitialize (const ModulePtrList &modulePathList, bool setEventInfo=true)
	Initializes the modules.
void	processCore (const PathPtr &startPath, const ModulePtrList &modulePathList, long maxEvent=0, bool isInputProcess=true)
	Processes the full module chain consisting of an arbitrary number of connected paths, starting with the first module in the specified path.
bool	processEvent (PathIterator moduleIter, bool skipMasterModule)
	Calls event() functions on all modules for the current event.
void	callEvent (Module *module)
	Calls event() on one single module, setting up logging and statistics as needed.
void	processTerminate (const ModulePtrList &modulePathList)
	Terminates the modules.
void	processBeginRun (bool skipDB=false)
	Calls the begin run methods of all modules.
void	processEndRun ()
	Calls the end run methods of all modules.
long	getMaximumEventNumber (long maxEvent) const
	Calculate the maximum event number out of the argument from command line and the environment.

Protected Attributes
const Module *	m_master
	The master module that determines the experiment/run/event number.
ModulePtrList	m_moduleList
	List of all modules in order initialized.
std::string	m_profileModuleName
	Name of the module which should be profiled, empty if no profiling is requested.
Module *	m_profileModule = nullptr
	Adress of the module which we want to profile, nullptr if no profiling is requested.
StoreObjPtr< EventMetaData >	m_eventMetaDataPtr
	EventMetaData is used by processEvent()/processCore().
EventMetaData	m_previousEventMetaData
	Stores state of EventMetaData before it was last changed.
StoreObjPtr< ProcessStatistics >	m_processStatisticsPtr
	Also used in a number of places.
bool	m_inRun
	Are we currently in a run? If yes, processEndRun() needs to do something.
double	m_lastMetadataUpdate
	Time in seconds of last call for metadata update in event loop.
double	m_metadataUpdateInterval
	Minimal time difference in seconds for metadata updates in event loop.
bool	m_steerRootInputModuleOn = false
	True if the SteerRootInputModule is in charge for event processing.

Private Member Functions
void	setProperties ()
	Set properties for this module based on the modules found in m_path.
void	setDoWhileConditions ()
	Set the necessary pointers for do_while(): the pointer to the module whose return value we'll use as well as the ModuleCondition object.
std::list< ModulePtr >	getModules () const override
	no submodules, return empty list
std::string	getPathString () const override
	return the module name.
void	setParamPython (const std::string &name, const boost::python::object &pyObj)
	Implements a method for setting boost::python objects.
void	setParamPythonDict (const boost::python::dict &dictionary)
	Implements a method for reading the parameter values from a boost::python dictionary.

Private Attributes
std::optional< std::string >	m_objectName {std::nullopt}
	name of our loop variable in case of forEach.
std::optional< std::string >	m_loopOverName {std::nullopt}
	name for m_loopOver in case of forEach.
StoreArray< TObject >	m_loopOver
	array looped over in case of forEach
std::shared_ptr< Path >	m_path
	Path to execute.
int	m_processID { -1}
	when using multi-processing contains the ID of the process where event() is called (in that process only).
unsigned int	m_maxIterations {10000}
	maximum number of iterations before giving up in case of doWhile()
std::optional< std::string >	m_loopConditionString {std::nullopt}
	String for the condition when looping.
std::unique_ptr< ModuleCondition >	m_loopCondition
	Condition object to evaluate if the loop is finished in case of doWhile()
Module *	m_loopConditionModule {nullptr}
	pointer to the module to provide the returnValue in case of doWhile()
int	m_mode {c_ForEach}
	Mode for this module.
std::string	m_name
	The name of the module, saved as a string (user-modifiable)
std::string	m_type
	The type of the module, saved as a string.
std::string	m_package
	Package this module is found in (may be empty).
std::string	m_description
	The description of the module.
unsigned int	m_propertyFlags
	The properties of the module as bitwise or (with |) of EModulePropFlags.
LogConfig	m_logConfig
	The log system configuration of the module.
ModuleParamList	m_moduleParamList
	List storing and managing all parameter of the module.
bool	m_hasReturnValue
	True, if the return value is set.
int	m_returnValue
	The return value.
std::vector< ModuleCondition >	m_conditions
	Module condition, only non-null if set.

Detailed Description

Framework-internal module that implements the functionality of Path::forEach() as well as Path::doWhile().

Definition at line 26 of file SubEventModule.h.

Member Typedef Documentation

EAfterConditionPath

typedef ModuleCondition::EAfterConditionPath EAfterConditionPath

inherited

Forward the EAfterConditionPath definition from the ModuleCondition.

Definition at line 88 of file Module.h.

Member Enumeration Documentation

EModes

enum EModes

Define the constants for the different modes.

Definition at line 29 of file SubEventModule.h.

                {
      c_ForEach = 0,
      c_DoWhile = 1,
    };

EModulePropFlags

enum EModulePropFlags

inherited

Each module can be tagged with property flags, which indicate certain features of the module.

Enumerator
c_Input	This module is an input module (reads data).
c_Output	This module is an output module (writes data).
c_ParallelProcessingCertified	This module can be run in parallel processing mode safely (All I/O must be done through the data store, in particular, the module must not write any files.)
c_HistogramManager	This module is used to manage histograms accumulated by other modules.
c_InternalSerializer	This module is an internal serializer/deserializer for parallel processing.
c_TerminateInAllProcesses	When using parallel processing, call this module's terminate() function in all processes(). This will also ensure that there is exactly one process (single-core if no parallel modules found) or at least one input, one main and one output process.
c_DontCollectStatistics	No statistics is collected for this module.

Definition at line 77 of file Module.h.

                          {
      c_Input                       = 1,  
      c_Output                      = 2,  
      c_ParallelProcessingCertified = 4,  
      c_HistogramManager            = 8, 
      c_InternalSerializer          = 16,  
      c_TerminateInAllProcesses     = 32,  
      c_DontCollectStatistics       = 64,  
    };

Constructor & Destructor Documentation

SubEventModule()

SubEventModule

(

)

Definition at line 27 of file SubEventModule.cc.

                              : Module(), EventProcessor()
{
  // we need to deserialize a pickled execution path so this module needs to work
  // when using Path.add_module() directly ...
  setDescription(R"DOC(Internal module to handle Path.for_each() and Path.do_while().
 
  Warning:
    Don't add this module directly with `Path.add_module` or
    `basf2.register_module` but use `Path.for_each()` and `Path.do_while()`
 
  This module shouldn't appear in ``basf2 -m`` output.
  If it does, check REG_MODULE() handling.)DOC");
 
  addParam("mode", m_mode, "SubEvent mode: 0 = for_each, 1 = do_while", m_mode);
  addParam("loopOver", m_loopOverName, "Name of array to iterate over.", m_loopOverName);
  addParam("objectName", m_objectName, "Name of the object holding the current iteration's item.", m_objectName);
  addParam("path", m_path, "Path to execute for each iteration.", PathPtr(nullptr));
  addParam("maxIterations", m_maxIterations, "Maximum iterations in case of do_while", m_maxIterations);
  addParam("loopCondition", m_loopConditionString, "Loop condition in case of do_while", m_loopConditionString);
}

Member Function Documentation

beginRun()

void beginRun ( void  )

overridevirtual

Called when entering a new run.

Called at the beginning of each run, the method gives you the chance to change run dependent constants like alignment parameters, etc.

This method can be implemented by subclasses.

Reimplemented from Module.

Definition at line 210 of file SubEventModule.cc.

{
  m_processID = ProcHandler::EvtProcID();
 
  StoreObjPtr<ProcessStatistics> processStatistics("", DataStore::c_Persistent);
  processStatistics->suspendGlobal();
  processBeginRun();
 
  //don't screw up statistics for this module
  processStatistics->startModule();
  processStatistics->resumeGlobal();
}

callEvent()

void callEvent ( Module *  module )

protectedinherited

Calls event() on one single module, setting up logging and statistics as needed.

Parameters

module

Module to call the event() function

Definition at line 226 of file EventProcessor.cc.

{
  LogSystem& logSystem = LogSystem::Instance();
  const bool collectStats = !Environment::Instance().getNoStats();
  // set up logging
  logSystem.updateModule(&(module->getLogConfig()), module->getName());
  // set up statistics is requested
  if (collectStats) m_processStatisticsPtr->startModule();
  // call module
  CALL_MODULE(module, event);
  // stop timing
  if (collectStats) m_processStatisticsPtr->stopModule(module, ModuleStatistics::c_Event);
  // reset logging
  logSystem.updateModule(nullptr);
};

clone()

std::shared_ptr< PathElement > clone ( ) const

overridevirtualinherited

Create an independent copy of this module.

Note that parameters are shared, so changing them on a cloned module will also affect the original module.

Implements PathElement.

Definition at line 179 of file Module.cc.

{
  ModulePtr newModule = ModuleManager::Instance().registerModule(getType());
  newModule->m_moduleParamList.setParameters(getParamList());
  newModule->setName(getName());
  newModule->m_package = m_package;
  newModule->m_propertyFlags = m_propertyFlags;
  newModule->m_logConfig = m_logConfig;
  newModule->m_conditions = m_conditions;
 
  return newModule;
}

def_beginRun()

virtual void def_beginRun ( )

inlineprotectedvirtualinherited

Wrapper method for the virtual function beginRun() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 426 of file Module.h.

{ beginRun(); }

def_endRun()

virtual void def_endRun ( )

inlineprotectedvirtualinherited

This method can receive that the current run ends as a call from the Python side.

For regular C++-Modules that forwards the call to the regular endRun() method.

Reimplemented in PyModule.

Definition at line 439 of file Module.h.

{ endRun(); }

def_event()

virtual void def_event ( )

inlineprotectedvirtualinherited

Wrapper method for the virtual function event() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 432 of file Module.h.

{ event(); }

def_initialize()

virtual void def_initialize ( )

inlineprotectedvirtualinherited

Wrappers to make the methods without "def_" prefix callable from Python.

Overridden in PyModule. Wrapper method for the virtual function initialize() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 420 of file Module.h.

{ initialize(); }

def_terminate()

virtual void def_terminate ( )

inlineprotectedvirtualinherited

Wrapper method for the virtual function terminate() that has the implementation to be used in a call from Python.

Reimplemented in PyModule.

Definition at line 445 of file Module.h.

{ terminate(); }

endRun()

void endRun ( void  )

overridevirtual

This method is called if the current run ends.

Use this method to store information, which should be aggregated over one run.

This method can be implemented by subclasses.

Reimplemented from Module.

Definition at line 222 of file SubEventModule.cc.

{
  StoreObjPtr<ProcessStatistics> processStatistics("", DataStore::c_Persistent);
  processStatistics->suspendGlobal();
  processEndRun();
 
  //don't screw up statistics for this module
  processStatistics->startModule();
  processStatistics->resumeGlobal();
}

evalCondition()

bool evalCondition ( ) const

inherited

If at least one condition was set, it is evaluated and true returned if at least one condition returns true.

If no condition or result value was defined, the method returns false. Otherwise, the condition is evaluated and true returned, if at least one condition returns true. To speed up the evaluation, the condition strings were already parsed in the method if_value().

Returns

True if at least one condition and return value exists and at least one condition expression was evaluated to true.

Definition at line 96 of file Module.cc.

{
  if (m_conditions.empty()) return false;
 
  //okay, a condition was set for this Module...
  if (!m_hasReturnValue) {
    B2FATAL("A condition was set for '" << getName() << "', but the module did not set a return value!");
  }
 
  for (const auto& condition : m_conditions) {
    if (condition.evaluate(m_returnValue)) {
      return true;
    }
  }
  return false;
}

event()

void event ( void  )

overridevirtual

This method is the core of the module.

This method is called for each event. All processing of the event has to take place in this method.

This method can be implemented by subclasses.

Reimplemented from Module.

Definition at line 233 of file SubEventModule.cc.

{
  // Nothing to do? fine, don't do anything;
  if (m_mode == c_ForEach && m_loopOver.getEntries() == 0) return;
 
  StoreObjPtr<ProcessStatistics> processStatistics("", DataStore::c_Persistent);
  // We don't want call processBeginRun/EndRun() again (we do that in beginRun()/endRun() anyways) so
  // set the previous event meta to the current one to make sure this isn't triggered
  m_previousEventMetaData = *(StoreObjPtr<EventMetaData>());
  //and don't reinitialize the random numbers in this path which is done after
  //the master module ... so make sure there's none
  m_master = nullptr;
 
  if (m_mode == c_ForEach) {
    //get event map and make a deep copy of the StoreEntry objects
    //(we want to revert changes to the StoreEntry objects, but not to the arrays/objects)
    DataStore::StoreEntryMap& eventMap = DataStore::Instance().getStoreEntryMap(DataStore::c_Event);
    DataStore::StoreEntryMap eventMapCopy = eventMap;
    DataStore::StoreEntry& objectEntry = DataStore::Instance().getStoreEntryMap(DataStore::c_Event).at(*m_objectName);
 
 
    //remember the state of the object before we loop over it
    TObject* prevObject = objectEntry.object;
    TObject* prevPtr = objectEntry.ptr;
 
    const int numEntries = m_loopOver.getEntries();
    for (int i = 0; i < numEntries; i++) {
      //set loopObject
      objectEntry.object = m_loopOver[i];
      objectEntry.ptr = m_loopOver[i];
 
      //stuff usually done in processCore()
      PathIterator moduleIter(m_path);
      processEvent(moduleIter, false);
 
      //restore datastore
      restoreContents(eventMapCopy, eventMap);
    }
 
    // Let's reset the object to the way it was
    objectEntry.object = prevObject;
    objectEntry.ptr = prevPtr;
  } else {
    int returnValue{false};
    unsigned int curIteration{0};
    do {
      if (++curIteration > m_maxIterations) {
        B2FATAL(getName() << ": Maximum number of " << m_maxIterations << " iterations reached");
      }
      //stuff usually done in processCore()
      PathIterator moduleIter(m_path);
      processEvent(moduleIter, false);
      B2ASSERT("Module " << m_loopConditionModule->getName() << " did not set a return value, cannot loop",
               m_loopConditionModule->hasReturnValue());
      returnValue = m_loopConditionModule->getReturnValue();
    } while (m_loopCondition->evaluate(returnValue));
  }
 
  //don't screw up statistics for this module
  processStatistics->startModule();
}

exposePythonAPI()

void exposePythonAPI ( )

staticinherited

Exposes methods of the Module class to Python.

Definition at line 325 of file Module.cc.

{
  // to avoid confusion between std::arg and boost::python::arg we want a shorthand namespace as well
  namespace bp = boost::python;
 
  docstring_options options(true, true, false); //userdef, py sigs, c++ sigs
 
  void (Module::*setReturnValueInt)(int) = &Module::setReturnValue;
 
  enum_<Module::EAfterConditionPath>("AfterConditionPath",
                                     R"(Determines execution behaviour after a conditional path has been executed:
 
.. attribute:: END
 
  End processing of this path after the conditional path. (this is the default for if_value() etc.)
 
.. attribute:: CONTINUE
 
  After the conditional path, resume execution after this module.)")
  .value("END", Module::EAfterConditionPath::c_End)
  .value("CONTINUE", Module::EAfterConditionPath::c_Continue)
  ;
 
  /* Do not change the names of >, <, ... we use them to serialize conditional pathes */
  enum_<Belle2::ModuleCondition::EConditionOperators>("ConditionOperator")
  .value(">", Belle2::ModuleCondition::EConditionOperators::c_GT)
  .value("<", Belle2::ModuleCondition::EConditionOperators::c_ST)
  .value(">=", Belle2::ModuleCondition::EConditionOperators::c_GE)
  .value("<=", Belle2::ModuleCondition::EConditionOperators::c_SE)
  .value("==", Belle2::ModuleCondition::EConditionOperators::c_EQ)
  .value("!=", Belle2::ModuleCondition::EConditionOperators::c_NE)
  ;
 
  enum_<Module::EModulePropFlags>("ModulePropFlags",
                                  R"(Flags to indicate certain low-level features of modules, see :func:`Module.set_property_flags()`, :func:`Module.has_properties()`. Most useful flags are:
 
.. attribute:: PARALLELPROCESSINGCERTIFIED
 
    This module can be run in parallel processing mode safely (All I/O must be done through the data store, in particular, the module must not write any files.)
 
.. attribute:: HISTOGRAMMANAGER
 
  This module is used to manage histograms accumulated by other modules
 
.. attribute:: TERMINATEINALLPROCESSES
 
  When using parallel processing, call this module's terminate() function in all processes. This will also ensure that there is exactly one process (single-core if no parallel modules found) or at least one input, one main and one output process.
)")
  .value("INPUT", Module::EModulePropFlags::c_Input)
  .value("OUTPUT", Module::EModulePropFlags::c_Output)
  .value("PARALLELPROCESSINGCERTIFIED", Module::EModulePropFlags::c_ParallelProcessingCertified)
  .value("HISTOGRAMMANAGER", Module::EModulePropFlags::c_HistogramManager)
  .value("INTERNALSERIALIZER", Module::EModulePropFlags::c_InternalSerializer)
  .value("TERMINATEINALLPROCESSES", Module::EModulePropFlags::c_TerminateInAllProcesses)
  ;
 
  //Python class definition
  class_<Module, PyModule> module("Module", R"(
Base class for Modules.
 
A module is the smallest building block of the framework.
A typical event processing chain consists of a Path containing
modules. By inheriting from this base class, various types of
modules can be created. To use a module, please refer to
:func:`Path.add_module()`.  A list of modules is available by running
``basf2 -m`` or ``basf2 -m package``, detailed information on parameters is
given by e.g. ``basf2 -m RootInput``.
 
The 'Module Development' section in the manual provides detailed information
on how to create modules, setting parameters, or using return values/conditions:
https://xwiki.desy.de/xwiki/rest/p/f4fa4/#HModuleDevelopment
 
)");
  module
  .def("__str__", &Module::getPathString)
  .def("name", &Module::getName, return_value_policy<copy_const_reference>(),
       "Returns the name of the module. Can be changed via :func:`set_name() <Module.set_name()>`, use :func:`type() <Module.type()>` for identifying a particular module class.")
  .def("type", &Module::getType, return_value_policy<copy_const_reference>(),
       "Returns the type of the module (i.e. class name minus 'Module')")
  .def("set_name", &Module::setName, args("name"), R"(
Set custom name, e.g. to distinguish multiple modules of the same type.
 
>>> path.add_module('EventInfoSetter')
>>> ro = path.add_module('RootOutput', branchNames=['EventMetaData'])
>>> ro.set_name('RootOutput_metadata_only')
>>> print(path)
[EventInfoSetter -> RootOutput_metadata_only]
 
)")
  .def("description", &Module::getDescription, return_value_policy<copy_const_reference>(),
       "Returns the description of this module.")
  .def("package", &Module::getPackage, return_value_policy<copy_const_reference>(),
       "Returns the package this module belongs to.")
  .def("available_params", &_getParamInfoListPython,
       "Return list of all module parameters as `ModuleParamInfo` instances")
  .def("has_properties", &Module::hasProperties, (bp::arg("properties")),
       R"DOCSTRING(Allows to check if the module has the given properties out of `ModulePropFlags` set.
 
>>> if module.has_properties(ModulePropFlags.PARALLELPROCESSINGCERTIFIED):
>>>     ...
 
Parameters:
  properties (int): bitmask of `ModulePropFlags` to check for.
)DOCSTRING")
  .def("set_property_flags", &Module::setPropertyFlags, args("property_mask"),
       "Set module properties in the form of an OR combination of `ModulePropFlags`.");
  {
    // python signature is too crowded, make ourselves
    docstring_options subOptions(true, false, false); //userdef, py sigs, c++ sigs
    module
    .def("if_value", &Module::if_value,
         (bp::arg("expression"), bp::arg("condition_path"), bp::arg("after_condition_path")= Module::EAfterConditionPath::c_End),
         R"DOCSTRING(if_value(expression, condition_path, after_condition_path=AfterConditionPath.END)
 
Sets a conditional sub path which will be executed after this
module if the return value set in the module passes the given ``expression``.
 
Modules can define a return value (int or bool) using ``setReturnValue()``,
which can be used in the steering file to split the Path based on this value, for example
 
>>> module_with_condition.if_value("<1", another_path)
 
In case the return value of the ``module_with_condition`` for a given event is
less than 1, the execution will be diverted into ``another_path`` for this event.
 
You could for example set a special return value if an error occurs, and divert
the execution into a path containing :b2:mod:`RootOutput` if it is found;
saving only the data producing/produced by the error.
 
After a conditional path has executed, basf2 will by default stop processing
the path for this event. This behaviour can be changed by setting the
``after_condition_path`` argument.
 
Parameters:
  expression (str): Expression to determine if the conditional path should be executed.
      This should be one of the comparison operators ``<``, ``>``, ``<=``,
      ``>=``, ``==``, or ``!=`` followed by a numerical value for the return value
  condition_path (Path): path to execute in case the expression is fulfilled
  after_condition_path (AfterConditionPath): What to do once the ``condition_path`` has been executed.
)DOCSTRING")
    .def("if_false", &Module::if_false,
         (bp::arg("condition_path"), bp::arg("after_condition_path")= Module::EAfterConditionPath::c_End),
         R"DOC(if_false(condition_path, after_condition_path=AfterConditionPath.END)
 
Sets a conditional sub path which will be executed after this module if
the return value of the module evaluates to False. This is equivalent to
calling `if_value` with ``expression=\"<1\"``)DOC")
    .def("if_true", &Module::if_true,
         (bp::arg("condition_path"), bp::arg("after_condition_path")= Module::EAfterConditionPath::c_End),
         R"DOC(if_true(condition_path, after_condition_path=AfterConditionPath.END)
 
Sets a conditional sub path which will be executed after this module if
the return value of the module evaluates to True. It is equivalent to
calling `if_value` with ``expression=\">=1\"``)DOC");
  }
  module
  .def("has_condition", &Module::hasCondition,
       "Return true if a conditional path has been set for this module "
       "using `if_value`, `if_true` or `if_false`")
  .def("get_all_condition_paths", &_getAllConditionPathsPython,
       "Return a list of all conditional paths set for this module using "
       "`if_value`, `if_true` or `if_false`")
  .def("get_all_conditions", &_getAllConditionsPython,
       "Return a list of all conditional path expressions set for this module using "
       "`if_value`, `if_true` or `if_false`")
  .add_property("logging", make_function(&Module::getLogConfig, return_value_policy<reference_existing_object>()),
                &Module::setLogConfig)

getAfterConditionPath()

Module::EAfterConditionPath getAfterConditionPath ( ) const

inherited

What to do after the conditional path is finished.

(defaults to c_End if no condition is set)

Definition at line 133 of file Module.cc.

{
  if (m_conditions.empty()) return EAfterConditionPath::c_End;
 
  //okay, a condition was set for this Module...
  if (!m_hasReturnValue) {
    B2FATAL("A condition was set for '" << getName() << "', but the module did not set a return value!");
  }
 
  for (const auto& condition : m_conditions) {
    if (condition.evaluate(m_returnValue)) {
      return condition.getAfterConditionPath();
    }
  }
 
  return EAfterConditionPath::c_End;
}

getAllConditionPaths()

std::vector< std::shared_ptr< Path > > getAllConditionPaths ( ) const

inherited

Return all condition paths currently set (no matter if the condition is true or not).

Definition at line 150 of file Module.cc.

{
  std::vector<std::shared_ptr<Path>> allConditionPaths;
  for (const auto& condition : m_conditions) {
    allConditionPaths.push_back(condition.getPath());
  }
 
  return allConditionPaths;
}

getAllConditions()

const std::vector< ModuleCondition > & getAllConditions ( ) const

inlineinherited

Return all set conditions for this module.

Definition at line 324 of file Module.h.

    {
      return m_conditions;
    }

getCondition()

const ModuleCondition * getCondition ( ) const

inlineinherited

Return a pointer to the first condition (or nullptr, if none was set)

Definition at line 314 of file Module.h.

    {
      if (m_conditions.empty()) {
        return nullptr;
      } else {
        return &m_conditions.front();
      }
    }

getConditionPath()

std::shared_ptr< Path > getConditionPath ( ) const

inherited

Returns the path of the last true condition (if there is at least one, else reaturn a null pointer).

Definition at line 113 of file Module.cc.

{
  PathPtr p;
  if (m_conditions.empty()) return p;
 
  //okay, a condition was set for this Module...
  if (!m_hasReturnValue) {
    B2FATAL("A condition was set for '" << getName() << "', but the module did not set a return value!");
  }
 
  for (const auto& condition : m_conditions) {
    if (condition.evaluate(m_returnValue)) {
      return condition.getPath();
    }
  }
 
  // if none of the conditions were true, return a null pointer.
  return p;
}

getDescription()

const std::string & getDescription ( ) const

inlineinherited

Returns the description of the module.

Definition at line 202 of file Module.h.

{return m_description;}

getFileNames()

virtual std::vector< std::string > getFileNames ( bool  outputFiles )

inlinevirtualinherited

Return a list of output filenames for this modules.

This will be called when basf2 is run with "--dry-run" if the module has set either the c_Input or c_Output properties.

If the parameter outputFiles is false (for modules with c_Input) the list of input filenames should be returned (if any). If outputFiles is true (for modules with c_Output) the list of output files should be returned (if any).

If a module has sat both properties this member is called twice, once for each property.

The module should return the actual list of requested input or produced output filenames (including handling of input/output overrides) so that the grid system can handle input/output files correctly.

This function should return the same value when called multiple times. This is especially important when taking the input/output overrides from Environment as they get consumed when obtained so the finalized list of output files should be stored for subsequent calls.

Reimplemented in RootInputModule, StorageRootOutputModule, and RootOutputModule.

Definition at line 134 of file Module.h.

    {
      return std::vector<std::string>();
    }

getLogConfig()

LogConfig & getLogConfig ( )

inlineinherited

Returns the log system configuration.

Definition at line 225 of file Module.h.

{return m_logConfig;}

getMaximumEventNumber()

long getMaximumEventNumber ( long  maxEvent ) const

protectedinherited

Calculate the maximum event number out of the argument from command line and the environment.

Definition at line 113 of file EventProcessor.cc.

{
  //Check whether the number of events was set via command line argument
  unsigned int numEventsArgument = Environment::Instance().getNumberEventsOverride();
  if ((numEventsArgument > 0) && ((maxEvent == 0) || (maxEvent > numEventsArgument))) {
    return numEventsArgument;
  }
  return maxEvent;
}

getModules()

std::list< ModulePtr > getModules ( ) const

inlineoverrideprivatevirtualinherited

no submodules, return empty list

Implements PathElement.

Definition at line 506 of file Module.h.

{ return std::list<ModulePtr>(); }

getName()

const std::string & getName ( ) const

inlineinherited

Returns the name of the module.

This can be changed via e.g. set_name() in the steering file to give more useful names if there is more than one module of the same type.

For identifying the type of a module, using getType() (or type() in Python) is recommended.

Definition at line 187 of file Module.h.

{return m_name;}

getPackage()

const std::string & getPackage ( ) const

inlineinherited

Returns the package this module is in.

Definition at line 197 of file Module.h.

{return m_package;}

getParamInfoListPython()

std::shared_ptr< boost::python::list > getParamInfoListPython ( ) const

inherited

Returns a python list of all parameters.

Each item in the list consists of the name of the parameter, a string describing its type, a python list of all default values and the description of the parameter.

Returns

A python list containing the parameters of this parameter list.

Definition at line 279 of file Module.cc.

{
  return m_moduleParamList.getParamInfoListPython();
}

getParamList()

const ModuleParamList & getParamList ( ) const

inlineinherited

Return module param list.

Definition at line 363 of file Module.h.

{ return m_moduleParamList; }

getPathString()

std::string getPathString ( ) const

overrideprivatevirtualinherited

return the module name.

Implements PathElement.

Definition at line 192 of file Module.cc.

{
 
  std::string output = getName();
 
  for (const auto& condition : m_conditions) {
    output += condition.getString();
  }
 
  return output;
}

getReturnValue()

int getReturnValue ( ) const

inlineinherited

Return the return value set by this module.

This value is only meaningful if hasReturnValue() is true

Definition at line 381 of file Module.h.

{ return m_returnValue; }

getType()

const std::string & getType ( ) const

inherited

Returns the type of the module (i.e.

class name minus 'Module')

Definition at line 41 of file Module.cc.

{
  if (m_type.empty())
    B2FATAL("Module type not set for " << getName());
  return m_type;
}

hasCondition()

bool hasCondition ( ) const

inlineinherited

Returns true if at least one condition was set for the module.

Definition at line 311 of file Module.h.

{ return not m_conditions.empty(); };

hasProperties()

bool hasProperties ( unsigned int  propertyFlags ) const

inherited

Returns true if all specified property flags are available in this module.

Parameters

propertyFlags

Ored EModulePropFlags which should be compared with the module flags.

Definition at line 160 of file Module.cc.

{
  return (propertyFlags & m_propertyFlags) == propertyFlags;
}

hasReturnValue()

bool hasReturnValue ( ) const

inlineinherited

Return true if this module has a valid return value set.

Definition at line 378 of file Module.h.

{ return m_hasReturnValue; }

hasUnsetForcedParams()

bool hasUnsetForcedParams ( ) const

inherited

Returns true and prints error message if the module has unset parameters which the user has to set in the steering file.

Definition at line 166 of file Module.cc.

{
  auto missing = m_moduleParamList.getUnsetForcedParams();
  std::string allMissing = "";
  for (const auto& s : missing)
    allMissing += s + " ";
  if (!missing.empty())
    B2ERROR("The following required parameters of Module '" << getName() << "' were not specified: " << allMissing <<
            "\nPlease add them to your steering file.");
  return !missing.empty();
}

if_false()

void if_false ( const std::shared_ptr< Path > &  path, EAfterConditionPath  afterConditionPath = EAfterConditionPath::c_End  )

inherited

A simplified version to add a condition to the module.

Please note that successive calls of this function will add more than one condition to the module. If more than one condition results in true, only the last of them will be used.

Please be careful: Avoid creating cyclic paths, e.g. by linking a condition to a path which is processed before the path where this module is located in.

It is equivalent to the if_value() method, using the expression "<1". This method is meant to be used together with the setReturnValue(bool value) method.

Parameters

path	Shared pointer to the Path which will be executed if the return value is false.
afterConditionPath	What to do after executing 'path'.

Definition at line 85 of file Module.cc.

{
  if_value("<1", path, afterConditionPath);
}

if_true()

void if_true ( const std::shared_ptr< Path > &  path, EAfterConditionPath  afterConditionPath = EAfterConditionPath::c_End  )

inherited

A simplified version to set the condition of the module.

Please note that successive calls of this function will add more than one condition to the module. If more than one condition results in true, only the last of them will be used.

Please be careful: Avoid creating cyclic paths, e.g. by linking a condition to a path which is processed before the path where this module is located in.

It is equivalent to the if_value() method, using the expression ">=1". This method is meant to be used together with the setReturnValue(bool value) method.

Parameters

path	Shared pointer to the Path which will be executed if the return value is true.
afterConditionPath	What to do after executing 'path'.

Definition at line 90 of file Module.cc.

{
  if_value(">=1", path, afterConditionPath);
}

if_value()

void if_value ( const std::string &  expression, const std::shared_ptr< Path > &  path, EAfterConditionPath  afterConditionPath = EAfterConditionPath::c_End  )

inherited

Add a condition to the module.

Please note that successive calls of this function will add more than one condition to the module. If more than one condition results in true, only the last of them will be used.

See https://xwiki.desy.de/xwiki/rest/p/a94f2 or ModuleCondition for a description of the syntax.

Please be careful: Avoid creating cyclic paths, e.g. by linking a condition to a path which is processed before the path where this module is located in.

Parameters

expression	The expression of the condition.
path	Shared pointer to the Path which will be executed if the condition is evaluated to true.
afterConditionPath	What to do after executing 'path'.

Definition at line 79 of file Module.cc.

{
  m_conditions.emplace_back(expression, path, afterConditionPath);
}

initialize()

void initialize ( void  )

overridevirtual

Initialize the Module.

This method is called once before the actual event processing starts. Use this method to initialize variables, open files etc.

This method can be implemented by subclasses.

Reimplemented from Module.

Definition at line 145 of file SubEventModule.cc.

{
  if (!m_path) {
    B2FATAL("SubEvent module not initialised properly.");
  }
 
  if (m_mode == c_DoWhile) {
    setDoWhileConditions();
  } else if (m_mode == c_ForEach) {
    if (!(m_objectName && m_loopOverName)) {
      B2FATAL("loopOver and objectName parameters not setup");
    }
  } else {
    B2FATAL("Invalid SubEvent mode: " << m_mode);
  }
 
  StoreObjPtr<ProcessStatistics> processStatistics("", DataStore::c_Persistent);
  processStatistics->suspendGlobal();
 
  if (m_mode == c_ForEach) {
    m_loopOver.isRequired(*m_loopOverName);
    //register loop object (c_DontWriteOut to disable writing the object)
    const DataStore::StoreEntry& arrayEntry = DataStore::Instance().getStoreEntryMap(DataStore::c_Event).at(m_loopOver.getName());
    DataStore::Instance().registerEntry(*m_objectName, DataStore::c_Event, arrayEntry.objClass, false, DataStore::c_DontWriteOut);
  }
 
  m_moduleList = m_path->buildModulePathList();
  processInitialize(m_moduleList, false);
 
  //don't screw up statistics for this module
  processStatistics->startModule();
  processStatistics->resumeGlobal();
}

initSubEvent()

void initSubEvent	(	const std::string &	objectName,
		const std::string &	loopOver,
		std::shared_ptr< Path >	path
	)

used by Path::forEach() to actually set parameters.

Definition at line 50 of file SubEventModule.cc.

{
  m_objectName = objectName;
  m_loopOverName = loopOver;
  m_path = std::move(path);
  setName("for_each(" + objectName + " : " + loopOver + ")");
  setProperties();
}

initSubLoop()

void initSubLoop	(	std::shared_ptr< Path >	path,
		const std::string &	condition,
		unsigned int	maxIterations
	)

ised by Path::doWhile() to actually set parameters

Definition at line 59 of file SubEventModule.cc.

{
  m_mode = c_DoWhile;
  m_path = std::move(path);
  m_maxIterations = maxIterations;
  m_loopConditionString = condition.empty() ? "<1" : condition;
  setDoWhileConditions();
  setName("do_while(" + m_loopConditionModule->getName() + *m_loopConditionString + ")");
  setProperties();
}

installMainSignalHandlers()

void installMainSignalHandlers ( void(*)(int)  fn = nullptr )

staticinherited

Install signal handler for INT, TERM and QUIT signals.

If argument is NULL, EventProcessor's own signal handler will be installed.

Definition at line 312 of file EventProcessor.cc.

{
  if (!fn)
    fn = signalHandler;
  installSignalHandler(SIGINT, fn);
  installSignalHandler(SIGTERM, fn);
  installSignalHandler(SIGQUIT, fn);
}

installSignalHandler()

void installSignalHandler ( int  sig, void(*)(int)  fn  )

staticinherited

Install a signal handler 'fn' for given signal.

Definition at line 297 of file EventProcessor.cc.

{
  struct sigaction s;
  memset(&s, '\0', sizeof(s));
 
  s.sa_handler = fn;
  sigemptyset(&s.sa_mask);
  if (sig == SIGCHLD)
    s.sa_flags |= SA_NOCLDSTOP; //don't produce signal when children are stopped
 
  if (sigaction(sig, &s, nullptr) != 0) {
    B2FATAL("Cannot setup signal handler for signal " << sig);
  }
}

process()

void process ( const PathPtr &  startPath, long  maxEvent = 0  )

inherited

Processes the full module chain, starting with the first module in the given path.

Processes all events for the given run number and for events from 0 to maxEvent. If maxEvent is smaller or equal 0 the maximum number check is disabled and all events are processed. If runNumber is smaller than 0, the run number has to be set externally by a module and not the given number is used.

Parameters

startPath	The processing starts with the first module of this path.
maxEvent	Optional: The maximum number of events that will be processed. If the number is smaller or equal 0, all events will be processed.

Definition at line 123 of file EventProcessor.cc.

{
  maxEvent = getMaximumEventNumber(maxEvent);
  // Make sure the NumberEventsOverride reflects the actual number if
  // process(path, N) was used instead of -n and that it's reset to what it was
  // after we're done with processing()
  NumberEventsOverrideGuard numberOfEventsOverrideGuard(maxEvent);
 
  //Get list of modules which could be executed during the data processing.
  ModulePtrList moduleList = startPath->buildModulePathList();
 
  //Find the adress of the module we want to profile
  if (!m_profileModuleName.empty()) {
    for (const auto& module : moduleList) {
      if (module->getName() == m_profileModuleName) {
        m_profileModule = module.get();
        break;
      }
    }
    if (!m_profileModule)
      B2FATAL("Module profiling was requested via --profile, but no module '" << m_profileModuleName << "' was found!");
  }
 
  //Initialize modules
  processInitialize(moduleList);
 
  // SteerRootInputModule might have changed the number of events to be processed
  for (const auto& module : moduleList) {
    if (module->getType() == "SteerRootInput") {
      if (maxEvent != Environment::Instance().getNumberEventsOverride()) {
        B2INFO("Module 'SteerRootInputModule' is controlling the number of processed events.");
        maxEvent = Environment::Instance().getNumberEventsOverride();
        m_steerRootInputModuleOn = true;
      }
      break;
    }
  }
 
  //do we want to visualize DataStore input/ouput?
  if (Environment::Instance().getVisualizeDataFlow()) {
    DataFlowVisualization v(&DataStore::Instance().getDependencyMap());
    v.visualizePath("dataflow.dot", *startPath);
  }
 
  //Don't start processing in case of no master module
  if (!m_master) {
    B2ERROR("There is no module that provides event and run numbers (EventMetaData). You must add either the EventInfoSetter or an input module (e.g. RootInput) to the beginning of your path.");
  }
 
  //Check if errors appeared. If yes, don't start the event processing.
  int numLogError = LogSystem::Instance().getMessageCounter(LogConfig::c_Error);
  if ((numLogError == 0) && m_master) {
    installMainSignalHandlers();
    try {
      processCore(startPath, moduleList, maxEvent); //Do the event processing
    } catch (StoppedBySignalException& e) {
      if (e.signal != SIGINT) {
        // close all open ROOT files, ROOT's exit handler will crash otherwise
        gROOT->GetListOfFiles()->Delete();
 
        B2FATAL(e.what());
      }
      //in case of SIGINT, we move on to processTerminate() to shut down safely
    } catch (...) {
      if (m_eventMetaDataPtr)
        B2ERROR("Exception occured in exp/run/evt: "
                << m_eventMetaDataPtr->getExperiment() << " / "
                << m_eventMetaDataPtr->getRun() << " / "
                << m_eventMetaDataPtr->getEvent());
      throw;
    }
 
  } else {
    B2FATAL(numLogError << " ERROR(S) occurred! The processing of events will not be started.");
  }
 
  //Terminate modules
  processTerminate(moduleList);
 
  LogSystem::Instance().printErrorSummary();
 
  if (gSignalReceived == SIGINT) {
    const auto msg = R"(Processing aborted via SIGINT, terminating.
    Output files have been closed safely and should be readable. However
    processing was NOT COMPLETE. The output files do contain only events
    processed until this point.)";
    if (m_eventMetaDataPtr)
      B2ERROR(msg
              << LogVar("last_experiment", m_eventMetaDataPtr->getExperiment())
              << LogVar("last_run", m_eventMetaDataPtr->getRun())
              << LogVar("last_event", m_eventMetaDataPtr->getEvent()));
    else
      B2ERROR(msg);
    installSignalHandler(SIGINT, SIG_DFL);
    raise(SIGINT);
  }
}

processBeginRun()

void processBeginRun ( bool  skipDB = false )

protectedinherited

Calls the begin run methods of all modules.

Loops over all module instances specified in a list and calls their beginRun() method. Please note: the beginRun() method of the module which triggered the beginRun() loop will also be called.

Definition at line 467 of file EventProcessor.cc.

{
  MetadataService::Instance().addBasf2Status("beginning run");
 
  m_inRun = true;
  auto dbsession = Database::Instance().createScopedUpdateSession(); // cppcheck-suppress unreadVariable
 
  LogSystem& logSystem = LogSystem::Instance();
  m_processStatisticsPtr->startGlobal();
 
  if (!skipDB) DBStore::Instance().update();
 
  //initialize random generator for end run
  RandomNumbers::initializeBeginRun();
 
  for (const ModulePtr& modPtr : m_moduleList) {
    Module* module = modPtr.get();
 
    //Set the module dependent log level
    logSystem.updateModule(&(module->getLogConfig()), module->getName());
 
    //Do beginRun() call
    m_processStatisticsPtr->startModule();
    CALL_MODULE(module, beginRun);
    m_processStatisticsPtr->stopModule(module, ModuleStatistics::c_BeginRun);
 
    //Set the global log level
    logSystem.updateModule(nullptr);
  }
 
  m_processStatisticsPtr->stopGlobal(ModuleStatistics::c_BeginRun);
}

processCore()

void processCore ( const PathPtr &  startPath, const ModulePtrList &  modulePathList, long  maxEvent = 0, bool  isInputProcess = true  )

protectedinherited

Processes the full module chain consisting of an arbitrary number of connected paths, starting with the first module in the specified path.

Parameters

startPath	The processing starts with the first module of this path.
modulePathList	A list of all modules which could be executed during the data processing (used for calling the beginRun() and endRun() method).
maxEvent	The maximum number of events that will be processed. If the number is smaller or equal 0, all events are processed.
isInputProcess	true when this is either the only or the input process

Definition at line 406 of file EventProcessor.cc.

{
  DataStore::Instance().setInitializeActive(false);
  m_moduleList = modulePathList;
  //Remember the previous event meta data, and identify end of data meta data
  m_previousEventMetaData.setEndOfData(); //invalid start state
 
  const bool collectStats = !Environment::Instance().getNoStats();
 
  //Loop over the events
  long currEvent = 0;
  bool endProcess = false;
  while (!endProcess) {
    if (collectStats)
      m_processStatisticsPtr->startGlobal();
 
    PathIterator moduleIter(startPath);
    endProcess = processEvent(moduleIter, isInputProcess && currEvent == 0);
 
    //Delete event related data in DataStore
    DataStore::Instance().invalidateData(DataStore::c_Event);
 
    currEvent++;
    if ((maxEvent > 0) && (currEvent >= maxEvent)) endProcess = true;
    if (collectStats)
      m_processStatisticsPtr->stopGlobal(ModuleStatistics::c_Event);
  } //end event loop
 
  //End last run
  m_eventMetaDataPtr.create();
  processEndRun();
}

processEndRun()

void processEndRun ( )

protectedinherited

Calls the end run methods of all modules.

Loops over all module instances specified in a list and calls their endRun() method. Please note: the endRun() method of the module which triggered the endRun() loop will also be called.

Definition at line 501 of file EventProcessor.cc.

{
  MetadataService::Instance().addBasf2Status("ending run");
 
  if (!m_inRun)
    return;
  m_inRun = false;
 
  LogSystem& logSystem = LogSystem::Instance();
  m_processStatisticsPtr->startGlobal();
 
  const EventMetaData newEventMetaData = *m_eventMetaDataPtr;
  *m_eventMetaDataPtr = m_previousEventMetaData;
 
  //initialize random generator for end run
  RandomNumbers::initializeEndRun();
 
  for (const ModulePtr& modPtr : m_moduleList) {
    Module* module = modPtr.get();
 
    //Set the module dependent log level
    logSystem.updateModule(&(module->getLogConfig()), module->getName());
 
    //Do endRun() call
    m_processStatisticsPtr->startModule();
    CALL_MODULE(module, endRun);
    m_processStatisticsPtr->stopModule(module, ModuleStatistics::c_EndRun);
 
    //Set the global log level
    logSystem.updateModule(nullptr);
  }
  *m_eventMetaDataPtr = newEventMetaData;
 
  m_processStatisticsPtr->stopGlobal(ModuleStatistics::c_EndRun);
}

processEvent()

bool processEvent ( PathIterator  moduleIter, bool  skipMasterModule  )

protectedinherited

Calls event() functions on all modules for the current event.

Used by processCore.

Parameters

moduleIter	iterator of the path containing all the modules
skipMasterModule	skip the execution of the master module, presumably because this is the first event and it's already been done in initialize()

Returns

true if execution should stop.

Definition at line 321 of file EventProcessor.cc.

{
  double time = Utils::getClock() / Unit::s;
  if (time > m_lastMetadataUpdate + m_metadataUpdateInterval) {
    MetadataService::Instance().addBasf2Status("running event loop");
    m_lastMetadataUpdate = time;
  }
 
  const bool collectStats = !Environment::Instance().getNoStats();
 
  while (!moduleIter.isDone()) {
    Module* module = moduleIter.get();
 
    // run the module ... unless we don't want to
    if (!(skipMasterModule && module == m_master)) {
      callEvent(module);
    }
 
    //Check for end of data
    if ((m_eventMetaDataPtr && (m_eventMetaDataPtr->isEndOfData())) ||
        ((module == m_master) && !m_eventMetaDataPtr)) {
      if ((module != m_master) && !m_steerRootInputModuleOn) {
        B2WARNING("Event processing stopped by module '" << module->getName() <<
                  "', which is not in control of event processing (does not provide EventMetaData)");
      }
      return true;
    }
 
    //Handle EventMetaData changes by master module
    if (module == m_master) {
 
      //initialize random number state for the event
      RandomNumbers::initializeEvent();
 
      //Check for a change of the run
      if ((m_eventMetaDataPtr->getExperiment() != m_previousEventMetaData.getExperiment()) ||
          (m_eventMetaDataPtr->getRun() != m_previousEventMetaData.getRun())) {
 
        if (collectStats)
          m_processStatisticsPtr->suspendGlobal();
 
        processEndRun();
        processBeginRun(skipMasterModule);
 
        if (collectStats)
          m_processStatisticsPtr->resumeGlobal();
      }
 
      m_previousEventMetaData = *m_eventMetaDataPtr;
 
      //make sure we use the event dependent generator again
      RandomNumbers::useEventDependent();
 
      DBStore::Instance().updateEvent();
 
    } else {
      //Check for a second master module. Cannot do this if we skipped the
      //master module as the EventMetaData is probably set before we call this
      //function
      if (!skipMasterModule && m_eventMetaDataPtr &&
          (*m_eventMetaDataPtr != m_previousEventMetaData)) {
        B2FATAL("Two modules setting EventMetaData were discovered: " << m_master->getName() << " and " << module->getName());
      }
    }
 
    if (gSignalReceived != 0) {
      throw StoppedBySignalException(gSignalReceived);
    }
 
    //Check for the module conditions, evaluate them and if one is true switch to the new path
    if (module->evalCondition()) {
      PathPtr condPath = module->getConditionPath();
      //continue with parent Path after condition path is executed?
      if (module->getAfterConditionPath() == Module::EAfterConditionPath::c_Continue) {
        moduleIter = PathIterator(condPath, moduleIter);
      } else {
        moduleIter = PathIterator(condPath);
      }
    } else {
      moduleIter.next();
    }
  } //end module loop
  return false;
}

processInitialize()

void processInitialize ( const ModulePtrList &  modulePathList, bool  setEventInfo = true  )

protectedinherited

Initializes the modules.

Loops over all module instances specified in a list and calls their initialize() method.

Parameters

modulePathList	A list of all modules which could be executed during the data processing.
setEventInfo	if true the first event call of the master module will be called immidiately to load the event info right away so that it's available for subsequent modules

Definition at line 242 of file EventProcessor.cc.

{
  LogSystem& logSystem = LogSystem::Instance();
  auto dbsession = Database::Instance().createScopedUpdateSession();
 
  m_processStatisticsPtr.registerInDataStore();
  //TODO I might want to overwrite it in initialize (e.g. if read from file)
  //     For parallel processing or subevents, I don't want that, though.
  //     Maybe make this a function argument?
  if (!m_processStatisticsPtr)
    m_processStatisticsPtr.create();
  m_processStatisticsPtr->startGlobal();
 
  MetadataService::Instance().addBasf2Status("initializing");
 
  for (const ModulePtr& modPtr : modulePathList) {
    Module* module = modPtr.get();
 
    if (module->hasUnsetForcedParams()) {
      //error message was printed by module
      continue;
    }
 
    //Set the module dependent log level
    logSystem.updateModule(&(module->getLogConfig()), module->getName());
    DataStore::Instance().getDependencyMap().setModule(*module);
 
    //Do initialization
    m_processStatisticsPtr->initModule(module);
    m_processStatisticsPtr->startModule();
    CALL_MODULE(module, initialize);
    m_processStatisticsPtr->stopModule(module, ModuleStatistics::c_Init);
 
    //Set the global log level
    logSystem.updateModule(nullptr);
 
    //Check whether this is the master module
    if (!m_master && DataStore::Instance().getEntry(m_eventMetaDataPtr) != nullptr) {
      B2DEBUG(100, "Found module providing EventMetaData: " << module->getName());
      m_master = module;
      if (setEventInfo) {
        callEvent(module);
        // update Database payloads: we now have valid event meta data unless
        // we don't process any events
        if (m_eventMetaDataPtr) DBStore::Instance().update();
      }
    }
 
    if (gSignalReceived != 0) {
      throw StoppedBySignalException(gSignalReceived);
    }
  }
  m_processStatisticsPtr->stopGlobal(ModuleStatistics::c_Init);
}

processTerminate()

void processTerminate ( const ModulePtrList &  modulePathList )

protectedinherited

Terminates the modules.

Loops over all module instances in reverse order specified in a list and calls their terminate() method.

Parameters

modulePathList

A list of all modules which could be executed during the data processing.

Definition at line 440 of file EventProcessor.cc.

{
  MetadataService::Instance().addBasf2Status("terminating");
 
  LogSystem& logSystem = LogSystem::Instance();
  ModulePtrList::const_reverse_iterator listIter;
  m_processStatisticsPtr->startGlobal();
 
  for (listIter = modulePathList.rbegin(); listIter != modulePathList.rend(); ++listIter) {
    Module* module = listIter->get();
 
    //Set the module dependent log level
    logSystem.updateModule(&(module->getLogConfig()), module->getName());
 
    //Do termination
    m_processStatisticsPtr->startModule();
    CALL_MODULE(module, terminate);
    m_processStatisticsPtr->stopModule(module, ModuleStatistics::c_Term);
 
    //Set the global log level
    logSystem.updateModule(nullptr);
  }
 
  m_processStatisticsPtr->stopGlobal(ModuleStatistics::c_Term);
}

setAbortLevel()

void setAbortLevel ( int  abortLevel )

inherited

Configure the abort log level.

Definition at line 67 of file Module.cc.

{
  m_logConfig.setAbortLevel(static_cast<LogConfig::ELogLevel>(abortLevel));
}

setDebugLevel()

void setDebugLevel ( int  debugLevel )

inherited

Configure the debug messaging level.

Definition at line 61 of file Module.cc.

{
  m_logConfig.setDebugLevel(debugLevel);
}

setDescription()

void setDescription ( const std::string &  description )

protectedinherited

Sets the description of the module.

Parameters

description

A description of the module.

Definition at line 214 of file Module.cc.

{
  m_description = description;
}

setDoWhileConditions()

void setDoWhileConditions ( )

private

Set the necessary pointers for do_while(): the pointer to the module whose return value we'll use as well as the ModuleCondition object.

Does nothing if both are setup already

Definition at line 70 of file SubEventModule.cc.

{
  if (!m_loopConditionString) {
    B2FATAL("No Loop conditions specified");
  }
  if (!m_loopCondition) {
    m_loopCondition = std::make_unique<ModuleCondition>(*m_loopConditionString, m_path, EAfterConditionPath::c_End);
  }
  // We also need the last module to get its return value. If it's already set then fine.
  if (m_loopConditionModule) return;
 
  // Otherwise loop over everything
  PathIterator iter(m_path);
  while (!iter.isDone()) {
    m_loopConditionModule = iter.get();
    // Sooo, we have one problem: If the module has a condition with c_End as
    // after condition path the execution does not resume on our loop path and
    // the final module would not be executed and thus the return value is not
    // set correctly. Now we could check for exactly this case and just refuse
    // to have c_End conditions in our loop path but even if the condition is
    // c_Continue, as soon as the conditional path contains another c_End
    // condition processing will stop there. Which is debatable but currently
    // the implementation. This means the only safe thing to do is to not allow any
    // conditions in our loop for now
    if (m_loopConditionModule->hasCondition()) {
      B2FATAL("Modules in a Path.do_while() cannot have any conditions");
      //If we fix the path behavior that c_Continue will always come back
      //correctly we could be less restrictive here and only disallow c_End
      //conditions as those would really lead to undefined loop conditions
      for (const auto& condition : m_loopConditionModule->getAllConditions()) {
        if (condition.getAfterConditionPath() == ModuleCondition::EAfterConditionPath::c_End) {
          B2FATAL("do_while(): Modules in a loop cannot have conditions which end processing");
        }
      }
    }
    iter.next();
  }
  //apparently no modules at all?
  if (!m_loopConditionModule) {
    B2FATAL("do_while(): Cannot loop over empty path");
  }
 
  // If the last module has a condition attached the looping cannot be done
  // reliably as event processing might take that conditional path.
  // Obviously this does nothing right now until we fix the nested
  // c_Continue->c_End inconsitency and allow some conditions
  if (m_loopConditionModule->hasCondition()) {
    B2FATAL("do_while(): The last module in the loop path (" <<
            m_loopConditionModule->getName() << ") cannot have a condition");
  }
}

setLogConfig()

void setLogConfig ( const LogConfig &  logConfig )

inlineinherited

Set the log system configuration.

Definition at line 230 of file Module.h.

{m_logConfig = logConfig;}

setLogInfo()

void setLogInfo ( int  logLevel, unsigned int  logInfo  )

inherited

Configure the printed log information for the given level.

Parameters

logLevel	The log level (one of LogConfig::ELogLevel)
logInfo	What kind of info should be printed? ORed combination of LogConfig::ELogInfo flags.

Definition at line 73 of file Module.cc.

{
  m_logConfig.setLogInfo(static_cast<LogConfig::ELogLevel>(logLevel), logInfo);
}

setLogLevel()

void setLogLevel ( int  logLevel )

inherited

Configure the log level.

Definition at line 55 of file Module.cc.

{
  m_logConfig.setLogLevel(static_cast<LogConfig::ELogLevel>(logLevel));
}

setName()

void setName ( const std::string &  name )

inlineinherited

Set the name of the module.

Note

The module name is set when using the REG_MODULE macro, but the module can be renamed before calling process() using the set_name() function in your steering file.

Parameters

name	The name of the module

Definition at line 214 of file Module.h.

{ m_name = name; };

setParamList()

void setParamList ( const ModuleParamList &  params )

inlineprotectedinherited

Replace existing parameter list.

Definition at line 501 of file Module.h.

{ m_moduleParamList = params; }

setParamPython()

void setParamPython ( const std::string &  name, const boost::python::object &  pyObj  )

privateinherited

Implements a method for setting boost::python objects.

The method supports the following types: list, dict, int, double, string, bool The conversion of the python object to the C++ type and the final storage of the parameter value is done in the ModuleParam class.

Parameters

name	The unique name of the parameter.
pyObj	The object which should be converted and stored as the parameter value.

Definition at line 234 of file Module.cc.

{
  LogSystem& logSystem = LogSystem::Instance();
  logSystem.updateModule(&(getLogConfig()), getName());
  try {
    m_moduleParamList.setParamPython(name, pyObj);
  } catch (std::runtime_error& e) {
    throw std::runtime_error("Cannot set parameter '" + name + "' for module '"
                             + m_name + "': " + e.what());
  }
 
  logSystem.updateModule(nullptr);
}

setParamPythonDict()

void setParamPythonDict ( const boost::python::dict &  dictionary )

privateinherited

Implements a method for reading the parameter values from a boost::python dictionary.

The key of the dictionary has to be the name of the parameter and the value has to be of one of the supported parameter types.

Parameters

dictionary

The python dictionary from which the parameter values are read.

Definition at line 249 of file Module.cc.

{
 
  LogSystem& logSystem = LogSystem::Instance();
  logSystem.updateModule(&(getLogConfig()), getName());
 
  boost::python::list dictKeys = dictionary.keys();
  int nKey = boost::python::len(dictKeys);
 
  //Loop over all keys in the dictionary
  for (int iKey = 0; iKey < nKey; ++iKey) {
    boost::python::object currKey = dictKeys[iKey];
    boost::python::extract<std::string> keyProxy(currKey);
 
    if (keyProxy.check()) {
      const boost::python::object& currValue = dictionary[currKey];
      setParamPython(keyProxy, currValue);
    } else {
      B2ERROR("Setting the module parameters from a python dictionary: invalid key in dictionary!");
    }
  }
 
  logSystem.updateModule(nullptr);
}

setProfileModuleName()

void setProfileModuleName ( const std::string &  name )

inlineinherited

Set the name of the module we want to profile.

Parameters

name	Name of the module as returned by getName()

Definition at line 57 of file EventProcessor.h.

{ m_profileModuleName = name; }

setProperties()

void setProperties ( )

private

Set properties for this module based on the modules found in m_path.

Definition at line 122 of file SubEventModule.cc.

{
  m_moduleList = m_path->buildModulePathList();
  //set c_ParallelProcessingCertified flag if _all_ modules have it set
  auto flag = Module::c_ParallelProcessingCertified;
  if (ModuleManager::allModulesHaveFlag(m_moduleList, flag))
    setPropertyFlags(c_TerminateInAllProcesses | flag);
  else
    setPropertyFlags(c_TerminateInAllProcesses);
}

setPropertyFlags()

void setPropertyFlags ( unsigned int  propertyFlags )

inherited

Sets the flags for the module properties.

Parameters

propertyFlags

bitwise OR of EModulePropFlags

Definition at line 208 of file Module.cc.

{
  m_propertyFlags = propertyFlags;
}

setReturnValue() [1/2]

void setReturnValue ( bool  value )

protectedinherited

Sets the return value for this module as bool.

The bool value is saved as an integer with the convention 1 meaning true and 0 meaning false. The value can be used in the steering file to divide the analysis chain into several paths.

Parameters

value

The value of the return value.

Definition at line 227 of file Module.cc.

{
  m_hasReturnValue = true;
  m_returnValue = value;
}

setReturnValue() [2/2]

void setReturnValue ( int  value )

protectedinherited

Sets the return value for this module as integer.

The value can be used in the steering file to divide the analysis chain into several paths.

Parameters

value

The value of the return value.

Definition at line 220 of file Module.cc.

{
  m_hasReturnValue = true;
  m_returnValue = value;
}

setType()

void setType ( const std::string &  type )

protectedinherited

Set the module type.

Only for use by internal modules (which don't use the normal REG_MODULE mechanism).

Definition at line 48 of file Module.cc.

{
  if (!m_type.empty())
    B2FATAL("Trying to change module type from " << m_type << " is not allowed, the value is assumed to be fixed.");
  m_type = type;
}

terminate()

void terminate ( void  )

overridevirtual

This method is called at the end of the event processing.

This method is called only once after the event processing finished. Use this method for cleaning up, closing files, etc.

This method can be implemented by subclasses.

Reimplemented from Module.

Definition at line 179 of file SubEventModule.cc.

{
  StoreObjPtr<ProcessStatistics> processStatistics("", DataStore::c_Persistent);
  processStatistics->suspendGlobal();
 
  //get event map and make a deep copy of the StoreEntry objects
  //(we want to revert changes to the StoreEntry objects, but not to the arrays/objects)
  DataStore::StoreEntryMap& persistentMap = DataStore::Instance().getStoreEntryMap(DataStore::c_Persistent);
  DataStore::StoreEntryMap persistentMapCopy = persistentMap;
 
  if (!ProcHandler::parallelProcessingUsed() or m_processID == ProcHandler::EvtProcID()) {
    processTerminate(m_moduleList);
  } else {
    //we're in another process than we actually belong to, only call terminate where approriate
    ModulePtrList tmpModuleList;
    for (const ModulePtr& m : m_moduleList) {
      if (m->hasProperties(c_TerminateInAllProcesses))
        tmpModuleList.push_back(m);
    }
    processTerminate(tmpModuleList);
  }
 
  if (m_mode == c_ForEach) {
    restoreContents(persistentMapCopy, persistentMap);
  }
 
  //don't screw up statistics for this module
  processStatistics->startModule();
  processStatistics->resumeGlobal();
}

writeToStdErr()

void writeToStdErr ( const char  msg[] )

staticinherited

async-safe method to write something to STDERR.

Definition at line 72 of file EventProcessor.cc.

{
  //signal handlers are called asynchronously, making many standard functions (including output) dangerous
  //write() is, however, safe, so we'll use that to write to stderr.
 
  //strlen() not explicitly in safe list, but doesn't have any error handling routines that might alter global state
  const int len = strlen(msg);
 
  int rc = write(STDERR_FILENO, msg, len);
  (void) rc; //ignore return value (there's nothing we can do about a failed write)
 
}

Member Data Documentation

m_conditions

std::vector<ModuleCondition> m_conditions

privateinherited

Module condition, only non-null if set.

Definition at line 521 of file Module.h.

m_description

std::string m_description

privateinherited

The description of the module.

Definition at line 511 of file Module.h.

m_eventMetaDataPtr

StoreObjPtr<EventMetaData> m_eventMetaDataPtr

protectedinherited

EventMetaData is used by processEvent()/processCore().

Definition at line 163 of file EventProcessor.h.

m_hasReturnValue

bool m_hasReturnValue

privateinherited

True, if the return value is set.

Definition at line 518 of file Module.h.

m_inRun

bool m_inRun

protectedinherited

Are we currently in a run? If yes, processEndRun() needs to do something.

Definition at line 172 of file EventProcessor.h.

m_lastMetadataUpdate

double m_lastMetadataUpdate

protectedinherited

Time in seconds of last call for metadata update in event loop.

Definition at line 175 of file EventProcessor.h.

m_logConfig

LogConfig m_logConfig

privateinherited

The log system configuration of the module.

Definition at line 514 of file Module.h.

m_loopCondition

std::unique_ptr<ModuleCondition> m_loopCondition

private

Condition object to evaluate if the loop is finished in case of doWhile()

Definition at line 71 of file SubEventModule.h.

m_loopConditionModule

Module* m_loopConditionModule {nullptr}

private

pointer to the module to provide the returnValue in case of doWhile()

Definition at line 73 of file SubEventModule.h.

m_loopConditionString

std::optional<std::string> m_loopConditionString {std::nullopt}

private

String for the condition when looping.

Definition at line 69 of file SubEventModule.h.

m_loopOver

StoreArray<TObject> m_loopOver

private

array looped over in case of forEach

Definition at line 60 of file SubEventModule.h.

m_loopOverName

std::optional<std::string> m_loopOverName {std::nullopt}

private

name for m_loopOver in case of forEach.

Definition at line 58 of file SubEventModule.h.

m_master

const Module* m_master

protectedinherited

The master module that determines the experiment/run/event number.

Definition at line 153 of file EventProcessor.h.

m_maxIterations

unsigned int m_maxIterations {10000}

private

maximum number of iterations before giving up in case of doWhile()

Definition at line 67 of file SubEventModule.h.

m_metadataUpdateInterval

double m_metadataUpdateInterval

protectedinherited

Minimal time difference in seconds for metadata updates in event loop.

Definition at line 178 of file EventProcessor.h.

m_mode

int m_mode {c_ForEach}

private

Mode for this module.

Should be 0=for_each, 1=do_while

Definition at line 75 of file SubEventModule.h.

m_moduleList

ModulePtrList m_moduleList

protectedinherited

List of all modules in order initialized.

Definition at line 154 of file EventProcessor.h.

m_moduleParamList

ModuleParamList m_moduleParamList

privateinherited

List storing and managing all parameter of the module.

Definition at line 516 of file Module.h.

m_name

std::string m_name

privateinherited

The name of the module, saved as a string (user-modifiable)

Definition at line 508 of file Module.h.

m_objectName

std::optional<std::string> m_objectName {std::nullopt}

private

name of our loop variable in case of forEach.

Definition at line 56 of file SubEventModule.h.

m_package

std::string m_package

privateinherited

Package this module is found in (may be empty).

Definition at line 510 of file Module.h.

m_path

std::shared_ptr<Path> m_path

private

Path to execute.

Definition at line 62 of file SubEventModule.h.

m_previousEventMetaData

EventMetaData m_previousEventMetaData

protectedinherited

Stores state of EventMetaData before it was last changed.

Useful since processEndRun() needs info about which run it needs to end.

Definition at line 166 of file EventProcessor.h.

m_processID

int m_processID { -1}

private

when using multi-processing contains the ID of the process where event() is called (in that process only).

-1 otherwise.

Definition at line 65 of file SubEventModule.h.

m_processStatisticsPtr

StoreObjPtr<ProcessStatistics> m_processStatisticsPtr

protectedinherited

Also used in a number of places.

Definition at line 169 of file EventProcessor.h.

m_profileModule

Module* m_profileModule = nullptr

protectedinherited

Adress of the module which we want to profile, nullptr if no profiling is requested.

Definition at line 160 of file EventProcessor.h.

m_profileModuleName

std::string m_profileModuleName

protectedinherited

Name of the module which should be profiled, empty if no profiling is requested.

Definition at line 157 of file EventProcessor.h.

m_propertyFlags

unsigned int m_propertyFlags

privateinherited

The properties of the module as bitwise or (with |) of EModulePropFlags.

Definition at line 512 of file Module.h.

m_returnValue

int m_returnValue

privateinherited

The return value.

Definition at line 519 of file Module.h.

m_steerRootInputModuleOn

bool m_steerRootInputModuleOn = false

protectedinherited

True if the SteerRootInputModule is in charge for event processing.

Definition at line 181 of file EventProcessor.h.

m_type

std::string m_type

privateinherited

The type of the module, saved as a string.

Definition at line 509 of file Module.h.

---

The documentation for this class was generated from the following files:

• framework/core/include/SubEventModule.h
• framework/core/src/SubEventModule.cc