ProcessStatistics Class Reference

Class to collect call statistics for all modules. More...

#include <ProcessStatistics.h>

Inheritance diagram for ProcessStatistics:

Collaboration diagram for ProcessStatistics:

Public Member Functions
	ProcessStatistics ()
	Constructor.
std::string	getStatisticsString (ModuleStatistics::EStatisticCounters type=ModuleStatistics::c_Event, const std::vector< Belle2::ModuleStatistics > *modules=nullptr, bool html=false) const
	Return string with statistics for all modules. More...
const ModuleStatistics &	getGlobal () const
	Get global statistics.
const std::vector< Belle2::ModuleStatistics > &	getAll () const
	Get entire statistics map.
void	startGlobal ()
	Start timer for global measurement.
void	suspendGlobal ()
	Suspend timer for global measurement, needed for newRun. More...
void	resumeGlobal ()
	Resume timer after call to suspendGlobal()
void	stopGlobal (ModuleStatistics::EStatisticCounters type)
	Stop global timer and add values to the statistic counter.
void	startModule ()
	Start module timer.
void	stopModule (const Module *module, ModuleStatistics::EStatisticCounters type)
	Stop module counter and attribute values to appropriate module.
void	initModule (const Module *module)
	Init module statistics: Set name from module if still empty and remember initialization index for display.
ModuleStatistics &	getStatistics (const Module *module)
	Get statistics for single module. More...
int	getIndex (const Module *module)
	get m_stats index for given module, inserting it if not found.
virtual void	merge (const Mergeable *other) override
	Merge other ProcessStatistics object into this one.
virtual void	clear () override
	Clear collected statistics but keep names of modules.
virtual TObject *	Clone (const char *newname="") const override
	Reimplement TObject::Clone() since we also need m_modulesToStatsIndex.
std::string	getInfoHTML () const
	Return a short summary of this object's contents in HTML format.
virtual void	removeSideEffects ()
	An ugly little method that is called before event() for input and worker processes. More...
virtual Long64_t	Merge (TCollection *hlist)
	Allow merging using TFileMerger if saved directly to a file. More...
virtual void	Reset ()
	Root-like Reset function for "template compatibility" with ROOT objects. More...
virtual void	SetDirectory (TDirectory *)
	Root-like SetDirectory function for "template compatibility" with ROOT objects. More...

Private Member Functions
	ProcessStatistics (const ProcessStatistics &)=default
	Hide copy constructor.
ProcessStatistics &	operator= (ProcessStatistics &)
	Prohibit assignment operator.
void	appendUnmergedModules (const ProcessStatistics *otherObject)
	Merge dissimilar objects (mainly loading ProcessStatistics from file).
void	setTransientCounters (const ProcessStatistics *otherObject)
	Set transient counters from otherObject. More...
void	setCounters (double &time, double &memory, double startTime=0, double startMemory=0)
	Set counters time and memory to contain the current clock value and memory consumption respectively. More...
	ClassDefOverride (ProcessStatistics, 2)
	(transient) More...
	ClassDef (Mergeable, 0)
	Abstract base class for objects that can be merged.

Private Attributes
ModuleStatistics	m_global
	Statistics object for global time and memory consumption.
std::vector< Belle2::ModuleStatistics >	m_stats
	module statistics
std::map< const Module *, int >	m_modulesToStatsIndex
	transient, maps Module* to m_stats index.
double	m_globalTime
	store clock counter for global time consumption
double	m_globalMemory
	(transient) More...
double	m_moduleTime
	(transient) More...
double	m_moduleMemory
	(transient) More...
double	m_suspendedTime
	(transient) More...
double	m_suspendedMemory
	(transient) More...

Detailed Description

Class to collect call statistics for all modules.

This class is used to collect call and time statistics for all modules. It is implemented as a singleton and will keep track of the number of times a module will be called and the time the module spends in these calls.

Altough this class can be used in C++, its main purpose is to be used in python. In the python environment it is reachable through the "statistics" object in the pybasf2 module. Most simple use is to just print the event statistics after the process loop:

process(...) print(statistics)

Different types of statistics can be printed using

print(statistics(type))

where type can be one of

• statistics.INIT -> time/calls spent in initialize()
• statistics.BEGIN_RUN -> time/calls spent in beginRun()
• statistics.EVENT -> time/calls spent in event()
• statistics.END_RUN -> time/calls spent in endRun()
• statistics.TERM -> time/calls spent in terminate()
• statistics.TOTAL -> sum of all the above

It is also possible to restrict the event statistics to a list of modules one is interested in

foo = register_module("Foo") bar = register_module("Bar")

...

process(...) print(statistics([foo,bar])) print(statistics([foo,bar],statistics.BEGIN_RUN))

More detailed statistics can be reached by accessing the statistics for all modules directly:

process(...) for stats in statistics.modules: print(stats.name, stats.time(statistics.EVENT), stats.calls(statistics.BEGIN_RUN))

Available attributes/methods for the statistics objects are

• name: name of the module
• time(type=statistics.EVENT): time in seconds spent in function
• calls(type=statistics.EVENT): number of calls to function

The global statistics for the framework can be accessed via statistics.framework

The name shown in the statistics can be modified. This is particular useful if there is more than one instance of a given module in the path

foo = register_module("Foo") statistics.set_name(foo,"Footastic")

Definition at line 84 of file ProcessStatistics.h.

Member Function Documentation

ClassDefOverride()

ClassDefOverride ( ProcessStatistics  , 2    )

private

(transient)

Class to collect call statistics for all modules.

getStatistics()

ModuleStatistics& getStatistics ( const Module *  module )

inline

Get statistics for single module.

Parameters

module

Shared pointer to the Module for which the statistics should be obtained

Definition at line 163 of file ProcessStatistics.h.

    {
      return m_stats[getIndex(module)];
    }

getStatisticsString()

string getStatisticsString	(	ModuleStatistics::EStatisticCounters	type = ModuleStatistics::c_Event,
		const std::vector< Belle2::ModuleStatistics > *	modules = nullptr,
		bool	html = false
	)		const

Return string with statistics for all modules.

Can be used in steering file with 'print(statistics)'.

Parameters

type	counter type to use for statistics
modules	map of modules to use. If NULL, default map will be used
html	if true return the output as html table instead of an ascii table

Definition at line 53 of file ProcessStatistics.cc.

{
  const ModuleStatistics& global = getGlobal();
  if (!modules) modules = &(getAll());
  int moduleNameLength = 21; //minimum: 80 characters
  const int lengthOfRest = 80 - moduleNameLength;
  for (const ModuleStatistics& stats : *modules) {
    int len = stats.getName().length();
    if (len > moduleNameLength)
      moduleNameLength = len;
  }
  const std::string numTabsModule = (boost::format("%d") % (moduleNameLength + 1)).str();
  const std::string numWidth = (boost::format("%d") % (moduleNameLength + 1 + lengthOfRest)).str();
  boost::format outputheader("%s %|" + numTabsModule + "t|| %10s | %10s | %10s | %17s\n");
  boost::format output("%s %|" + numTabsModule + "t|| %10.0f | %10.0f | %10.2f | %7.2f +-%7.2f\n");
  if (html) {
    outputheader = boost::format("<thead><tr><th>%s</th><th>%s</th><th>%s</th><th>%s</th><th>%s</th></tr></thead>");
    output = boost::format("<tr><td>%s</td><td>%.0f</td><td>%.0f</td><td>%.2f</td><td>%.2f &plusmn; %.2f</td></tr>");
  }
 
  stringstream out;
  if (!html) {
    out << boost::format("%|" + numWidth + "T=|\n");
    out << outputheader % "Name" % "Calls" % "Memory(MB)" % "Time(s)" % "Time(ms)/Call";
    out << boost::format("%|" + numWidth + "T=|\n");
  } else {
    out << "<table border=0>";
    out << outputheader % "Name" % "Calls" % "Memory(MB)" % "Time(s)" % "Time(ms)/Call";
    out << "<tbody>";
  }
 
  std::vector<ModuleStatistics> modulesSortedByIndex(*modules);
  sort(modulesSortedByIndex.begin(), modulesSortedByIndex.end(), [](const ModuleStatistics & a, const ModuleStatistics & b) { return a.getIndex() < b.getIndex(); });
 
  for (const ModuleStatistics& stats : modulesSortedByIndex) {
    out << output
        % stats.getName()
        % stats.getCalls(mode)
        % (stats.getMemorySum(mode) / 1024)
        % (stats.getTimeSum(mode) / Unit::s)
        % (stats.getTimeMean(mode) / Unit::ms)
        % (stats.getTimeStddev(mode) / Unit::ms);
  }
 
  if (!html) {
    out << boost::format("%|" + numWidth + "T=|\n");
  } else {
    out << "</tbody><tfoot>";
  }
  out << output
      % (ProcHandler::isOutputProcess() ? "Total (output proc.)" : "Total")
      % global.getCalls(mode)
      % (global.getMemorySum(mode) / 1024)
      % (global.getTimeSum(mode) / Unit::s)
      % (global.getTimeMean(mode) / Unit::ms)
      % (global.getTimeStddev(mode) / Unit::ms);
  if (!html) {
    out << boost::format("%|" + numWidth + "T=|\n");
  } else {
    out << "</tfoot></table>";
  }
  return out.str();
}

Merge()

Long64_t Merge ( TCollection *  hlist )

virtualinherited

Allow merging using TFileMerger if saved directly to a file.

Note

dictionaries containing your Mergeable class need to be loaded, so 'hadd' will not work currently.

Definition at line 14 of file Mergeable.cc.

{
  Long64_t nMerged = 0;
  if (hlist) {
    const Mergeable* xh = nullptr;
    TIter nxh(hlist);
    while ((xh = dynamic_cast<Mergeable*>(nxh()))) {
      // Add xh to me
      merge(xh);
      ++nMerged;
    }
  }
  return nMerged;
}

removeSideEffects()

virtual void removeSideEffects ( )

inlinevirtualinherited

An ugly little method that is called before event() for input and worker processes.

Main use case is to detach any attached TFile from this object. In the output process, it can stay attached (and grow as much as it likes).

Reimplemented in RootMergeable< T >.

Definition at line 58 of file Mergeable.h.

{}

Reset()

virtual void Reset ( )

inlinevirtualinherited

Root-like Reset function for "template compatibility" with ROOT objects.

Alias for clear().

Definition at line 66 of file Mergeable.h.

setCounters()

void setCounters ( double &  time, double &  memory, double  startTime = 0, double  startMemory = 0  )

private

Set counters time and memory to contain the current clock value and memory consumption respectively.

Parameters

time	variable to store clock counter
memory	variable to store heap size
startTime	value to subtract from clock counter
startMemory	value to subtract from heap size

Definition at line 203 of file ProcessStatistics.cc.

SetDirectory()

virtual void SetDirectory ( TDirectory *  )

inlinevirtualinherited

Root-like SetDirectory function for "template compatibility" with ROOT objects.

Does nothing.

Definition at line 68 of file Mergeable.h.

setTransientCounters()

void setTransientCounters ( const ProcessStatistics *  otherObject )

private

Set transient counters from otherObject.

Needed since we swap objects inside input modules.

Definition at line 187 of file ProcessStatistics.cc.

suspendGlobal()

void suspendGlobal ( )

inline

Suspend timer for global measurement, needed for newRun.

resumeGlobal should be called once endRun/newRun handling is finished

Definition at line 116 of file ProcessStatistics.h.

Member Data Documentation

m_globalMemory

double m_globalMemory

private

(transient)

store heap size for global memory consumption in KB

Definition at line 216 of file ProcessStatistics.h.

m_moduleMemory

double m_moduleMemory

private

(transient)

store heap size for memory consumption by modules

Definition at line 220 of file ProcessStatistics.h.

m_moduleTime

double m_moduleTime

private

(transient)

store clock counter for time consumption by modules

Definition at line 218 of file ProcessStatistics.h.

m_suspendedMemory

double m_suspendedMemory

private

(transient)

store heap size for suspended measurement. Generally this would be a stack of values but we know that we need at most one element so we keep it a plain double.

Definition at line 228 of file ProcessStatistics.h.

m_suspendedTime

double m_suspendedTime

private

(transient)

store clock counter for suspended measurement. Generally this would be a stack of values but we know that we need at most one element so we keep it a plain double.

Definition at line 224 of file ProcessStatistics.h.

---

The documentation for this class was generated from the following files:

• framework/core/include/ProcessStatistics.h
• framework/core/src/ProcessStatistics.cc