ProcessStatisticsPython Class Reference

Python interface for ProcessStatistics. More...

#include <ProcessStatisticsPython.h>

Public Member Functions
	ProcessStatisticsPython (ModuleStatistics::EStatisticCounters type=ModuleStatistics::c_Event, const std::vector< ModuleStatistics > &modules={})
	Construct a new object to show statistics for a given call type.
	ProcessStatisticsPython (const ProcessStatisticsPython &)=default
	Default copy constructor.
ProcessStatisticsPython &	operator= (const ProcessStatisticsPython &)=default
	Default assignment operator.
	~ProcessStatisticsPython ()=default
	Default destructor.
ProcessStatistics *	getWrapped ()
	Get wrapped ProcessStatistics object.
std::string	getStatisticsString ()
	Return string with statistics for all selected modules.
std::string	getStatisticsStringHTML ()
	Return string with statistics for all selected modules as html table.
ProcessStatisticsPython	getModuleStatistics (ModuleStatistics::EStatisticCounters type, const boost::python::list &modulesPyList)
	Get a new statistics object for a different counter/different list of modules.
const ModuleStatistics *	get (const std::shared_ptr< Module > &module)
	Get statistics for given module.
const ModuleStatistics *	getGlobal ()
	Get statistics for the framework itself.
boost::python::list	getAll ()
	Get statistics for all modules as python list.
void	clear ()
	Clear collected statistics but keep names of modules.
void	csv (const char *filename)
	Write statistics to a csv file.

Static Public Member Functions
static void	exposePythonAPI ()
	Define python wrappers to make functionality available in python.

Private Attributes
ModuleStatistics::EStatisticCounters	m_type {ModuleStatistics::c_Event}
	Which counter to show when printing the statistics.
std::vector< ModuleStatistics >	m_modules
	Which modules to show.

Detailed Description

Python interface for ProcessStatistics.

Since ProcessStatistics itself resides in the data store, this wraps around it to export some functionality to python.

Definition at line 31 of file ProcessStatisticsPython.h.

Constructor & Destructor Documentation

ProcessStatisticsPython()

ProcessStatisticsPython ( ModuleStatistics::EStatisticCounters type = ModuleStatistics::c_Event, const std::vector< ModuleStatistics > & modules = {} )

inline

Construct a new object to show statistics for a given call type.

Possibly also restricting the list of modules to be shown

Definition at line 34 of file ProcessStatisticsPython.h.

                                                                         {}): m_type{type}, m_modules{modules} {}

Member Function Documentation

clear()

void clear

(

)

Clear collected statistics but keep names of modules.

Definition at line 95 of file ProcessStatisticsPython.cc.

{
  if (!getWrapped())
    return;
  getWrapped()->clear();
}

csv()

void csv

(

const char *

filename

)

Write statistics to a csv file.

Definition at line 102 of file ProcessStatisticsPython.cc.

{
  if (!getWrapped())
    return;
  getWrapped()->write_csv(filename);
}

exposePythonAPI()

void exposePythonAPI ( )

static

Define python wrappers to make functionality available in python.

Definition at line 110 of file ProcessStatisticsPython.cc.

{
  // to avoid confusion between std::arg and boost::python::arg we want a shorthand namespace as well
  namespace bp = boost::python;
 
  //Reference to global scope
  scope global;
 
  docstring_options options(true, true, false); //userdef, py sigs, c++ sigs
 
  //Wrap ProcessStatisticsPython as non-copy and non-instantiable in python
  class_<ProcessStatisticsPython> stats("ProcessStatistics", R"DOCSTRING(
Interface for retrieving statistics about module execution at runtime or after
:py:func:`basf2.process()` returns. Should be accessed through a global instance `basf2.statistics`.
 
Statistics for `event() <Module.event()>` calls are available as a string representation of the object:
 
>>> from basf2 import statistics
>>> print(statistics)
=================================================================================
Name                  |      Calls | Memory(MB) |    Time(s) |     Time(ms)/Call
=================================================================================
RootInput             |        101 |          0 |       0.01 |    0.05 +-   0.02
RootOutput            |        100 |          0 |       0.02 |    0.20 +-   0.87
ProgressBar           |        100 |          0 |       0.00 |    0.00 +-   0.00
=================================================================================
Total                 |        101 |          0 |       0.03 |    0.26 +-   0.86
=================================================================================
 
This provides information on the number of calls, elapsed time, and the average
difference in resident memory before and after the `event() <Module.event>` call.
 
.. note::
 
    The module responsible for reading (or generating) events usually has one
    additional event() call which is used to determine whether event processing
    should stop.
 
.. warning::
 
    Memory consumption is reporting the difference in memory usage as reported
    by the kernel before and after the call. This is not the maximum memory the
    module has consumed. Negative values indicate that this module has freed
    memory which was allocated in other modules or function calls.
 
Information on other calls like `initialize() <Module.initialize>`,
`terminate() <Module.terminate>`, etc. are also available through the different
counters defined in `StatisticCounters`:
 
>>> print(statistics(statistics.INIT))
>>> print(statistics(statistics.BEGIN_RUN))
>>> print(statistics(statistics.END_RUN))
>>> print(statistics(statistics.TERM))
)DOCSTRING", no_init);
 
  stats
  .def("get", &ProcessStatisticsPython::get, return_value_policy<reference_existing_object>(), bp::arg("module"),
       "Get `ModuleStatistics` for given Module.")
  .def("get_global", &ProcessStatisticsPython::getGlobal, return_value_policy<reference_existing_object>(),
       "Get global `ModuleStatistics` containing total elapsed time etc.")
  .def("clear", &ProcessStatisticsPython::clear, "Clear collected statistics but keep names of modules")
  .def_readonly("modules", &ProcessStatisticsPython::getAll, "List of all `ModuleStatistics` objects.")
  .def("csv", &ProcessStatisticsPython::csv, "Write statistics to a csv file")
  ;
 
  //Set scope to current class
  scope statistics{stats};
  //Define enum for all the counter types in scope of class
  enum_<ModuleStatistics::EStatisticCounters>("StatisticCounters", R"DOCSTRING(
Available types of statistic counters (corresponds to Module functions)
 
.. attribute:: INIT
 
Time spent or memory used in the `initialize() <Module.initialize>` function
 
.. attribute:: BEGIN_RUN
 
Time spent or memory used in the `beginRun() <Module.beginRun>` function
 
.. attribute:: EVENT
 
Time spent or memory used in the `event() <Module.event>` function
 
.. attribute:: END_RUN
 
Time spent or memory used in the `endRun() <Module.endRun>` function
 
.. attribute:: TERM
 
Time spent or memory used in the `terminate() <Module.terminate>` function
 
.. attribute:: TOTAL
 
Time spent or memory used in any module function. This is the sum of all of the above.
 
)DOCSTRING")
  .value("INIT", ModuleStatistics::c_Init)
  .value("BEGIN_RUN", ModuleStatistics::c_BeginRun)
  .value("EVENT", ModuleStatistics::c_Event)
  .value("END_RUN", ModuleStatistics::c_EndRun)
  .value("TERM", ModuleStatistics::c_Term)
  .value("TOTAL", ModuleStatistics::c_Total)
  .export_values()
  ;
 
  {
    // the overloaded __str__ and __call__ give very confusing signatures so hand-craft doc string.
  docstring_options custom_options(true, false, false); //userdef, py sigs, c++ sigs
  stats
  .def("__str__", &ProcessStatisticsPython::getStatisticsString,
       "Return the event statistics as a string in a human readable form")
  .def("_repr_html_", &ProcessStatisticsPython::getStatisticsStringHTML,
       "Return an html representation of the statistics (used by ipython/jupyter)")
  .def("__call__", &ProcessStatisticsPython::getModuleStatistics, (bp::arg("counter") = ModuleStatistics::EStatisticCounters::c_Event, bp::arg("modules") = boost::python::list()),
       R"DOCSTRING(__call__(counter=StatisticCounters.EVENT, modules=None)
 
Calling the statistics object directly like a function will return a string
with the execution statistics in human readable form.
 
Parameters:
    counter (StatisticCounters): Which counter to use
    modules (list[Module]): A list of modules to include in the returned string.
        If omitted the statistics for all modules will be included.
 
* print the `beginRun() <Module.beginRun>` statistics for all modules:
 
  >>> print(statistics(statistics.BEGIN_RUN))
 
* print the total execution times and memory consumption but only for the
  modules ``module1`` and ``module2``
 
  >>> print(statistics(statistics.TOTAL, [module1, module2]))
 
* print the event statistics (default) for only two modules
 
  >>> print(statistics(modules=[module1, module2]))
)DOCSTRING")
  ;
  }
 
  //Wrap statistics class. The default boost python docstring signature is way
  //to noisy for these simple getters so this time we do it ourselves ...
  docstring_options new_options(true, false, false); //userdef, py sigs, c++ sigs
  class_<ModuleStatistics>("ModuleStatistics", "Execution statistics for a single module. "
                           "All member functions take exactly one argument to select which "
                           "counter to query which defaults to `StatisticCounters.TOTAL` if omitted.")
  .add_property("name", make_function(&ModuleStatistics::getName, return_value_policy<copy_const_reference>()),

get()

const ModuleStatistics * get

(

const std::shared_ptr< Module > &

module

)

Get statistics for given module.

Definition at line 81 of file ProcessStatisticsPython.cc.

{
  if (!getWrapped())
    return nullptr;
  return &getWrapped()->getStatistics(module.get());
}

getAll()

boost::python::list getAll

(

)

Get statistics for all modules as python list.

Definition at line 70 of file ProcessStatisticsPython.cc.

{
  boost::python::list result;
  if (!getWrapped())
    return result;
  for (auto& module : (m_modules.empty()) ? getWrapped()->getAll() : m_modules) {
    result.append(module);
  }
  return result;
}

getGlobal()

const ModuleStatistics * getGlobal

(

)

Get statistics for the framework itself.

Definition at line 88 of file ProcessStatisticsPython.cc.

{
  if (!getWrapped())
    return nullptr;
  return &getWrapped()->getGlobal();
 
}

getModuleStatistics()

ProcessStatisticsPython getModuleStatistics	(	ModuleStatistics::EStatisticCounters	type,
		const boost::python::list &	modulesPyList )

Get a new statistics object for a different counter/different list of modules.

Definition at line 53 of file ProcessStatisticsPython.cc.

{
  if (!getWrapped())
    return ProcessStatisticsPython();
 
  std::vector<ModuleStatistics> moduleStats;
  auto modules = PyObjConvUtils::convertPythonObject(modulesPyList, std::vector<ModulePtr>());
  for (const ModulePtr& ptr : modules) {
    ModuleStatistics& stats = getWrapped()->getStatistics(ptr.get());
    //Name could be empty if module has never been called
    if (stats.getName().empty()) stats.setName(ptr->getName());
    moduleStats.push_back(stats);
  }
  return ProcessStatisticsPython(type, moduleStats);
}

getStatisticsString()

string getStatisticsString

(

)

Return string with statistics for all selected modules.

If none are selected show all modules

Can be used in steering file with 'print(statistics)'.

Definition at line 33 of file ProcessStatisticsPython.cc.

{
  if (!getWrapped())
    return "";
  if (getWrapped()->getStatisticsPrintStatus(m_type))
    return "";
  getWrapped()->setStatisticsPrintStatus(m_type, true);
  return getWrapped()->getStatisticsString(m_type, m_modules.empty() ? nullptr : &m_modules);
}

getStatisticsStringHTML()

string getStatisticsStringHTML

(

)

Return string with statistics for all selected modules as html table.

If none are selected show all modules

Is used in Jupyter notebooks

Definition at line 43 of file ProcessStatisticsPython.cc.

{
  if (!getWrapped())
    return "";
  if (getWrapped()->getStatisticsPrintStatus(m_type))
    return "";
  getWrapped()->setStatisticsPrintStatus(m_type, true);
  return getWrapped()->getStatisticsString(m_type, m_modules.empty() ? nullptr : &m_modules, true);
}

getWrapped()

ProcessStatistics * getWrapped

(

)

Get wrapped ProcessStatistics object.

Definition at line 21 of file ProcessStatisticsPython.cc.

{
  StoreObjPtr<ProcessStatistics> stats("", DataStore::c_Persistent);
  if (!stats) {
    if (!Environment::Instance().getDryRun()) {
      B2ERROR("ProcessStatistics data object is not available, you either didn't enable statistics with --stats or didn't run process(path) yet.");
    }
    return nullptr;
  }
  return &(*stats);
}

Member Data Documentation

m_modules

std::vector<ModuleStatistics> m_modules

private

Which modules to show.

If empty, show all modules

Definition at line 86 of file ProcessStatisticsPython.h.

m_type

ModuleStatistics::EStatisticCounters m_type {ModuleStatistics::c_Event}

private

Which counter to show when printing the statistics.

Definition at line 84 of file ProcessStatisticsPython.h.

{ModuleStatistics::c_Event};

---

The documentation for this class was generated from the following files:

• framework/pybasf2/include/ProcessStatisticsPython.h
• framework/pybasf2/src/ProcessStatisticsPython.cc