Module Statistics

5.3. Module Statistics#

The basf2 software takes extensive statistics during event processing about the memory consumption and execution time of all modules. For most users a simple print of the statistics object will be enough and creates a text table of the execution times and memory conumption:

import basf2
print(basf2.statistics)

However the statistics object provides full access to all the separate values directly in python if needed. See module_statistics.py for a full example.

Note

The memory consumption is measured by looking into /proc/PID/statm between execution calls so for short running modules this might not be accurate but it should give a general idea.

basf2.statistics#

Global instance of a ProcessStatistics object containing all the statistics

class basf2.ProcessStatistics#

Interface for retrieving statistics about module execution at runtime or after basf2.process() returns. Should be accessed through a global instance basf2.statistics.

Statistics for event() calls are available as a string representation of the object:

>>> from basf2 import statistics
>>> print(statistics)
=================================================================================
Name                  |      Calls | Memory(MB) |    Time(s) |     Time(ms)/Call
=================================================================================
RootInput             |        101 |          0 |       0.01 |    0.05 +-   0.02
RootOutput            |        100 |          0 |       0.02 |    0.20 +-   0.87
ProgressBar           |        100 |          0 |       0.00 |    0.00 +-   0.00
=================================================================================
Total                 |        101 |          0 |       0.03 |    0.26 +-   0.86
=================================================================================

This provides information on the number of calls, elapsed time, and the average difference in resident memory before and after the event() call.

Note

The module responsible for reading (or generating) events usually has one additional event() call which is used to determine whether event processing should stop.

Warning

Memory consumption is reporting the difference in memory usage as reported by the kernel before and after the call. This is not the maximum memory the module has consumed. Negative values indicate that this module has freed momemory which was allocated in other modules or function calls.

Information on other calls like initialize(), terminate(), etc. are also available through the different counters defined in StatisticCounters:

>>> print(statistics(statistics.INIT))
>>> print(statistics(statistics.BEGIN_RUN))
>>> print(statistics(statistics.END_RUN))
>>> print(statistics(statistics.TERM))
class ModuleStatistics#

Execution statistics for a single module. All member functions take exactly one argument to select which counter to query which defaults to StatisticCounters.TOTAL if omitted.

__reduce__()#

Helper for pickle.

calls(counter=StatisticCounters.TOTAL)#

Return the total number of calls

memory_mean(counter=StatisticCounters.TOTAL)#

Return the mean of the memory usage

memory_stddev(counter=StatisticCounters.TOTAL)#

Return the standard deviation of the memory usage

memory_sum(counter=StatisticCounters.TOTAL)#

Return the sum of the total memory usage

property name#

property to get the name of the module to be displayed in the statistics

time_mean(counter=StatisticCounters.TOTAL)#

Return the mean of all execution times

time_memory_corr(counter=StatisticCounters.TOTAL)#

Return the correlaction factor between time and memory consumption

time_stddev(counter=StatisticCounters.TOTAL)#

Return the standard deviation of all execution times

time_sum(counter=StatisticCounters.TOTAL)#

Return the sum of all execution times

class StatisticCounters#

Available types of statistic counters (corresponds to Module functions)

INIT#

Time spent or memory used in the initialize() function

BEGIN_RUN#

Time spent or memory used in the beginRun() function

EVENT#

Time spent or memory used in the event() function

END_RUN#

Time spent or memory used in the endRun() function

TERM#

Time spent or memory used in the terminate() function

TOTAL#

Time spent or memory used in any module function. This is the sum of all of the above.

__call__(counter=StatisticCounters.EVENT, modules=None)#

Calling the statistics object directly like a function will return a string with the execution statistics in human readable form.

Parameters:

  • counter (StatisticCounters) – Which counter to use

  • modules (list[Module]) – A list of modules to include in the returned string. If omitted the statistics for all modules will be included.

  • print the beginRun() statistics for all modules:

    >>> print(statistics(statistics.BEGIN_RUN))

  • print the total execution times and memory consumption but only for the modules module1 and module2

    >>> print(statistics(statistics.TOTAL, [module1, module2]))

  • print the event statistics (default) for only two modules

    >>> print(statistics(modules=[module1, module2]))
__reduce__()#

Helper for pickle.

__str__()#

Return the event statistics as a string in a human readable form

clear() → None :#

Clear collected statistics but keep names of modules

csv((str)arg2) → None :#

Write statistics to a csv file

get((Module)module) → ModuleStatistics :#

Get ModuleStatistics for given Module.

get_global() → ModuleStatistics :#

Get global ModuleStatistics containing total elapsed time etc.

property modules#

List of all ModuleStatistics objects.

Contents