Module Statistics
5.3. Module Statistics#
The basf2 software takes extensive statistics during event processing about the
memory consumption and execution time of all modules. For most users a simple
print of the statistics
object will be enough and creates a text table of the
execution times and memory conumption:
import basf2
print(basf2.statistics)
However the statistics object provides full access to all the separate values
directly in python if needed. See
module_statistics.py
for a full example.
Note
The memory consumption is measured by looking into /proc/PID/statm
between execution calls so for short running modules this might not be
accurate
but it should give a general idea.
- basf2.statistics#
Global instance of a
ProcessStatistics
object containing all the statistics
- class basf2.ProcessStatistics#
Interface for retrieving statistics about module execution at runtime or after
basf2.process()
returns. Should be accessed through a global instancebasf2.statistics
.Statistics for
event()
calls are available as a string representation of the object:>>> from basf2 import statistics >>> print(statistics) ================================================================================= Name | Calls | Memory(MB) | Time(s) | Time(ms)/Call ================================================================================= RootInput | 101 | 0 | 0.01 | 0.05 +- 0.02 RootOutput | 100 | 0 | 0.02 | 0.20 +- 0.87 ProgressBar | 100 | 0 | 0.00 | 0.00 +- 0.00 ================================================================================= Total | 101 | 0 | 0.03 | 0.26 +- 0.86 =================================================================================
This provides information on the number of calls, elapsed time, and the average difference in resident memory before and after the
event()
call.Note
The module responsible for reading (or generating) events usually has one additional event() call which is used to determine whether event processing should stop.
Warning
Memory consumption is reporting the difference in memory usage as reported by the kernel before and after the call. This is not the maximum memory the module has consumed. Negative values indicate that this module has freed momemory which was allocated in other modules or function calls.
Information on other calls like
initialize()
,terminate()
, etc. are also available through the different counters defined inStatisticCounters
:- class ModuleStatistics#
Execution statistics for a single module. All member functions take exactly one argument to select which counter to query which defaults to
StatisticCounters.TOTAL
if omitted.- __reduce__()#
Helper for pickle.
- calls(counter=StatisticCounters.TOTAL)#
Return the total number of calls
- memory_mean(counter=StatisticCounters.TOTAL)#
Return the mean of the memory usage
- memory_stddev(counter=StatisticCounters.TOTAL)#
Return the standard deviation of the memory usage
- memory_sum(counter=StatisticCounters.TOTAL)#
Return the sum of the total memory usage
- property name#
property to get the name of the module to be displayed in the statistics
- time_mean(counter=StatisticCounters.TOTAL)#
Return the mean of all execution times
- time_memory_corr(counter=StatisticCounters.TOTAL)#
Return the correlaction factor between time and memory consumption
- time_stddev(counter=StatisticCounters.TOTAL)#
Return the standard deviation of all execution times
- time_sum(counter=StatisticCounters.TOTAL)#
Return the sum of all execution times
- class StatisticCounters#
Available types of statistic counters (corresponds to Module functions)
- INIT#
Time spent or memory used in the
initialize()
function- BEGIN_RUN#
Time spent or memory used in the
beginRun()
function- EVENT#
Time spent or memory used in the
event()
function- END_RUN#
Time spent or memory used in the
endRun()
function- TERM#
Time spent or memory used in the
terminate()
function- TOTAL#
Time spent or memory used in any module function. This is the sum of all of the above.
- __call__(counter=StatisticCounters.EVENT, modules=None)#
Calling the statistics object directly like a function will return a string with the execution statistics in human readable form.
- Parameters
counter (StatisticCounters) – Which counter to use
modules (list[Module]) – A list of modules to include in the returned string. If omitted the statistics for all modules will be included.
print the
beginRun()
statistics for all modules:>>> print(statistics(statistics.BEGIN_RUN))
print the total execution times and memory consumption but only for the modules
module1
andmodule2
>>> print(statistics(statistics.TOTAL, [module1, module2]))
print the event statistics (default) for only two modules
>>> print(statistics(modules=[module1, module2]))
- __reduce__()#
Helper for pickle.
- __str__()#
Return the event statistics as a string in a human readable form
- clear() None : #
Clear collected statistics but keep names of modules
- get((Module)module) ModuleStatistics : #
Get
ModuleStatistics
for given Module.
- get_global() ModuleStatistics : #
Get global
ModuleStatistics
containing total elapsed time etc.
- property modules#
List of all
ModuleStatistics
objects.