KolmogorovTest Class Reference

Inheritance diagram for KolmogorovTest:

Public Member Functions
	__init__ (self, *args, **kwargs)
	ensure_compute (self)
	comparison_result (self)
	comparison_result_long (self)
	can_compare (self)

Public Attributes
	object_a = object_a
	store the first object to compare
	object_b = object_b
	store the second object to compare
	mop = mop
	MetaOptionParser.
	debug = debug
	enable debug?
bool	computed = False
	used to store, whether the quantities have already been compared

Protected Member Functions
	_compute (self)
str	_get_comparison_result_long (self)
str	_get_comparison_result (self)
bool	_has_correct_types (self)
None	_raise_has_correct_types (self)
bool	_has_compatible_bins (self)
None	_raise_has_compatible_bins (self)

Static Protected Member Functions
	_convert_teff_to_hist (teff_a)

Protected Attributes
	_pvalue = None
	pvalue
float	_pvalue_warn = self.mop.pvalue_warn()
	pvalue below which a warning is issued
float	_pvalue_error = self.mop.pvalue_error()
	pvalue below which an error is issued
str	_comparison_result = "not_compared"
	Comparison result, i.e.
str	_comparison_result_long = ""
	Longer description of the comparison result (e.g.

Static Protected Attributes
float	_default_pvalue_warn = 0.99
	Default pvalue below which a warning is issued (unless supplied in metaoptions)
float	_default_pvalue_error = 0.01
	Default pvalue below which an error is issued (unless supplied in metaoptions)

Detailed Description

 Kolmogorov-Smirnov Test 

Definition at line 589 of file validationcomparison.py.

Constructor & Destructor Documentation

__init__()

__init__	(		self,
		*	args,
		**	kwargs )

Initialize Kolmogorov test.
@param args: See arguments of :class:`ComparisonBase`
@param kwargs:  See arguments of :class:`ComparisonBase`

Definition at line 592 of file validationcomparison.py.

    def __init__(self, *args, **kwargs):
        """
        Initialize Kolmogorov test.
        @param args: See arguments of :class:`ComparisonBase`
        @param kwargs:  See arguments of :class:`ComparisonBase`
        """
        super().__init__(*args, **kwargs)
 

Member Function Documentation

_compute()

_compute ( self )

protected

Perform the actual test
@return: None

Reimplemented from PvalueTest.

Definition at line 600 of file validationcomparison.py.

    def _compute(self):
        """
        Perform the actual test
        @return: None
        """
        self._raise_has_correct_types()
        self._raise_has_compatible_bins()
 
        local_object_a = self.object_a
        local_object_b = self.object_b
 
        # very special handling for TEfficiencies
        if self.object_a.ClassName() == "TEfficiency":
            local_object_a = self._convert_teff_to_hist(self.object_a)
            local_object_b = self._convert_teff_to_hist(self.object_b)
            if self.debug:
                print("Converting TEfficiency objects to histograms.")
 
        option_str = "UON"
        if self.debug:
            option_str += "D"
 
        self._pvalue = local_object_a.KolmogorovTest(local_object_b, option_str)
 

_convert_teff_to_hist()

_convert_teff_to_hist ( teff_a )

staticprotectedinherited

Convert the content of a TEfficiency plot to a histogram and set
the bin content and errors

Definition at line 313 of file validationcomparison.py.

    def _convert_teff_to_hist(teff_a):
        """
        Convert the content of a TEfficiency plot to a histogram and set
        the bin content and errors
        """
        conv_hist = teff_a.GetTotalHistogram()
        xbin_count = conv_hist.GetNbinsX()
        xbin_low = conv_hist.GetXaxis().GetXmin()
        xbin_max = conv_hist.GetXaxis().GetXmax()
 
        th1 = ROOT.TH1D(
            teff_a.GetName() + "root_conversion",
            teff_a.GetName(),
            xbin_count,
            xbin_low,
            xbin_max,
        )
        # starting from the first to the last bin, ignoring the under/overflow
        # bins
        for i in range(1, xbin_count):
            th1.SetBinContent(i, teff_a.GetEfficiency(i))
            th1.SetBinError(i, teff_a.GetEfficiencyErrorLow(i))
 
        return th1
 
 

_get_comparison_result()

str _get_comparison_result ( self )

protectedinherited

 Used to format the value of :attr:`_comparison_result`. 

Reimplemented from ComparisonBase.

Definition at line 370 of file validationcomparison.py.

    def _get_comparison_result(self) -> str:
        if self._pvalue is None:
            return "error"
 
        if self._pvalue < self._pvalue_error:
            return "error"
        elif self._pvalue < self._pvalue_warn:
            return "warning"
        else:
            return "equal"
 

_get_comparison_result_long()

str _get_comparison_result_long ( self )

protected

 Used to format the value of :attr:`_comparison_result_long`. 

Reimplemented from PvalueTest.

Definition at line 624 of file validationcomparison.py.

    def _get_comparison_result_long(self) -> str:
        if self._pvalue is None:
            return (
                r"Could not perform Kolmogorov test between {{revision1}} "
                r"and {{revision2}} due to an unknown error. Please submit "
                r"a bug report."
            )
 
        return (
            r"Performed Kolmogorov test between {{revision1}} "
            r"and {{revision2}} "
            r" <b>p-value: {pvalue:.6f}</b> (p-value warn: {pvalue_warn}, "
            r"p-value error: {pvalue_error})".format(
                pvalue=self._pvalue,
                pvalue_warn=self._pvalue_warn,
                pvalue_error=self._pvalue_error,
            )
        )
 
 
# ------------------------------------------------------------------------------
# Anderson-Darling Test
# ------------------------------------------------------------------------------
 
 

_has_compatible_bins()

bool _has_compatible_bins ( self )

protectedinherited

Check if both ROOT objects have the same amount of bins
@return: True if the bins are equal, otherwise False

Definition at line 275 of file validationcomparison.py.

    def _has_compatible_bins(self) -> bool:
        """
        Check if both ROOT objects have the same amount of bins
        @return: True if the bins are equal, otherwise False
        """
        if (
            self.object_a.ClassName()
            == "TEfficiency"
            == self.object_b.ClassName()
        ):
            nbins_a = self.object_a.GetTotalHistogram().GetNbinsX()
            nbins_b = self.object_b.GetTotalHistogram().GetNbinsX()
        else:
            nbins_a = self.object_a.GetNbinsX()
            nbins_b = self.object_b.GetNbinsX()
 
        return nbins_a == nbins_b
 

_has_correct_types()

bool _has_correct_types ( self )

protectedinherited

@return: True if the two objects have a) a type supported for
    comparison and b) can be compared with each other

Definition at line 231 of file validationcomparison.py.

    def _has_correct_types(self) -> bool:
        """
        @return: True if the two objects have a) a type supported for
            comparison and b) can be compared with each other
        """
        if self.object_a is None or self.object_b is None:
            return False
 
        # check if the supplied object inherit from one of the supported types
        # and if they are of the same type
        supported_types = ["TProfile", "TH1D", "TH1F", "TEfficiency"]
        if self.object_a.ClassName() != self.object_b.ClassName():
            return False
        if self.object_a.ClassName() not in supported_types:
            return False
 
        if self.object_a.ClassName() == "TEfficiency":
            # can only handle TEfficiencies with dimension one atm
            if self.object_a.GetDimension() > 1:
                return False
 
        return True
 

_raise_has_compatible_bins()

None _raise_has_compatible_bins ( self )

protectedinherited

Raise Exception if not both ROOT objects have the same amount of bins
@return: None

Definition at line 293 of file validationcomparison.py.

    def _raise_has_compatible_bins(self) -> None:
        """
        Raise Exception if not both ROOT objects have the same amount of bins
        @return: None
        """
        if not self._has_compatible_bins():
            msg = (
                "The objects have differing x bin count: {} has {} vs. {} "
                "has {}."
            )
            raise DifferingBinCount(
                msg.format(
                    self.object_a.GetName(),
                    self.object_a.GetNbinsX(),
                    self.object_b.GetName(),
                    self.object_b.GetNbinsX(),
                )
            )
 

_raise_has_correct_types()

None _raise_has_correct_types ( self )

protectedinherited

Raise Exception if not the two objects have a) a type supported for
comparison and b) can be compared with each other
@return: None

Definition at line 254 of file validationcomparison.py.

    def _raise_has_correct_types(self) -> None:
        """
        Raise Exception if not the two objects have a) a type supported for
        comparison and b) can be compared with each other
        @return: None
        """
        if not self._has_correct_types():
            msg = (
                "Comparison of {} (Type {}) with {} (Type {}) not "
                "supported.\nPlease open a GitLab issue (validation "
                "label) if you need this supported. "
            )
            raise ObjectsNotSupported(
                msg.format(
                    self.object_a.GetName(),
                    self.object_a.ClassName(),
                    self.object_b.GetName(),
                    self.object_b.ClassName(),
                )
            )
 

can_compare()

can_compare ( self )

inherited

@return: True if the two objects can be compared, False otherwise

Definition at line 225 of file validationcomparison.py.

    def can_compare(self):
        """
        @return: True if the two objects can be compared, False otherwise
        """
        return self._has_correct_types() and self._has_compatible_bins()
 

comparison_result()

comparison_result ( self )

inherited

 Comparison result, i.e. pass/warning/error 

Definition at line 210 of file validationcomparison.py.

    def comparison_result(self):
        """ Comparison result, i.e. pass/warning/error """
        self.ensure_compute()
        return self._comparison_result
 

comparison_result_long()

comparison_result_long ( self )

inherited

 Longer description of the comparison result 

Definition at line 216 of file validationcomparison.py.

    def comparison_result_long(self):
        """ Longer description of the comparison result """
        self.ensure_compute()
        return self._comparison_result_long
 

ensure_compute()

ensure_compute ( self )

inherited

Ensure all required quantities get computed and are cached inside the
class

Definition at line 159 of file validationcomparison.py.

    def ensure_compute(self):
        """
        Ensure all required quantities get computed and are cached inside the
        class
        """
        if self.computed:
            return
 
        if self.mop.has_option("nocompare"):
            # is comparison disabled for this plot ?
            self._comparison_result_long = "Testing is disabled for this plot"
            return
 
        fail_message = "Comparison failed: "
 
        # Note: default for comparison_result is "not_compared"
        try:
            self._compute()
        except ObjectsNotSupported as e:
            self._comparison_result_long = fail_message + str(e)
        except DifferingBinCount as e:
            self._comparison_result = "error"
            self._comparison_result_long = fail_message + str(e)
        except TooFewBins as e:
            self._comparison_result_long = fail_message + str(e)
        except ComparisonFailed as e:
            self._comparison_result = "error"
            self._comparison_result_long = fail_message + str(e)
        except Exception as e:
            self._comparison_result = "error"
            self._comparison_result_long = (
                "Unknown error occurred. Please "
                "submit a bug report. " + str(e)
            )
        else:
            # Will be already set in case of errors above and we don't want
            # to overwrite this.
            self._comparison_result_long = self._get_comparison_result_long()
            self._comparison_result = self._get_comparison_result()
 
        self.computed = True
 

Member Data Documentation

_comparison_result

str _comparison_result = "not_compared"

protectedinherited

Comparison result, i.e.

equal/warning/error

Definition at line 154 of file validationcomparison.py.

_comparison_result_long

str _comparison_result_long = ""

protectedinherited

Longer description of the comparison result (e.g.

'performed Chi2 Test ... with chi2 = ...').

Definition at line 157 of file validationcomparison.py.

_default_pvalue_error

float _default_pvalue_error = 0.01

staticprotectedinherited

Default pvalue below which an error is issued (unless supplied in metaoptions)

Definition at line 348 of file validationcomparison.py.

_default_pvalue_warn

float _default_pvalue_warn = 0.99

staticprotectedinherited

Default pvalue below which a warning is issued (unless supplied in metaoptions)

Definition at line 344 of file validationcomparison.py.

_pvalue

_pvalue = None

protectedinherited

pvalue

Definition at line 359 of file validationcomparison.py.

_pvalue_error

float _pvalue_error = self.mop.pvalue_error()

protectedinherited

pvalue below which an error is issued

Definition at line 363 of file validationcomparison.py.

_pvalue_warn

float _pvalue_warn = self.mop.pvalue_warn()

protectedinherited

pvalue below which a warning is issued

Definition at line 361 of file validationcomparison.py.

computed

bool computed = False

inherited

used to store, whether the quantities have already been compared

Definition at line 151 of file validationcomparison.py.

debug

debug = debug

inherited

enable debug?

Definition at line 148 of file validationcomparison.py.

mop

mop = mop

inherited

MetaOptionParser.

Definition at line 145 of file validationcomparison.py.

object_a

object_a = object_a

inherited

store the first object to compare

Definition at line 137 of file validationcomparison.py.

object_b

object_b = object_b

inherited

store the second object to compare

Definition at line 140 of file validationcomparison.py.

---

The documentation for this class was generated from the following file:

• validation/scripts/validationcomparison.py