TestComparison Class Reference

Inheritance diagram for TestComparison:

Collaboration diagram for TestComparison:

Public Member Functions
	root_name (self, name)
	setUp (self)
	test_compare_profiles (self)
	test_compare_profiles_identical (self)
	test_compare_identical_profiles_kolmogorov (self)
	test_compare_profiles_almost_equal (self)
	test_compare_zero_error_profiles (self)
	test_compare_histograms (self)
	test_compare_unsupported_object (self)
	test_compare_differing_objects (self)
	test_compare_tefficiencies (self)
	test_compare_tefficiencies_same (self)
	test_compare_differing_bins (self)

Static Public Member Functions
	create_profile (name, entries=5000, mu=10, sigma=0.3, max_fill=50, fixed_number=None)
	create_teff (name, bins=20, eff=0.9)
	create_histogram (name, entries=5000, mu=10, sigma=3)

Public Attributes
int	call_iteration = 0
	if we would at some point later want to implement several runs, we use this as a counter variable to set up different names.
	profileA = self.create_profile(self.root_name("profileA"))
	Profile A.
	profileB = self.create_profile(self.root_name("profileB"))
	Profile B.
	profileC
	Profile C.
	profileZeroErrorBins
	Profile with bins with 0 error.
	profileZeroErrorBinsTwo
	Profile with bins with 0 error.
	histogramA
	Histogram A.
	histogramB
	Histogram B (should be almost equal to profile A)
	profileAequal
	Profile should be almost equal to A.
	profileBequal
	Profile should be almost equal to B.
	profileDifferentBins
	Profile with different bins.
	teffA = self.create_teff(self.root_name("teffA"))
	TEfficiemcy A.
	teffB = self.create_teff(self.root_name("teffB"))
	TEfficiency B.

Detailed Description

Various test cases for the Chi2Test wrapper class

Definition at line 72 of file test_comparison.py.

Member Function Documentation

create_histogram()

create_histogram ( name, entries = 5000, mu = 10, sigma = 3 )

static

Create a TH1D and fill with random content

Definition at line 110 of file test_comparison.py.

    def create_histogram(name, entries=5000, mu=10, sigma=3):
        """
        Create a TH1D and fill with random content
        """
        p = ROOT.TH1D(name, name, 50, 0, 20.0)
        for i in range(0, entries):
            p.Fill(random.gauss(mu, sigma))
        return p
 

create_profile()

create_profile ( name, entries = 5000, mu = 10, sigma = 0.3, max_fill = 50, fixed_number = None )

static

Create a TProfile object with various content

Definition at line 78 of file test_comparison.py.

    ):
        """
        Create a TProfile object with various content
        """
        p = ROOT.TProfile(name, name, 50, 0, 50.0)
        entries_per_bin = int(entries / max_fill)
        for i in range(0, max_fill + 1):
            for e in range(0, entries_per_bin):
                if fixed_number is None:
                    p.Fill(i + 0.5, random.gauss(mu, sigma))
                else:
                    p.Fill(i + 0.5, fixed_number)
        return p
 

create_teff()

create_teff ( name, bins = 20, eff = 0.9 )

static

Creates and fills a root TEfficiency plot

Definition at line 95 of file test_comparison.py.

    def create_teff(name, bins=20, eff=0.9):
        """
        Creates and fills a root TEfficiency plot
        """
        p = ROOT.TEfficiency(name, name, bins, 0, 50)
 
        for i in range(0, 5000):
            passed = random.uniform(0, 1.0) < eff
            bin_content = random.uniform(0.0, 50.0)
            p.Fill(passed, bin_content)
 
        return p
 

root_name()

root_name	(		self,
			name )

Generates unique names for ROOT objects

Definition at line 119 of file test_comparison.py.

    def root_name(self, name):
        """
        Generates unique names for ROOT objects
        """
        return f"{name}_{self.call_iteration}"
 

setUp()

setUp

(

self

)

Setup method to generate profiles and histograms for tests

Definition at line 125 of file test_comparison.py.

    def setUp(self):
        """
        Setup method to generate profiles and histograms for tests
        """
        random.seed(23)
 
        
        self.call_iteration = 0
 
        
        self.profileA = self.create_profile(self.root_name("profileA"))
 
        
        self.profileB = self.create_profile(self.root_name("profileB"))
 
        
        self.profileC = self.create_profile(
            self.root_name("profileC"), 5000, 5, 3
        )
 
        
        self.profileZeroErrorBins = self.create_profile(
            self.root_name("profileZeroErrorBins"), max_fill=49
        )
        self.profileZeroErrorBins.SetBinError(35, 0.0)
 
        
        self.profileZeroErrorBinsTwo = self.create_profile(
            self.root_name("profileZeroErrorBinsTwo"), max_fill=49
        )
        self.profileZeroErrorBinsTwo.SetBinError(35, 0.0)
 
        
        self.histogramA = self.create_histogram(
            self.root_name("histogramA"), 5000, 5, 3
        )
 
        
        self.histogramB = self.create_histogram(
            self.root_name("histogramB"), 5000, 5, 3
        )
 
        
        self.profileAequal = self.create_profile(
            self.root_name("profileA_almostequal"), sigma=0.4
        )
 
        
        self.profileBequal = self.create_profile(
            self.root_name("profileB_almostequal"), sigma=0.4
        )
 
        
        self.profileDifferentBins = ROOT.TProfile(
            self.root_name("profileDifferentBins"),
            self.root_name("profileDifferentBins"),
            40,
            0,
            50.0,
        )
 
        
        self.teffA = self.create_teff(self.root_name("teffA"))
 
        
        self.teffB = self.create_teff(self.root_name("teffB"))
 

test_compare_differing_bins()

test_compare_differing_bins

(

self

)

Test if the comparison attempt of profiles with differing bin count
fails properly

Definition at line 321 of file test_comparison.py.

    def test_compare_differing_bins(self):
        """
        Test if the comparison attempt of profiles with differing bin count
        fails properly
        """
        c = validationcomparison.Chi2Test(
            self.profileA, self.profileDifferentBins
        )
        self.assertFalse(c.can_compare())
 
        with self.assertRaises(validationcomparison.DifferingBinCount):
            c._compute()
 
 

test_compare_differing_objects()

test_compare_differing_objects

(

self

)

Test if the comparison of differing objects is rejected

Definition at line 281 of file test_comparison.py.

    def test_compare_differing_objects(self):
        """
        Test if the comparison of differing objects is rejected
        """
        c = validationcomparison.Chi2Test(self.profileA, self.histogramA)
        self.assertFalse(c.can_compare())
 
        with self.assertRaises(validationcomparison.ObjectsNotSupported):
            c._compute()
 

test_compare_histograms()

test_compare_histograms

(

self

)

Test comparison of regular histograms

Definition at line 259 of file test_comparison.py.

    def test_compare_histograms(self):
        """
        Test comparison of regular histograms
        """
 
        c = validationcomparison.Chi2Test(self.histogramA, self.histogramB)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.371600562118221)
        self.assertAlmostEqual(c._chi2, 42.308970111484086)
        self.assertAlmostEqual(c._chi2ndf, 1.0577242527871022)
        self.assertEqual(c._ndf, 40)
 

test_compare_identical_profiles_kolmogorov()

test_compare_identical_profiles_kolmogorov

(

self

)

 Test if comparing to identical TProfiles with Kolmo Test works

Definition at line 214 of file test_comparison.py.

    def test_compare_identical_profiles_kolmogorov(self):
        """ Test if comparing to identical TProfiles with Kolmo Test works"""
        c = validationcomparison.KolmogorovTest(self.profileA, self.profileA)
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 1)
 

test_compare_profiles()

test_compare_profiles

(

self

)

Test if comparing two similar TProfiles works

Definition at line 194 of file test_comparison.py.

    def test_compare_profiles(self):
        """
        Test if comparing two similar TProfiles works
        """
        c = validationcomparison.Chi2Test(self.profileA, self.profileB)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.22495088947037362)
 

test_compare_profiles_almost_equal()

test_compare_profiles_almost_equal

(

self

)

Test if the comparison of two TProfiles with very similar content works

Definition at line 221 of file test_comparison.py.

    def test_compare_profiles_almost_equal(self):
        """
        Test if the comparison of two TProfiles with very similar content works
        """
        c = validationcomparison.Chi2Test(
            self.profileAequal, self.profileBequal
        )
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.43093514577898634)
        self.assertAlmostEqual(c._ndf, 49)
 

test_compare_profiles_identical()

test_compare_profiles_identical

(

self

)

Test if comparing two identical TProfiles works

Definition at line 204 of file test_comparison.py.

    def test_compare_profiles_identical(self):
        """
        Test if comparing two identical TProfiles works
        """
        c = validationcomparison.Chi2Test(self.profileA, self.profileA)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 1)
 

test_compare_tefficiencies()

test_compare_tefficiencies

(

self

)

Test if two TEfficiency objects can be compared. Is a bit tricky
as TEfficiency does not support

Definition at line 291 of file test_comparison.py.

    def test_compare_tefficiencies(self):
        """
        Test if two TEfficiency objects can be compared. Is a bit tricky
        as TEfficiency does not support
        """
 
        c = validationcomparison.Chi2Test(self.teffA, self.teffB)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.9760318312199932)
        self.assertAlmostEqual(c._chi2, 8.16784873)
        self.assertAlmostEqual(c._chi2ndf, 0.45376937)
        self.assertEqual(c._ndf, 18)
 

test_compare_tefficiencies_same()

test_compare_tefficiencies_same

(

self

)

Test if two TEfficiency objects can be compared. Is a bit tricky as
TEfficiency does not support Comparing the exact same TEfficiency
object should give back 100% agreement

Definition at line 306 of file test_comparison.py.

    def test_compare_tefficiencies_same(self):
        """
        Test if two TEfficiency objects can be compared. Is a bit tricky as
        TEfficiency does not support Comparing the exact same TEfficiency
        object should give back 100% agreement
        """
 
        c = validationcomparison.Chi2Test(self.teffA, self.teffA)
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 1.0)
        self.assertAlmostEqual(c._chi2, 0.0)
        self.assertAlmostEqual(c._chi2ndf, 0.0)
        self.assertEqual(c._ndf, 18)
 

test_compare_unsupported_object()

test_compare_unsupported_object

(

self

)

Test if unsupported ROOT objects are treated properly

Definition at line 273 of file test_comparison.py.

    def test_compare_unsupported_object(self):
        """
        Test if unsupported ROOT objects are treated properly
        """
        obj_not_supported = ROOT.TH2D("h2d", "h2d", 50, 0, 50, 50, 0, 50)
        c = validationcomparison.Chi2Test(self.profileA, obj_not_supported)
        self.assertFalse(c.can_compare())
 

test_compare_zero_error_profiles()

test_compare_zero_error_profiles

(

self

)

Test if bins with zero error are treated properly

Definition at line 234 of file test_comparison.py.

    def test_compare_zero_error_profiles(self):
        """
        Test if bins with zero error are treated properly
        """
 
        # add entry in the last bin, which will increase the bin weight,
        # but leave the error at zero
        self.profileZeroErrorBins.Fill(49.8, 1.0)
        self.profileZeroErrorBins.Fill(49.8, 1.0)
 
        # the comparison should set the bin content of this bin to zero
        # to disable comparison of this bin instead of
        # not doing the comparison at all
        c = validationcomparison.Chi2Test(
            self.profileZeroErrorBins, self.profileZeroErrorBinsTwo
        )
 
        self.assertTrue(c.can_compare())
 
        c.ensure_compute()
 
        self.assertAlmostEqual(c._pvalue, 0.4835651485797353)
        # should still be only 49 ndf
        self.assertEqual(c._ndf, 49)
 

Member Data Documentation

call_iteration

int call_iteration = 0

if we would at some point later want to implement several runs, we use this as a counter variable to set up different names.

However not implemented yet.

Definition at line 134 of file test_comparison.py.

histogramA

histogramA

Initial value:

=  self.create_histogram(
            self.root_name("histogramA"), 5000, 5, 3
        )

Histogram A.

Definition at line 160 of file test_comparison.py.

histogramB

histogramB

Initial value:

=  self.create_histogram(
            self.root_name("histogramB"), 5000, 5, 3
        )

Histogram B (should be almost equal to profile A)

Definition at line 165 of file test_comparison.py.

profileA

profileA = self.create_profile(self.root_name("profileA"))

Profile A.

Definition at line 137 of file test_comparison.py.

profileAequal

profileAequal

Initial value:

=  self.create_profile(
            self.root_name("profileA_almostequal"), sigma=0.4
        )

Profile should be almost equal to A.

Definition at line 170 of file test_comparison.py.

profileB

profileB = self.create_profile(self.root_name("profileB"))

Profile B.

Definition at line 140 of file test_comparison.py.

profileBequal

profileBequal

Initial value:

=  self.create_profile(
            self.root_name("profileB_almostequal"), sigma=0.4
        )

Profile should be almost equal to B.

Definition at line 175 of file test_comparison.py.

profileC

profileC

Initial value:

=  self.create_profile(
            self.root_name("profileC"), 5000, 5, 3
        )

Profile C.

Definition at line 143 of file test_comparison.py.

profileDifferentBins

profileDifferentBins

Initial value:

=  ROOT.TProfile(
            self.root_name("profileDifferentBins"),
            self.root_name("profileDifferentBins"),
            40,
            0,
            50.0,
        )

Profile with different bins.

Definition at line 180 of file test_comparison.py.

profileZeroErrorBins

profileZeroErrorBins

Initial value:

=  self.create_profile(
            self.root_name("profileZeroErrorBins"), max_fill=49
        )

Profile with bins with 0 error.

Definition at line 148 of file test_comparison.py.

profileZeroErrorBinsTwo

profileZeroErrorBinsTwo

Initial value:

=  self.create_profile(
            self.root_name("profileZeroErrorBinsTwo"), max_fill=49
        )

Profile with bins with 0 error.

Definition at line 154 of file test_comparison.py.

teffA

teffA = self.create_teff(self.root_name("teffA"))

TEfficiemcy A.

Definition at line 189 of file test_comparison.py.

teffB

teffB = self.create_teff(self.root_name("teffB"))

TEfficiency B.

Definition at line 192 of file test_comparison.py.

---

The documentation for this class was generated from the following file:

• validation/tests/test_comparison.py