TestComparison Class Reference

Inheritance diagram for TestComparison:

Public Member Functions
def	root_name (self, name)
def	setUp (self)
def	test_compare_profiles (self)
def	test_compare_profiles_identical (self)
def	test_compare_identical_profiles_kolmogorov (self)
def	test_compare_profiles_almost_equal (self)
def	test_compare_zero_error_profiles (self)
def	test_compare_histograms (self)
def	test_compare_unsupported_object (self)
def	test_compare_differing_objects (self)
def	test_compare_tefficiencies (self)
def	test_compare_tefficiencies_same (self)
def	test_compare_differing_bins (self)

Static Public Member Functions
def	create_profile (name, entries=5000, mu=10, sigma=0.3, max_fill=50, fixed_number=None)
def	create_teff (name, bins=20, eff=0.9)
def	create_histogram (name, entries=5000, mu=10, sigma=3)

Public Attributes
	call_iteration
	if we would at some point later want to implement several runs, we use this as a counter variable to set up different names.
	profileA
	Profile A.
	profileB
	Profile B.
	profileC
	Profile C.
	profileZeroErrorBins
	Profile with bins with 0 error.
	profileZeroErrorBinsTwo
	Profile with bins with 0 error.
	histogramA
	Histogram A.
	histogramB
	Histogram B (should be almost equal to profile A)
	profileAequal
	Profile should be almost equal to A.
	profileBequal
	Profile should be almost equal to B.
	profileDifferentBins
	Profile with different bins.
	teffA
	TEfficiemcy A.
	teffB
	TEfficiency B.

Detailed Description

Various test cases for the Chi2Test wrapper class

Definition at line 72 of file test_comparison.py.

Member Function Documentation

create_histogram()

def create_histogram (   name,   entries = 5000,   mu = 10,   sigma = 3  )

static

Create a TH1D and fill with random content

Definition at line 110 of file test_comparison.py.

    def create_histogram(name, entries=5000, mu=10, sigma=3):
        """
        Create a TH1D and fill with random content
        """
        p = ROOT.TH1D(name, name, 50, 0, 20.0)
        for i in range(0, entries):
            p.Fill(random.gauss(mu, sigma))
        return p
 

create_profile()

def create_profile (   name,   entries = 5000,   mu = 10,   sigma = 0.3,   max_fill = 50,   fixed_number = None  )

static

Create a TProfile object with various content

Definition at line 78 of file test_comparison.py.

    ):
        """
        Create a TProfile object with various content
        """
        p = ROOT.TProfile(name, name, 50, 0, 50.0)
        entries_per_bin = int(entries / max_fill)
        for i in range(0, max_fill + 1):
            for e in range(0, entries_per_bin):
                if fixed_number is None:
                    p.Fill(i + 0.5, random.gauss(mu, sigma))
                else:
                    p.Fill(i + 0.5, fixed_number)
        return p
 

create_teff()

def create_teff (   name,   bins = 20,   eff = 0.9  )

static

Creates and fills a root TEfficiency plot

Definition at line 95 of file test_comparison.py.

    def create_teff(name, bins=20, eff=0.9):
        """
        Creates and fills a root TEfficiency plot
        """
        p = ROOT.TEfficiency(name, name, bins, 0, 50)
 
        for i in range(0, 5000):
            passed = random.uniform(0, 1.0) < eff
            bin_content = random.uniform(0.0, 50.0)
            p.Fill(passed, bin_content)
 
        return p
 

root_name()

def root_name	(		self,
			name
	)

Generates unique names for ROOT objects

Definition at line 119 of file test_comparison.py.

    def root_name(self, name):
        """
        Generates unique names for ROOT objects
        """
        return f"{name}_{self.call_iteration}"
 

setUp()

def setUp

(

self

)

Setup method to generate profiles and histograms for tests

Definition at line 125 of file test_comparison.py.

    def setUp(self):
        """
        Setup method to generate profiles and histograms for tests
        """
        random.seed(23)
 
        
        self.call_iteration = 0
 
        
        self.profileA = self.create_profile(self.root_name("profileA"))
 
        
        self.profileB = self.create_profile(self.root_name("profileB"))
 
        
        self.profileC = self.create_profile(
            self.root_name("profileC"), 5000, 5, 3
        )
 
        
        self.profileZeroErrorBins = self.create_profile(
            self.root_name("profileZeroErrorBins"), max_fill=49
        )
        self.profileZeroErrorBins.SetBinError(35, 0.0)
 
        
        self.profileZeroErrorBinsTwo = self.create_profile(
            self.root_name("profileZeroErrorBinsTwo"), max_fill=49
        )
        self.profileZeroErrorBinsTwo.SetBinError(35, 0.0)
 
        
        self.histogramA = self.create_histogram(
            self.root_name("histogramA"), 5000, 5, 3
        )
 
        
        self.histogramB = self.create_histogram(
            self.root_name("histogramB"), 5000, 5, 3
        )
 
        
        self.profileAequal = self.create_profile(
            self.root_name("profileA_almostequal"), sigma=0.4
        )
 
        
        self.profileBequal = self.create_profile(
            self.root_name("profileB_almostequal"), sigma=0.4
        )
 
        
        self.profileDifferentBins = ROOT.TProfile(
            self.root_name("profileDifferentBins"),
            self.root_name("profileDifferentBins"),
            40,
            0,
            50.0,
        )
 
        
        self.teffA = self.create_teff(self.root_name("teffA"))
 
        
        self.teffB = self.create_teff(self.root_name("teffB"))
 

test_compare_differing_bins()

def test_compare_differing_bins

(

self

)

Test if the comparison attempt of profiles with differing bin count
fails properly

Definition at line 321 of file test_comparison.py.

    def test_compare_differing_bins(self):
        """
        Test if the comparison attempt of profiles with differing bin count
        fails properly
        """
        c = validationcomparison.Chi2Test(
            self.profileA, self.profileDifferentBins
        )
        self.assertFalse(c.can_compare())
 
        with self.assertRaises(validationcomparison.DifferingBinCount):
            c._compute()
 
 

test_compare_differing_objects()

def test_compare_differing_objects

(

self

)

Test if the comparison of differing objects is rejected

Definition at line 281 of file test_comparison.py.

    def test_compare_differing_objects(self):
        """
        Test if the comparison of differing objects is rejected
        """
        c = validationcomparison.Chi2Test(self.profileA, self.histogramA)
        self.assertFalse(c.can_compare())
 
        with self.assertRaises(validationcomparison.ObjectsNotSupported):
            c._compute()
 

test_compare_histograms()

def test_compare_histograms

(

self

)

Test comparison of regular histograms

Definition at line 259 of file test_comparison.py.

    def test_compare_histograms(self):
        """
        Test comparison of regular histograms
        """
 
        c = validationcomparison.Chi2Test(self.histogramA, self.histogramB)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.371600562118221)
        self.assertAlmostEqual(c._chi2, 42.308970111484086)
        self.assertAlmostEqual(c._chi2ndf, 1.0577242527871022)
        self.assertEqual(c._ndf, 40)
 

test_compare_identical_profiles_kolmogorov()

def test_compare_identical_profiles_kolmogorov

(

self

)

 Test if comparing to identical TProfiles with Kolmo Test works

Definition at line 214 of file test_comparison.py.

    def test_compare_identical_profiles_kolmogorov(self):
        """ Test if comparing to identical TProfiles with Kolmo Test works"""
        c = validationcomparison.KolmogorovTest(self.profileA, self.profileA)
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 1)
 

test_compare_profiles()

def test_compare_profiles

(

self

)

Test if comparing two similar TProfiles works

Definition at line 194 of file test_comparison.py.

    def test_compare_profiles(self):
        """
        Test if comparing two similar TProfiles works
        """
        c = validationcomparison.Chi2Test(self.profileA, self.profileB)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.22495088947037362)
 

test_compare_profiles_almost_equal()

def test_compare_profiles_almost_equal

(

self

)

Test if the comparison of two TProfiles with very similar content works

Definition at line 221 of file test_comparison.py.

    def test_compare_profiles_almost_equal(self):
        """
        Test if the comparison of two TProfiles with very similar content works
        """
        c = validationcomparison.Chi2Test(
            self.profileAequal, self.profileBequal
        )
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.43093514577898634)
        self.assertAlmostEqual(c._ndf, 49)
 

test_compare_profiles_identical()

def test_compare_profiles_identical

(

self

)

Test if comparing two identical TProfiles works

Definition at line 204 of file test_comparison.py.

    def test_compare_profiles_identical(self):
        """
        Test if comparing two identical TProfiles works
        """
        c = validationcomparison.Chi2Test(self.profileA, self.profileA)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 1)
 

test_compare_tefficiencies()

def test_compare_tefficiencies

(

self

)

Test if two TEfficiency objects can be compared. Is a bit tricky
as TEfficiency does not support

Definition at line 291 of file test_comparison.py.

    def test_compare_tefficiencies(self):
        """
        Test if two TEfficiency objects can be compared. Is a bit tricky
        as TEfficiency does not support
        """
 
        c = validationcomparison.Chi2Test(self.teffA, self.teffB)
 
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 0.9760318312199932)
        self.assertAlmostEqual(c._chi2, 8.16784873)
        self.assertAlmostEqual(c._chi2ndf, 0.45376937)
        self.assertEqual(c._ndf, 18)
 

test_compare_tefficiencies_same()

def test_compare_tefficiencies_same

(

self

)

Test if two TEfficiency objects can be compared. Is a bit tricky as
TEfficiency does not support Comparing the exact same TEfficiency
object should give back 100% agreement

Definition at line 306 of file test_comparison.py.

    def test_compare_tefficiencies_same(self):
        """
        Test if two TEfficiency objects can be compared. Is a bit tricky as
        TEfficiency does not support Comparing the exact same TEfficiency
        object should give back 100% agreement
        """
 
        c = validationcomparison.Chi2Test(self.teffA, self.teffA)
        self.assertTrue(c.can_compare())
        c.ensure_compute()
        self.assertAlmostEqual(c._pvalue, 1.0)
        self.assertAlmostEqual(c._chi2, 0.0)
        self.assertAlmostEqual(c._chi2ndf, 0.0)
        self.assertEqual(c._ndf, 18)
 

test_compare_unsupported_object()

def test_compare_unsupported_object

(

self

)

Test if unsupported ROOT objects are treated properly

Definition at line 273 of file test_comparison.py.

    def test_compare_unsupported_object(self):
        """
        Test if unsupported ROOT objects are treated properly
        """
        obj_not_supported = ROOT.TH2D("h2d", "h2d", 50, 0, 50, 50, 0, 50)
        c = validationcomparison.Chi2Test(self.profileA, obj_not_supported)
        self.assertFalse(c.can_compare())
 

test_compare_zero_error_profiles()

def test_compare_zero_error_profiles

(

self

)

Test if bins with zero error are treated properly

Definition at line 234 of file test_comparison.py.

    def test_compare_zero_error_profiles(self):
        """
        Test if bins with zero error are treated properly
        """
 
        # add entry in the last bin, which will increase the bin weight,
        # but leave the error at zero
        self.profileZeroErrorBins.Fill(49.8, 1.0)
        self.profileZeroErrorBins.Fill(49.8, 1.0)
 
        # the comparison should set the bin content of this bin to zero
        # to disable comparison of this bin instead of
        # not doing the comparison at all
        c = validationcomparison.Chi2Test(
            self.profileZeroErrorBins, self.profileZeroErrorBinsTwo
        )
 
        self.assertTrue(c.can_compare())
 
        c.ensure_compute()
 
        self.assertAlmostEqual(c._pvalue, 0.4835651485797353)
        # should still be only 49 ndf
        self.assertEqual(c._ndf, 49)
 

Member Data Documentation

call_iteration

call_iteration

if we would at some point later want to implement several runs, we use this as a counter variable to set up different names.

However not implemented yet.

Definition at line 134 of file test_comparison.py.

histogramA

histogramA

Histogram A.

Definition at line 160 of file test_comparison.py.

histogramB

histogramB

Histogram B (should be almost equal to profile A)

Definition at line 165 of file test_comparison.py.

profileA

profileA

Profile A.

Definition at line 137 of file test_comparison.py.

profileAequal

profileAequal

Profile should be almost equal to A.

Definition at line 170 of file test_comparison.py.

profileB

profileB

Profile B.

Definition at line 140 of file test_comparison.py.

profileBequal

profileBequal

Profile should be almost equal to B.

Definition at line 175 of file test_comparison.py.

profileC

profileC

Profile C.

Definition at line 143 of file test_comparison.py.

profileDifferentBins

profileDifferentBins

Profile with different bins.

Definition at line 180 of file test_comparison.py.

profileZeroErrorBins

profileZeroErrorBins

Profile with bins with 0 error.

Definition at line 148 of file test_comparison.py.

profileZeroErrorBinsTwo

profileZeroErrorBinsTwo

Profile with bins with 0 error.

Definition at line 154 of file test_comparison.py.

teffA

teffA

TEfficiemcy A.

Definition at line 189 of file test_comparison.py.

teffB

teffB

TEfficiency B.

Definition at line 192 of file test_comparison.py.

---

The documentation for this class was generated from the following file:

• validation/tests/test_comparison.py