Overtraining Class Reference

Inheritance diagram for Overtraining:

Public Member Functions
def	__init__ (self, figure=None)
def	add (self, data, column, train_mask, test_mask, signal_mask, bckgrd_mask, weight_column=None)
def	finish (self)

Public Attributes
	figure
	create figure
	axis
	define first subplot
	axis_d1
	define second subplot
	axis_d2
	define third subplot

Static Public Attributes
None	figure = None
	figure which is used to draw
None	axis = None
	Main axis which is used to draw.
None	axis_d1 = None
	Axis which shows the difference between training and test signal.
None	axis_d2 = None
	Axis which shows the difference between training and test background.

Detailed Description

Create TMVA-like overtraining control plot for a classification training

Definition at line 859 of file plotting.py.

Constructor & Destructor Documentation

__init__()

def __init__	(		self,
			figure = None
	)

Creates a new figure if None is given, sets the default plot parameters
@param figure default draw figure which is used

Reimplemented from Plotter.

Definition at line 873 of file plotting.py.

    def __init__(self, figure=None):
        """
        Creates a new figure if None is given, sets the default plot parameters
        @param figure default draw figure which is used
        """
        if figure is None:
            
            self.figure = matplotlib.figure.Figure(figsize=(12, 8), dpi=120)
            self.figure.set_tight_layout(True)
        else:
            self.figure = figure
 
        gs = matplotlib.gridspec.GridSpec(5, 1)
        
        self.axis = self.figure.add_subplot(gs[:3, :])
        
        self.axis_d1 = self.figure.add_subplot(gs[3, :], sharex=self.axis)
        
        self.axis_d2 = self.figure.add_subplot(gs[4, :], sharex=self.axis)
 
        super().__init__(self.figure, self.axis)
 

Member Function Documentation

add()

def add	(		self,
			data,
			column,
			train_mask,
			test_mask,
			signal_mask,
			bckgrd_mask,
			weight_column = None
	)

Add a new overtraining plot, I recommend to draw only one overtraining plot at the time,
otherwise there are too many curves in the plot to recognize anything in the plot.
@param data pandas.DataFrame containing all data
@param column which is used to calculate distribution histogram
@param train_mask boolean numpy.array defining which events are training events
@param test_mask boolean numpy.array defining which events are test events
@param signal_mask boolean numpy.array defining which events are signal events
@param bckgrd_mask boolean numpy.array defining which events are background events
@param weight_column column in data containing the weights for each event

Reimplemented from Plotter.

Definition at line 895 of file plotting.py.

    def add(self, data, column, train_mask, test_mask, signal_mask, bckgrd_mask, weight_column=None):
        """
        Add a new overtraining plot, I recommend to draw only one overtraining plot at the time,
        otherwise there are too many curves in the plot to recognize anything in the plot.
        @param data pandas.DataFrame containing all data
        @param column which is used to calculate distribution histogram
        @param train_mask boolean numpy.array defining which events are training events
        @param test_mask boolean numpy.array defining which events are test events
        @param signal_mask boolean numpy.array defining which events are signal events
        @param bckgrd_mask boolean numpy.array defining which events are background events
        @param weight_column column in data containing the weights for each event
        """
        distribution = Distribution(self.figure, self.axis, normed_to_all_entries=True)
 
        distribution.set_plot_options(self.plot_kwargs)
        distribution.set_errorbar_options(self.errorbar_kwargs)
        distribution.set_errorband_options(self.errorband_kwargs)
        distribution.add(data, column, test_mask & signal_mask, weight_column)
        distribution.add(data, column, test_mask & bckgrd_mask, weight_column)
 
        distribution.set_plot_options(
            {'color': distribution.plots[0][0][0].get_color(), 'linestyle': '-', 'lw': 4, 'drawstyle': 'steps-mid'})
        distribution.set_fill_options({'color': distribution.plots[0][0][0].get_color(), 'alpha': 0.5, 'step': 'post'})
        distribution.set_errorbar_options(None)
        distribution.set_errorband_options(None)
        distribution.add(data, column, train_mask & signal_mask, weight_column)
        distribution.set_plot_options(
            {'color': distribution.plots[1][0][0].get_color(), 'linestyle': '-', 'lw': 4, 'drawstyle': 'steps-mid'})
        distribution.set_fill_options({'color': distribution.plots[1][0][0].get_color(), 'alpha': 0.5, 'step': 'post'})
        distribution.add(data, column, train_mask & bckgrd_mask, weight_column)
 
        distribution.labels = ['Test-Signal', 'Test-Background', 'Train-Signal', 'Train-Background']
        distribution.finish()
 
        self.plot_kwargs['color'] = distribution.plots[0][0][0].get_color()
        difference_signal = Difference(self.figure, self.axis_d1, shift_to_zero=True, normed=True)
        difference_signal.set_plot_options(self.plot_kwargs)
        difference_signal.set_errorbar_options(self.errorbar_kwargs)
        difference_signal.set_errorband_options(self.errorband_kwargs)
        difference_signal.add(data, column, train_mask & signal_mask, test_mask & signal_mask, weight_column)
        self.axis_d1.set_xlim((difference_signal.xmin, difference_signal.xmax))
        self.axis_d1.set_ylim((difference_signal.ymin, difference_signal.ymax))
        difference_signal.plots = difference_signal.labels = []
        difference_signal.finish(line_color=distribution.plots[0][0][0].get_color())
 
        self.plot_kwargs['color'] = distribution.plots[1][0][0].get_color()
        difference_bckgrd = Difference(self.figure, self.axis_d2, shift_to_zero=True, normed=True)
        difference_bckgrd.set_plot_options(self.plot_kwargs)
        difference_bckgrd.set_errorbar_options(self.errorbar_kwargs)
        difference_bckgrd.set_errorband_options(self.errorband_kwargs)
        difference_bckgrd.add(data, column, train_mask & bckgrd_mask, test_mask & bckgrd_mask, weight_column)
        self.axis_d2.set_xlim((difference_bckgrd.xmin, difference_bckgrd.xmax))
        self.axis_d2.set_ylim((difference_bckgrd.ymin, difference_bckgrd.ymax))
        difference_bckgrd.plots = difference_bckgrd.labels = []
        difference_bckgrd.finish(line_color=distribution.plots[1][0][0].get_color())
 
        try:
            import scipy.stats
            # Kolmogorov smirnov test
            if len(data[column][train_mask & signal_mask]) == 0 or len(data[column][test_mask & signal_mask]) == 0:
                b2.B2WARNING("Cannot calculate kolmogorov smirnov test for signal due to missing data")
            else:
                ks = scipy.stats.ks_2samp(data[column][train_mask & signal_mask], data[column][test_mask & signal_mask])
                props = dict(boxstyle='round', edgecolor='gray', facecolor='white', linewidth=0.1, alpha=0.5)
                self.axis_d1.text(0.1, 0.9, r'signal (train - test) difference $p={:.2f}$'.format(ks[1]),  bbox=props,
                                  verticalalignment='top', horizontalalignment='left', transform=self.axis_d1.transAxes)
            if len(data[column][train_mask & bckgrd_mask]) == 0 or len(data[column][test_mask & bckgrd_mask]) == 0:
                b2.B2WARNING("Cannot calculate kolmogorov smirnov test for background due to missing data")
            else:
                ks = scipy.stats.ks_2samp(data[column][train_mask & bckgrd_mask], data[column][test_mask & bckgrd_mask])
                props = dict(boxstyle='round', edgecolor='gray', facecolor='white', linewidth=0.1, alpha=0.5)
                self.axis_d2.text(0.1, 0.9, r'background (train - test) difference $p={:.2f}$'.format(ks[1]),
                                  bbox=props,
                                  verticalalignment='top', horizontalalignment='left', transform=self.axis_d2.transAxes)
        except ImportError:
            b2.B2WARNING("Cannot calculate kolmogorov smirnov test please install scipy!")
 
        return self
 

finish()

def finish

(

self

)

Sets limits, title, axis-labels and legend of the plot

Reimplemented from Plotter.

Definition at line 974 of file plotting.py.

    def finish(self):
        """
        Sets limits, title, axis-labels and legend of the plot
        """
        self.axis.set_title("Overtraining Plot")
        self.axis_d1.set_title("")
        self.axis_d2.set_title("")
        matplotlib.artist.setp(self.axis.get_xticklabels(), visible=False)
        matplotlib.artist.setp(self.axis_d1.get_xticklabels(), visible=False)
        self.axis.get_xaxis().set_label_text('')
        self.axis_d1.get_xaxis().set_label_text('')
        self.axis_d2.get_xaxis().set_label_text('Classifier Output')
        return self
 
 

Member Data Documentation

axis [1/2]

None axis = None

static

Main axis which is used to draw.

Definition at line 867 of file plotting.py.

axis [2/2]

axis

define first subplot

Definition at line 887 of file plotting.py.

axis_d1 [1/2]

None axis_d1 = None

static

Axis which shows the difference between training and test signal.

Definition at line 869 of file plotting.py.

axis_d1 [2/2]

axis_d1

define second subplot

Definition at line 889 of file plotting.py.

axis_d2 [1/2]

None axis_d2 = None

static

Axis which shows the difference between training and test background.

Definition at line 871 of file plotting.py.

axis_d2 [2/2]

axis_d2

define third subplot

Definition at line 891 of file plotting.py.

figure [1/2]

None figure = None

static

figure which is used to draw

Definition at line 865 of file plotting.py.

figure [2/2]

figure

create figure

Definition at line 880 of file plotting.py.

---

The documentation for this class was generated from the following file:

• mva/scripts/basf2_mva_evaluation/plotting.py