myBN Class Reference

Inheritance diagram for myBN:

Public Member Functions
def	__init__ (self, num_channels, eps=1e-5, momentum=0.1)
	Constructor.
def	reset_stats (self)
	reset standing stats
def	forward (self, x, gain, bias)
	forward

Public Attributes
	momentum
	momentum for updating running stats
	eps
	epsilon to avoid dividing by 0
	accumulate_standing
	Accumulate running means and vars.

Detailed Description

My batchnorm, supports standing stats

Definition at line 400 of file ieagan.py.

Constructor & Destructor Documentation

__init__()

def __init__	(		self,
			num_channels,
			eps = 1e-5,
			momentum = 0.1
	)

Constructor.

Definition at line 406 of file ieagan.py.

    def __init__(self, num_channels, eps=1e-5, momentum=0.1):
        super(myBN, self).__init__()
        
        self.momentum = momentum
        
        self.eps = eps
        # Momentum
        self.momentum = momentum
        # Register buffers
        self.register_buffer("stored_mean", torch.zeros(num_channels))
        self.register_buffer("stored_var", torch.ones(num_channels))
        self.register_buffer("accumulation_counter", torch.zeros(1))
        
        self.accumulate_standing = False
 

Member Function Documentation

forward()

def forward	(		self,
			x,
			gain,
			bias
	)

forward

Definition at line 429 of file ieagan.py.

    def forward(self, x, gain, bias):
        # pylint: disable=no-member
        if self.training:
            out, mean, var = manual_bn(
                x, gain, bias, return_mean_var=True, eps=self.eps
            )
            # If accumulating standing stats, increment them
            if self.accumulate_standing:
                self.stored_mean[:] = self.stored_mean + mean.data
                self.stored_var[:] = self.stored_var + var.data
                self.accumulation_counter += 1.0
            # If not accumulating standing stats, take running averages
            else:
                self.stored_mean[:] = (
                    self.stored_mean * (1 - self.momentum) + mean * self.momentum
                )
                self.stored_var[:] = (
                    self.stored_var * (1 - self.momentum) + var * self.momentum
                )
            return out
        # If not in training mode, use the stored statistics
        else:
            mean = self.stored_mean.view(1, -1, 1, 1)
            var = self.stored_var.view(1, -1, 1, 1)
            # If using standing stats, divide them by the accumulation counter
            if self.accumulate_standing:
                mean = mean / self.accumulation_counter
                var = var / self.accumulation_counter
            return fused_bn(x, mean, var, gain, bias, self.eps)
 
 

reset_stats()

def reset_stats

(

self

)

reset standing stats

Definition at line 422 of file ieagan.py.

    def reset_stats(self):
        # pylint: disable=no-member
        self.stored_mean[:] = 0
        self.stored_var[:] = 0
        self.accumulation_counter[:] = 0
 

Member Data Documentation

accumulate_standing

accumulate_standing

Accumulate running means and vars.

Definition at line 419 of file ieagan.py.

eps

eps

epsilon to avoid dividing by 0

Definition at line 411 of file ieagan.py.

momentum

momentum

momentum for updating running stats

Definition at line 409 of file ieagan.py.

---

The documentation for this class was generated from the following file:

• pxd/scripts/pxd/background_generator/models/ieagan.py