SNLinear Class Reference

Inheritance diagram for SNLinear:

Public Member Functions
	__init__ (self, in_features, out_features, bias=True, num_svs=1, num_itrs=1, eps=1e-12)
	Constructor.
	forward (self, x)
	forward
	u (self)
	sv (self)
	W_ (self)

Public Attributes
	bias
	num_itrs = num_itrs
	Number of power iterations per step.
	num_svs = num_svs
	Number of singular values.
	transpose = transpose
	Transposed?
	eps = eps
	Epsilon value for avoiding divide-by-0.
	training

Detailed Description

Linear layer with spectral norm

Definition at line 330 of file ieagan.py.

Constructor & Destructor Documentation

__init__()

__init__	(		self,
			in_features,
			out_features,
			bias = True,
			num_svs = 1,
			num_itrs = 1,
			eps = 1e-12 )

Constructor.

Definition at line 336 of file ieagan.py.

    ):
        nn.Linear.__init__(self, in_features, out_features, bias)
        SN.__init__(self, num_svs, num_itrs, out_features, eps=eps)
 

Member Function Documentation

forward()

forward	(		self,
			x )

forward

Definition at line 349 of file ieagan.py.

    def forward(self, x):
        return F.linear(x, self.W_(), self.bias)
 
 

sv()

sv ( self )

inherited

Singular values
note that these buffers are just for logging and are not used in training.

Definition at line 256 of file ieagan.py.

    def sv(self):
        """
        Singular values
        note that these buffers are just for logging and are not used in training.
        """
        return [getattr(self, f"sv{i:d}") for i in range(self.num_svs)]
 

u()

u ( self )

inherited

Singular vectors (u side)

Definition at line 249 of file ieagan.py.

    def u(self):
        """
        Singular vectors (u side)
        """
        return [getattr(self, f"u{i:d}") for i in range(self.num_svs)]
 

W_()

W_ ( self )

inherited

Compute the spectrally-normalized weight

Definition at line 263 of file ieagan.py.

    def W_(self):
        """
        Compute the spectrally-normalized weight
        """
        W_mat = self.weight.view(self.weight.size(0), -1)
        if self.transpose:
            W_mat = W_mat.t()
        # Apply num_itrs power iterations
        for _ in range(self.num_itrs):
            svs, _, _ = power_iteration(
                W_mat, self.u, update=self.training, eps=self.eps
            )
            # Update the svs
        if self.training:
            # Make sure to do this in a no_grad() context or you'll get memory leaks!  # noqa
            with torch.no_grad():
                for i, sv in enumerate(svs):
                    self.sv[i][:] = sv
        return self.weight / svs[0]
 
 

Member Data Documentation

bias

bias

Definition at line 350 of file ieagan.py.

eps

eps = eps

inherited

Epsilon value for avoiding divide-by-0.

Definition at line 242 of file ieagan.py.

num_itrs

num_itrs = num_itrs

inherited

Number of power iterations per step.

Definition at line 236 of file ieagan.py.

num_svs

num_svs = num_svs

inherited

Number of singular values.

Definition at line 238 of file ieagan.py.

training

training

inherited

Initial value:

=  power_iteration(
                W_mat, self.u, update=self.training, eps=self.eps
            )

Definition at line 276 of file ieagan.py.

transpose

transpose = transpose

inherited

Transposed?

Definition at line 240 of file ieagan.py.

---

The documentation for this class was generated from the following file:

• pxd/scripts/pxd/background_generator/models/ieagan.py