SNEmbedding Class Reference

Inheritance diagram for SNEmbedding:

Public Member Functions
	__init__ (self, num_embeddings, embedding_dim, padding_idx=None, max_norm=None, norm_type=2, scale_grad_by_freq=False, sparse=False, _weight=None, num_svs=1, num_itrs=1, eps=1e-12)
	Constructor.
	forward (self, x)
	forward
	u (self)
	sv (self)
	W_ (self)

Public Attributes
	num_itrs = num_itrs
	Number of power iterations per step.
	num_svs = num_svs
	Number of singular values.
	transpose = transpose
	Transposed?
	eps = eps
	Epsilon value for avoiding divide-by-0.
bool	training
	Training mode flag (inherited from nn.Module).

Detailed Description

Embedding layer with spectral norm
We use num_embeddings as the dim instead of embedding_dim here
for convenience sake

Definition at line 827 of file ieagan.py.

Constructor & Destructor Documentation

__init__()

__init__	(		self,
			num_embeddings,
			embedding_dim,
			padding_idx = None,
			max_norm = None,
			norm_type = 2,
			scale_grad_by_freq = False,
			sparse = False,
			_weight = None,
			num_svs = 1,
			num_itrs = 1,
			eps = 1e-12 )

Constructor.

Definition at line 835 of file ieagan.py.

    ):
        nn.Embedding.__init__(
            self,
            num_embeddings,
            embedding_dim,
            padding_idx,
            max_norm,
            norm_type,
            scale_grad_by_freq,
            sparse,
            _weight,
        )
        SN.__init__(self, num_svs, num_itrs, num_embeddings, eps=eps)
 

Member Function Documentation

forward()

forward	(		self,
			x )

forward

Definition at line 863 of file ieagan.py.

    def forward(self, x):
        return F.embedding(x, self.W_())
 
 

sv()

sv ( self )

inherited

Singular values
note that these buffers are just for logging and are not used in training.

Definition at line 271 of file ieagan.py.

    def sv(self):
        """
        Singular values
        note that these buffers are just for logging and are not used in training.
        """
        return [getattr(self, f"sv{i:d}") for i in range(self.num_svs)]
 

u()

u ( self )

inherited

Singular vectors (u side)

Definition at line 264 of file ieagan.py.

    def u(self):
        """
        Singular vectors (u side)
        """
        return [getattr(self, f"u{i:d}") for i in range(self.num_svs)]
 

W_()

W_ ( self )

inherited

Compute the spectrally-normalized weight

Definition at line 278 of file ieagan.py.

    def W_(self):
        """
        Compute the spectrally-normalized weight
        """
        W_mat = self.weight.view(self.weight.size(0), -1)
        if self.transpose:
            W_mat = W_mat.t()
        # Apply num_itrs power iterations
        for _ in range(self.num_itrs):
            svs, _, _ = power_iteration(
                W_mat, self.u, update=self.training, eps=self.eps
            )
            # Update the svs
        if self.training:
            # Make sure to do this in a no_grad() context or you'll get memory leaks!  # noqa
            with torch.no_grad():
                for i, sv in enumerate(svs):
                    self.sv[i][:] = sv
        return self.weight / svs[0]
 
 

Member Data Documentation

eps

eps = eps

inherited

Epsilon value for avoiding divide-by-0.

Definition at line 255 of file ieagan.py.

num_itrs

num_itrs = num_itrs

inherited

Number of power iterations per step.

Definition at line 249 of file ieagan.py.

num_svs

num_svs = num_svs

inherited

Number of singular values.

Definition at line 251 of file ieagan.py.

training

bool training

inherited

Training mode flag (inherited from nn.Module).

True if the module is in training mode.

Definition at line 261 of file ieagan.py.

transpose

transpose = transpose

inherited

Transposed?

Definition at line 253 of file ieagan.py.

---

The documentation for this class was generated from the following file:

• pxd/scripts/pxd/background_generator/models/ieagan.py