SNEmbedding Class Reference

Inheritance diagram for SNEmbedding:

Public Member Functions
	__init__ (self, num_embeddings, embedding_dim, padding_idx=None, max_norm=None, norm_type=2, scale_grad_by_freq=False, sparse=False, _weight=None, num_svs=1, num_itrs=1, eps=1e-12)
	Constructor.
	forward (self, x)
	forward
	u (self)
	sv (self)
	W_ (self)

Public Attributes
	num_itrs = num_itrs
	Number of power iterations per step.
	num_svs = num_svs
	Number of singular values.
	transpose = transpose
	Transposed?
	eps = eps
	Epsilon value for avoiding divide-by-0.
	training

Detailed Description

Embedding layer with spectral norm
We use num_embeddings as the dim instead of embedding_dim here
for convenience sake

Definition at line 799 of file ieagan.py.

Constructor & Destructor Documentation

__init__()

__init__	(		self,
			num_embeddings,
			embedding_dim,
			padding_idx = None,
			max_norm = None,
			norm_type = 2,
			scale_grad_by_freq = False,
			sparse = False,
			_weight = None,
			num_svs = 1,
			num_itrs = 1,
			eps = 1e-12 )

Constructor.

Definition at line 807 of file ieagan.py.

    ):
        nn.Embedding.__init__(
            self,
            num_embeddings,
            embedding_dim,
            padding_idx,
            max_norm,
            norm_type,
            scale_grad_by_freq,
            sparse,
            _weight,
        )
        SN.__init__(self, num_svs, num_itrs, num_embeddings, eps=eps)
 

Member Function Documentation

forward()

forward	(		self,
			x )

forward

Definition at line 835 of file ieagan.py.

    def forward(self, x):
        return F.embedding(x, self.W_())
 
 

sv()

sv ( self )

inherited

Singular values
note that these buffers are just for logging and are not used in training.

Definition at line 256 of file ieagan.py.

    def sv(self):
        """
        Singular values
        note that these buffers are just for logging and are not used in training.
        """
        return [getattr(self, f"sv{i:d}") for i in range(self.num_svs)]
 

u()

u ( self )

inherited

Singular vectors (u side)

Definition at line 249 of file ieagan.py.

    def u(self):
        """
        Singular vectors (u side)
        """
        return [getattr(self, f"u{i:d}") for i in range(self.num_svs)]
 

W_()

W_ ( self )

inherited

Compute the spectrally-normalized weight

Definition at line 263 of file ieagan.py.

    def W_(self):
        """
        Compute the spectrally-normalized weight
        """
        W_mat = self.weight.view(self.weight.size(0), -1)
        if self.transpose:
            W_mat = W_mat.t()
        # Apply num_itrs power iterations
        for _ in range(self.num_itrs):
            svs, _, _ = power_iteration(
                W_mat, self.u, update=self.training, eps=self.eps
            )
            # Update the svs
        if self.training:
            # Make sure to do this in a no_grad() context or you'll get memory leaks!  # noqa
            with torch.no_grad():
                for i, sv in enumerate(svs):
                    self.sv[i][:] = sv
        return self.weight / svs[0]
 
 

Member Data Documentation

eps

eps = eps

inherited

Epsilon value for avoiding divide-by-0.

Definition at line 242 of file ieagan.py.

num_itrs

num_itrs = num_itrs

inherited

Number of power iterations per step.

Definition at line 236 of file ieagan.py.

num_svs

num_svs = num_svs

inherited

Number of singular values.

Definition at line 238 of file ieagan.py.

training

training

inherited

Initial value:

=  power_iteration(
                W_mat, self.u, update=self.training, eps=self.eps
            )

Definition at line 276 of file ieagan.py.

transpose

transpose = transpose

inherited

Transposed?

Definition at line 240 of file ieagan.py.

---

The documentation for this class was generated from the following file:

• pxd/scripts/pxd/background_generator/models/ieagan.py