LayerNorm
The LayerNorm class implements Layer Normalization as described in: "Layer Normalization" by Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton
Value parameters
- dModel
-
the number of features in the input
- eps
-
a small value to avoid division by zero
- ops
-
the autograd operations
Attributes
- See also
- Graph
-
- Supertypes
Members list
Value members
Concrete methods
Forward pass for a single input Variable. Must be implemented by subclasses.
Forward pass for a single input Variable. Must be implemented by subclasses.
Attributes
- Definition Classes
Return all trainable parameters, including those from submodules.
Return all trainable parameters, including those from submodules.
Attributes
- Definition Classes
Inherited methods
Alias for forward, allows calling the module as a function: module(x).
Alias for forward, allows calling the module as a function: module(x).
Attributes
- Inherited from:
- Module
Set the module to evaluation mode (and all submodules recursively).
Set the module to evaluation mode (and all submodules recursively).
Attributes
- Inherited from:
- BaseModule
Return the gradients of all parameters.
Replace the current parameters with new ones. Useful for weight updates, loading saved models, etc.
Replace the current parameters with new ones. Useful for weight updates, loading saved models, etc.
Value parameters
- newParams
-
The new parameter list to assign
Attributes
- Inherited from:
- BaseModule
Set the module to training mode (and all submodules recursively).
Set the module to training mode (and all submodules recursively).
Attributes
- Inherited from:
- BaseModule
Zero out all gradients (in-place).
Inherited fields
Flag to control training or evaluation behavior.
Automatically detect submodules (other BaseModules) within this module.
Automatically detect submodules (other BaseModules) within this module.
Attributes
- Inherited from:
- BaseModule