RecurrentBase
The RecurrentBase trait defines the base structure and operations for recurrent neural networks. It includes common hyperparameters, activation functions, and methods for parameter initialization, gradient clipping, and parameter updates.
Attributes
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
Members list
Type members
Classlikes
Case class representing a group of parameters, including the parameter itself, its velocity, and its gradient.
Case class representing a group of parameters, including the parameter itself, its velocity, and its gradient.
Value parameters
- grad
-
the gradient of the parameter, used for updating the parameter during training
- param
-
the parameter (e.g., weights or biases)
- velocity
-
the velocity associated with the parameter, used for momentum in optimization
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Value members
Abstract methods
Abstract method to clip the gradients of the model parameters to a specified threshold.
Abstract method to clip the gradients of the model parameters to a specified threshold.
Value parameters
- threshold
-
the threshold value for gradient clipping
Attributes
Abstract method for training the model with the given input and output tensors.
Abstract method for training the model with the given input and output tensors.
Value parameters
- x
-
the input tensor
- y
-
the output tensor
Attributes
Abstract method to update the model parameters based on the gradients.
Abstract method to update the model parameters based on the gradients.
Value parameters
- batch_size
-
the size of the batch used for training
- leaky
-
a boolean flag indicating whether to use leaky updates (default is true)
Attributes
Abstract method to zero the gradients of the model parameters.
Abstract method to zero the gradients of the model parameters.
Attributes
Concrete methods
Initialize a bias group with a specified size.
Initialize a bias group with a specified size.
Value parameters
- size
-
the size of the bias vector
Attributes
- Returns
-
A ParamGroup containing the initialized bias vector, velocity vector, and gradient vector
Initialize a parameter group with a specified number of rows, columns, and standard deviation.
Initialize a parameter group with a specified number of rows, columns, and standard deviation.
Value parameters
- cols
-
the number of columns in the parameter matrix
- rows
-
the number of rows in the parameter matrix
- stdDev
-
the standard deviation for initializing the parameter matrix
Attributes
- Returns
-
A ParamGroup containing the initialized parameter matrix, velocity matrix, and gradient matrix
Updates a specific batch of rows in a matrix with new values.
Updates a specific batch of rows in a matrix with new values.
Value parameters
- batch_end
-
the ending index (exclusive) of the batch in the matrix
- batch_start
-
the starting index of the batch in the matrix
- matrix
-
the matrix to be updated
- newBatch
-
the new matrix containing the values to be inserted
Attributes
- Throws
-
IllegalArgumentException
If the size of newBatch doesn't match the specified batch size