TensorInitializers
The TensorInitializers utility object for tensor initializations commonly used in neural networks. Provides methods to create tensors filled with zeros, ones, random values, and standardized initialization schemes like He and Xavier initialization. All returned tensors have batch-first shape: (batch, rows, cols).
Attributes
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
TensorInitializers.type
Members list
Value members
Concrete methods
Stack a sequence of matrices into a TensorD with batch dimension. Each matrix becomes one slice: resulting shape = (batch, rows, cols).
Stack a sequence of matrices into a TensorD with batch dimension. Each matrix becomes one slice: resulting shape = (batch, rows, cols).
Attributes
Convert a MatrixD to a TensorD with shape (1, rows, cols).
Convert a MatrixD to a TensorD with shape (1, rows, cols).
Attributes
He initialization (Kaiming initialization). Standard deviation: sqrt(2 / fanIn), where fanIn = number of input features.
He initialization (Kaiming initialization). Standard deviation: sqrt(2 / fanIn), where fanIn = number of input features.
Attributes
LeCun uniform initialization. Samples from U(± sqrt(3 / fanIn)). Suitable for tanh activations.
LeCun uniform initialization. Samples from U(± sqrt(3 / fanIn)). Suitable for tanh activations.
Value parameters
- batch
-
the number of tensor slices (default 1)
- fanIn
-
the number of input features (cols)
- fanOut
-
the number of output features (rows)
Attributes
- Returns
-
a TensorD initialized using LeCun uniform initialization
Create a tensor of ones with shape (batch, rows, cols).
Create a tensor of ones with shape (batch, rows, cols).
Attributes
Create a tensor with random values from a uniform distribution N(0, 1).
Create a tensor with random values from a uniform distribution N(0, 1).
Attributes
Create a tensor with random values from a normal distribution N(0, stdDev^2).
Create a tensor with random values from a normal distribution N(0, stdDev^2).
Attributes
PyTorch-style RNN parameter initialization. Samples from U(± 1/sqrt(hiddenSize)) for all recurrent parameters.
PyTorch-style RNN parameter initialization. Samples from U(± 1/sqrt(hiddenSize)) for all recurrent parameters.
Value parameters
- batch
-
the number of tensor slices (default 1)
- cols
-
the number of columns per slice
- hiddenSize
-
size of the hidden state used to compute bound
- rows
-
the number of rows per slice
Attributes
- Returns
-
a TensorD initialized using RNN uniform initialization
Uniform initializer: samples values from U(-bound, bound).
Uniform initializer: samples values from U(-bound, bound).
Value parameters
- batch
-
number of tensor slices
- bound
-
half-width of uniform interval
- cols
-
number of columns per slice
- rows
-
number of rows per slice
Attributes
- Returns
-
a TensorD with uniform random values in [-bound, bound]
Xavier initialization (Glorot initialization). Standard deviation: sqrt(2 / (fanIn + fanOut)).
Xavier initialization (Glorot initialization). Standard deviation: sqrt(2 / (fanIn + fanOut)).
Attributes
Xavier/Glorot uniform initialization. Samples from U(± sqrt(6 / (fanIn + fanOut))).
Xavier/Glorot uniform initialization. Samples from U(± sqrt(6 / (fanIn + fanOut))).
Value parameters
- batch
-
the number of tensor slices (default 1)
- fanIn
-
the number of input features (cols)
- fanOut
-
the number of output features (rows)
Attributes
- Returns
-
a TensorD initialized using Xavier uniform initialization