scalation
API
Generated with
scalation/scalation/scalation.modeling/scalation.modeling.forecasting/scalation.modeling.forecasting.neuralforecasting

scalation.modeling.forecasting.neuralforecasting

package scalation.modeling.forecasting.neuralforecasting

Members list

Type members

Classlikes

trait Attention(n_var: Int, n_mod: Int = ..., heads: Int = ..., n_v: Int = ...)

The Attention trait provides methods for computing context vectors, single-head attention matrices and multi-head attention matrices.

The Attention trait provides methods for computing context vectors, single-head attention matrices and multi-head attention matrices.

Value parameters

heads

the number of attention heads

n_mod

the size of the output (dimensionality of the model, d_model)

n_v

the size of the value vectors

n_var

the size of the input vector x_t (number of variables)

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object Attention

The Attention object contains sample a input matrix from

The Attention object contains sample a input matrix from

Attributes

See also

https://sebastianraschka.com/blog/2023/self-attention-from-scratch.html The example is from 6 words with 16 dimensional encoding.

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Attention.type
case class DenseLayer(n_x: Int, n_y: Int, f: AFF = ...)

The DenseLayer class applies an (optionally activated) linear transformation to the input matrix X. Yp = f(X W + b) When f is null, it acts as a Linear Layer.

The DenseLayer class applies an (optionally activated) linear transformation to the input matrix X. Yp = f(X W + b) When f is null, it acts as a Linear Layer.

Value parameters

f

the activation function family for layers 1->2 (input to output)

n_x

the second dimension of the input matrix (m by n_x)

n_y

the second dimension of the output matrix (m by n_y)

Attributes

See also

pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
case class DropoutLayer(p: Double = ...)

The DropoutLayer class will, in computing the output, set each element to zero with probability p; otherwise, multiply it by a scale factor.

The DropoutLayer class will, in computing the output, set each element to zero with probability p; otherwise, multiply it by a scale factor.

Value parameters

p

the probability of setting an element to zero

Attributes

See also

pytorch.org/docs/stable/generated/torch.nn.Dropout.html#torch.nn.Dropout

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
class GRU(x: MatrixD, y: MatrixD, fname: Array[String] = ..., n_mem: Int = ...)

The GRU class implements Gated Recurrent Unit (GRU) via Back Propagation Through Time (BPTT). At each time point x_t, there is a vector representing several variables or the encoding of a word. Intended to work for guessing the next work in a sentence or for multi-horizon forecasting. Time series: (x_t: t = 0, 1, ..., n_seq-1) where n_seq is the number of time points/words

The GRU class implements Gated Recurrent Unit (GRU) via Back Propagation Through Time (BPTT). At each time point x_t, there is a vector representing several variables or the encoding of a word. Intended to work for guessing the next work in a sentence or for multi-horizon forecasting. Time series: (x_t: t = 0, 1, ..., n_seq-1) where n_seq is the number of time points/words

Value parameters

fname

the feature/variable names

n_mem

the size for hidden state (h) (dimensionality of memory)

x

the input sequence/time series

y

the output sequence/time series

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object GRU

The GRU companion object provides factory methods.

The GRU companion object provides factory methods.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
GRU.type
case class Gate(n_seq: Int, n_mem: Int, n_var: Int)

The Gate case class holds information on the gate's value and its partial derivatives.

The Gate case class holds information on the gate's value and its partial derivatives.

Value parameters

n_mem

the size for hidden state (h) (dimensionality of memory)

n_seq

the length of the time series

n_var

the number of variables

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
class LSTM(x: MatrixD, y: MatrixD, fname: Array[String] = ..., n_mem: Int = ...)

The LSTM class implements Long Short-Term Memeory (LSTM) via Back Propagation Through Time (BPTT). At each time point x_t, there is a vector representing several variables or the encoding of a word. Intended to work for guessing the next work in a sentence or for multi-horizon forecasting. Time series: (x_t: t = 0, 1, ..., n_seq-1) where n_seq is the number of time points/words

The LSTM class implements Long Short-Term Memeory (LSTM) via Back Propagation Through Time (BPTT). At each time point x_t, there is a vector representing several variables or the encoding of a word. Intended to work for guessing the next work in a sentence or for multi-horizon forecasting. Time series: (x_t: t = 0, 1, ..., n_seq-1) where n_seq is the number of time points/words

Value parameters

fname

the feature/variable names

n_mem

the size for hidden state (h) (dimensionality of memory)

x

the input sequence/time series

y

the output sequence/time series

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object LSTM

The LSTM companion object provides factory methods.

The LSTM companion object provides factory methods.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
LSTM.type
case class LayerNorm(atransform: Boolean = ..., eps: Double = ...)

The LayerNorm class will, in computing the output, normalize by subtracting the mean and dividing by the standard deviation.

The LayerNorm class will, in computing the output, normalize by subtracting the mean and dividing by the standard deviation.

Value parameters

atransform

whether to apply an affine transformation to standard normalization

eps

the small value to prevent division by zero

Attributes

See also

pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html#torch.nn.LayerNorm

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object NeuralNet_3L4TS extends Scaling

The NeuralNet_3L4TS object supports 3-layer regression-like neural networks for Time Series data. Given a response vector y, a predictor matrix x is built that consists of lagged y vectors.

The NeuralNet_3L4TS object supports 3-layer regression-like neural networks for Time Series data. Given a response vector y, a predictor matrix x is built that consists of lagged y vectors.

Attributes

Companion
class
Supertypes
trait Scaling
class Object
trait Matchable
class Any
Self type
class NeuralNet_3L4TS(x: MatrixD, y: MatrixD, hh: Int, n_exo: Int, fname: Array[String] = ..., tRng: Range = ..., nz: Int = ..., hparam: HyperParameter = ..., f: AFF = ..., f1: AFF = ..., val itran: FunctionV2V = ..., bakcast: Boolean = ...) extends Forecaster_D

The NeuralNet_3L4TS class provides basic time series analysis capabilities for three layer neural network models that use DIRECT (as opposed to RECURSIVE) multi-horizon forecasting. Given time series data stored in vector y, its next value y_t = combination of last p values.

The NeuralNet_3L4TS class provides basic time series analysis capabilities for three layer neural network models that use DIRECT (as opposed to RECURSIVE) multi-horizon forecasting. Given time series data stored in vector y, its next value y_t = combination of last p values.

y_t = f1(bb dot f(aa dot x_t)) + e_t

where y_t is the value of y at time t and e_t is the residual/error term.

Value parameters

bakcast

whether a backcasted value is prepended to the time series (defaults to false)

f

the activation function family for layers 1->2 (input to hidden)

f1

the activation function family for layers 2->3 (hidden to output)

fname

the feature/variable names

hh

the maximum forecasting horizon (h = 1 to hh)

hparam

the hyper-parameters (defaults to MakeMatrix4TS.hp ++ Optimizer.hp)

itran

the inverse transformation function returns response matrix to original scale

n_exo

the number of exogenous variables

nz

the number of nodes in hidden layer (-1 => use default formula)

tRng

the time range, if relevant (time index may suffice)

x

the data/input matrix (lagged columns of y and ex) @see NeuralNet_3L4TS.apply

y

the response/output matrix (column per horizon) (time series data)

Attributes

Companion
object
Supertypes
class Forecaster_D
class Forecaster
trait Forecast
trait Model
class Diagnoser
trait Fit
trait FitM
class Object
trait Matchable
class Any
Show all
trait PositionalEnc(m: Int, d: Int = ...)

The PositionalEnc trait provides methods to convert a time t into an encoded vector. An encoded vector consists of numbers in [-1.0, 1.0]. It implements Absolute Fixed Vanilla Positional Encoding.

The PositionalEnc trait provides methods to convert a time t into an encoded vector. An encoded vector consists of numbers in [-1.0, 1.0]. It implements Absolute Fixed Vanilla Positional Encoding.

Value parameters

d

the dimensionality of the positional encoding (except for f0)

m

the length of the time series (number of time points)

Attributes

Supertypes
class Object
trait Matchable
class Any
case class RMSNorm()

The RMSNorm class will, in computing the output, normalize by dividing by the Root Mean Square (RMS).

The RMSNorm class will, in computing the output, normalize by dividing by the Root Mean Square (RMS).

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
class RNN(val x: TensorD, val y: TensorD, val y_orig: MatrixD, fname: Array[String] = ..., val n_mem: Int = ...) extends RNNCell, FitM

The RNN class implements Recurrent Neural Network (RNN) via Back Propagation Through Time (BPTT). At each time point x_t, there is a vector representing several variables or the encoding of a word. Intended to work for guessing the next work in a sentence or for multi-horizon forecasting.

The RNN class implements Recurrent Neural Network (RNN) via Back Propagation Through Time (BPTT). At each time point x_t, there is a vector representing several variables or the encoding of a word. Intended to work for guessing the next work in a sentence or for multi-horizon forecasting.

Value parameters

fname

the feature/variable names

n_mem

the size for hidden state (h) (dimensionality of memory)

x

the input sequence/time series

y

the output sequence/time series

y_orig

the original target matrix before any preprocessing

Attributes

Companion
object
Supertypes
trait FitM
trait RNNCell
class Object
trait Matchable
class Any
Show all
object RNN

The RNN companion object provides factory methods.

The RNN companion object provides factory methods.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
RNN.type
trait RNNCell extends RecurrentBase

The RNNCell trait defines the structure and operations for a Recurrent Neural Network (RNN) cell. It includes weight matrices, bias vectors, and hidden states, along with methods for gradient clipping and parameter updates.

The RNNCell trait defines the structure and operations for a Recurrent Neural Network (RNN) cell. It includes weight matrices, bias vectors, and hidden states, along with methods for gradient clipping and parameter updates.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class RNN
trait RecurrentBase

The RecurrentBase trait defines the base structure and operations for recurrent neural networks. It includes common hyperparameters, activation functions, and methods for parameter initialization, gradient clipping, and parameter updates.

The RecurrentBase trait defines the base structure and operations for recurrent neural networks. It includes common hyperparameters, activation functions, and methods for parameter initialization, gradient clipping, and parameter updates.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
trait RNNCell
class RNN
object SimpleEncoder

The SimpleEncoder object implements the attention method based on the scaled dot product.

The SimpleEncoder object implements the attention method based on the scaled dot product.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
class TrfEncoderLayer(x: MatrixD, heads: Int = ..., f: AFF = ..., initW: Array[MatrixD] = ..., p_drop: Double = ..., norm_eps: Double = ..., norm_first: Boolean = ...) extends Attention

The TrfEncoderLayer class consists of a Multi-Head Self-Attention and a Feed-Forward Neural Network (FFNN) sub-layers.

The TrfEncoderLayer class consists of a Multi-Head Self-Attention and a Feed-Forward Neural Network (FFNN) sub-layers.

Value parameters

f

the activation function family (used by alinear1)

heads

the number of attention heads (e.g., 1 to 8)

norm_eps

a small values used in normalization to avoid divide by zero

norm_first

whether layer normalization should be done first (see apply method)

p_drop

the probability of setting an element to zero in a dropout layer (e.g., .0 to .5)

x

the input data matrix after embedding (number of rows/instances by embedding dimension)

Attributes

See also

pytorch.org/docs/stable/generated/torch.nn.TransformerEncoderLayer.html#torch.nn.TransformerEncoderLayer

Supertypes
trait Attention
class Object
trait Matchable
class Any
final class attentionTest

Attributes

Supertypes
class Object
trait Matchable
class Any
final class attentionTest2

Attributes

Supertypes
class Object
trait Matchable
class Any
final class attentionTest3

Attributes

Supertypes
class Object
trait Matchable
class Any
final class attentionTest4

Attributes

Supertypes
class Object
trait Matchable
class Any
final class attentionTest5

Attributes

Supertypes
class Object
trait Matchable
class Any
final class gRUTest

Attributes

Supertypes
class Object
trait Matchable
class Any
final class gRUTest2

Attributes

Supertypes
class Object
trait Matchable
class Any
final class gRUTest3

Attributes

Supertypes
class Object
trait Matchable
class Any
final class lSTMTest

Attributes

Supertypes
class Object
trait Matchable
class Any
final class lSTMTest2

Attributes

Supertypes
class Object
trait Matchable
class Any
final class lSTMTest3

Attributes

Supertypes
class Object
trait Matchable
class Any
final class neuralNet_3L4TSTest

Attributes

Supertypes
class Object
trait Matchable
class Any
final class neuralNet_3L4TSTest2

Attributes

Supertypes
class Object
trait Matchable
class Any
final class neuralNet_3L4TSTest3

Attributes

Supertypes
class Object
trait Matchable
class Any
final class positionalEncTest

Attributes

Supertypes
class Object
trait Matchable
class Any
final class rNNTest4

Attributes

Supertypes
class Object
trait Matchable
class Any
final class simpleEncoder1

Attributes

Supertypes
class Object
trait Matchable
class Any
final class simpleEncoder2

Attributes

Supertypes
class Object
trait Matchable
class Any
final class trfEncoderTest

Attributes

Supertypes
class Object
trait Matchable
class Any

Value members

Concrete methods

def attentionTest(): Unit

The attentionTest main function tests the context and attention top-level functions. Test Self-Attention.

The attentionTest main function tests the context and attention top-level functions. Test Self-Attention.

runMain scalation.modeling.forecasting.neuralforecasting.attentionTest

Attributes

def attentionTest2(): Unit

The attentionTest2 main function tests the attentionMH top-level function. Test Multi-Head, Self-Attention.

The attentionTest2 main function tests the attentionMH top-level function. Test Multi-Head, Self-Attention.

runMain scalation.modeling.forecasting..neuralforecastingattentionTest2

Attributes

def attentionTest3(): Unit

The attentionTest3 main function tests the attention and context top-level function. Test Self-Attention. Read in weight matrices to compare with PyTorch.

The attentionTest3 main function tests the attention and context top-level function. Test Self-Attention. Read in weight matrices to compare with PyTorch.

runMain scalation.modeling.forecasting.neuralforecasting.attentionTest3

Attributes

def attentionTest4(): Unit

The attentionTest4 main function tests the attentionMH top-level function. Test Multi-Head, Self-Attention. Read in weight matrices to compare with PyTorch.

The attentionTest4 main function tests the attentionMH top-level function. Test Multi-Head, Self-Attention. Read in weight matrices to compare with PyTorch.

runMain scalation.modeling.forecasting.neuralforecasting.attentionTest4

Attributes

def attentionTest5(): Unit

The attentionTest5 main function tests the attention top-level function. Test Self-Attention with fixed weights

The attentionTest5 main function tests the attention top-level function. Test Self-Attention with fixed weights

runMain scalation.modeling.forecasting.neuralforecasting.attentionTest5

Attributes

def gRUTest(): Unit

The gRUTest main function tests the GRU class on randomly generated sequence data meant to represent encoded words

The gRUTest main function tests the GRU class on randomly generated sequence data meant to represent encoded words

runMain scalation.modeling.forecasting.neuralforecasting.gRUTest

Attributes

def gRUTest2(): Unit

The gRUTest2 main function tests the GRU class on sequence data read as words in a file that encoded and pass into GRU

The gRUTest2 main function tests the GRU class on sequence data read as words in a file that encoded and pass into GRU

runMain scalation.modeling.forecasting.neuralforecasting.gRUTest2

Attributes

def gRUTest3(): Unit

The gRUTest3 main function tests the GRU class on sequence/time series data corresponding to the lake level dataset using multiple lags.

The gRUTest3 main function tests the GRU class on sequence/time series data corresponding to the lake level dataset using multiple lags.

runMain scalation.modeling.forecasting.neuralforecasting.gRUTest3

Attributes

def genSequenceData(n_seq: Int, n_var: Int): (MatrixD, MatrixD)

Generate a fake sequence dataset: generate only one sentence for training. Only for testing. Needs to be changed to read in training data from files. The words are one-hot encoded into a column vector.

Generate a fake sequence dataset: generate only one sentence for training. Only for testing. Needs to be changed to read in training data from files. The words are one-hot encoded into a column vector.

Value parameters

n_seq

the sequence size (number of time points/words)

n_var

the number of variables/word encoding size

Attributes

def lSTMTest(): Unit

The lSTMTest main function tests the LSTM class on randomly generated sequence data meant to represent encoded words

The lSTMTest main function tests the LSTM class on randomly generated sequence data meant to represent encoded words

runMain scalation.modeling.forecasting.neuralforecasting.lSTMTest

Attributes

def lSTMTest2(): Unit

The lSTMTest2 main function tests the LSTM class on sequence data read as words in a file that encoded and pass into LSTM

The lSTMTest2 main function tests the LSTM class on sequence data read as words in a file that encoded and pass into LSTM

runMain scalation.modeling.forecasting.neuralforecasting.lSTMTest2

Attributes

def lSTMTest3(): Unit

The lSTMTest3 main function tests the LSTM class on sequence/time series data corresponding to the lake level dataset using multiple lags.

The lSTMTest3 main function tests the LSTM class on sequence/time series data corresponding to the lake level dataset using multiple lags.

runMain scalation.modeling.forecasting.neuralforecasting.lSTMTest3

Attributes

def log_(x: VectorD): VectorD
def neuralNet_3L4TSTest(): Unit

The neuralNet_3L4TSTest main function tests the NeuralNet_3L4TS class. This test is used to CHECK that the buildMatrix4TS function is working correctly. May get NaN for some maximum lags (p) due to multi-collinearity.

The neuralNet_3L4TSTest main function tests the NeuralNet_3L4TS class. This test is used to CHECK that the buildMatrix4TS function is working correctly. May get NaN for some maximum lags (p) due to multi-collinearity.

runMain scalation.modeling.forecasting.neuralforecasting.neuralNet_3L4TSTest

Attributes

def neuralNet_3L4TSTest2(): Unit

The neuralNet_3L4TSTest2 main function tests the NeuralNet_3L4TS class on real data: Forecasting lake levels.

The neuralNet_3L4TSTest2 main function tests the NeuralNet_3L4TS class on real data: Forecasting lake levels.

Attributes

See also

cran.r-project.org/web/packages/fpp/fpp.pdf

runMain scalation.modeling.forecasting.neuralforecasting.neuralNet_3L4TSTest2

def neuralNet_3L4TSTest3(): Unit

The neuralNet_3L4TSTest3 main function tests the NeuralNet_3L4TS class on real data: Forecasting COVID-19 Weekly Data.

The neuralNet_3L4TSTest3 main function tests the NeuralNet_3L4TS class on real data: Forecasting COVID-19 Weekly Data.

runMain scalation.modeling.forecasting.neuralforecasting.neuralNet_3L4TSTest3

Attributes

def patch(y: VectorD, ps: Int): MatrixD
def positionalEncTest(): Unit

The positionalEncTest main function is used to test the PositionalEnc class.

The positionalEncTest main function is used to test the PositionalEnc class.

runMain scalation.modeling.forecasting.neuralforecasting.positionalEncTest

Attributes

def rNNTest4(): Unit

Main function to test the RNN model on COVID-19 new deaths data. This function loads the COVID-19 new deaths data, preprocesses it, creates sequences for the RNN model, trains the model, and tests it. It prints the dimensions of the input and output matrices, and the value of the last element in the dataset.

Main function to test the RNN model on COVID-19 new deaths data. This function loads the COVID-19 new deaths data, preprocesses it, creates sequences for the RNN model, trains the model, and tests it. It prints the dimensions of the input and output matrices, and the value of the last element in the dataset.

runMain scalation.modeling.forecasting.neuralforecasting

Attributes

def simpleEncoder1(): Unit

The simpleEncoder1 main function illustrates the calculation of attention (Q, K, V) for a Single Head as used in a Transformer. SEE LINK BELOW FOR MORE DETAILS.

The simpleEncoder1 main function illustrates the calculation of attention (Q, K, V) for a Single Head as used in a Transformer. SEE LINK BELOW FOR MORE DETAILS.

Attributes

See also

pub.aimind.so/transformer-model-and-variants-of-transformer-chatgpt-3d423676e29c (URL)

runMain scalation.modeling.forecasting.neuralforecasting.simpleEncoder1

def simpleEncoder2(): Unit

The simpleEncoder2 main function illustrates the steps in an "Encoder-Only Transformer" consisting of a single encoder block with a "Prediction Head" added for making forecasts.

The simpleEncoder2 main function illustrates the steps in an "Encoder-Only Transformer" consisting of a single encoder block with a "Prediction Head" added for making forecasts.

runMain scalation.modeling.forecasting.neuralforecasting.simpleEncoder2

Attributes

def trfEncoderTest(): Unit

The trfEncoderTest main function illustrates the use of Gradient Descent (GD) to optimize the weights/parameters of a simple neural network (3-layer (1 hidden) Neural Network).

The trfEncoderTest main function illustrates the use of Gradient Descent (GD) to optimize the weights/parameters of a simple neural network (3-layer (1 hidden) Neural Network).

Attributes

See also

pub.aimind.so/transformer-model-and-variants-of-transformer-chatgpt-3d423676e29c (URL)

runMain scalation.modeling.forecasting.neuralforecasting.trfEncoderTest

Generated with