AutogradTest
The AutogradTest object contains various @main tests for autograd functionality. The tests validate basic arithmetic, complex expressions, activation functions, loss functions, and neural network layers with backpropagation.
Attributes
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
AutogradTest.type
Members list
Value members
Concrete methods
The autogradTest0 main function tests basic unary operations in the Autograd system. These include absolute value, negation, floor, ceil, rounding, sign, and clipping. For each operator, the test reports the forward results and performs gradient checking using GradCheck.gradCheck. This ensures that each scalar/tensor unary operation is correctly implemented in both the forward and backward passes.
The autogradTest0 main function tests basic unary operations in the Autograd system. These include absolute value, negation, floor, ceil, rounding, sign, and clipping. For each operator, the test reports the forward results and performs gradient checking using GradCheck.gradCheck. This ensures that each scalar/tensor unary operation is correctly implemented in both the forward and backward passes.
runMain scalation.modeling.autograd.AutogradTest.autogradTest0
Attributes
The autogradTest1 main function tests basic binary arithmetic operations in the Autograd system. These include addition, subtraction, element-wise multiplication, and element-wise division between two tensors, as well as unary operations involving constants. For each operation, the test applies GradCheck.gradCheck or gradCheckAll to verify that both the forward and backward passes are implemented correctly.
The autogradTest1 main function tests basic binary arithmetic operations in the Autograd system. These include addition, subtraction, element-wise multiplication, and element-wise division between two tensors, as well as unary operations involving constants. For each operation, the test applies GradCheck.gradCheck or gradCheckAll to verify that both the forward and backward passes are implemented correctly.
runMain scalation.modeling.autograd.AutogradTest.autogradTest1
Attributes
The autogradTest10 main function trains a three-layer neural network on the AutoMPG dataset using SGD with momentum and an early-stopping rule. Multiple nonlinearities (tanh and sigmoid) are used to exercise diverse gradient shapes. The test demonstrates the autograd system’s support for: multi-layer models, training loops, early stopping, loss monitoring, and post-training evaluation with rescaled predictions.
The autogradTest10 main function trains a three-layer neural network on the AutoMPG dataset using SGD with momentum and an early-stopping rule. Multiple nonlinearities (tanh and sigmoid) are used to exercise diverse gradient shapes. The test demonstrates the autograd system’s support for: multi-layer models, training loops, early stopping, loss monitoring, and post-training evaluation with rescaled predictions.
runMain scalation.modeling.autograd.AutogradTest.autogradTest10
Attributes
The autogradTest11 main function tests a three-layer neural network on a train/test split of the AutoMPG dataset. The network uses sigmoid and ReLU activations and trains with SGD + momentum and an early stopping rule based on validation loss. This test validates the autograd system’s ability to handle data splitting, model evaluation on unseen data, and training control flow involving patience-based stopping.
The autogradTest11 main function tests a three-layer neural network on a train/test split of the AutoMPG dataset. The network uses sigmoid and ReLU activations and trains with SGD + momentum and an early stopping rule based on validation loss. This test validates the autograd system’s ability to handle data splitting, model evaluation on unseen data, and training control flow involving patience-based stopping.
runMain scalation.modeling.autograd.AutogradTest.autogradTest11
Attributes
The autogradTest12 main function provides an alternative training pipeline for a three-layer neural network on AutoMPG, using randomized train/test splitting. It tests the autograd engine under variations in data sampling, training strategy, and network depth, while also exercising patience-based early stopping and full regression evaluation on both training and test sets.
The autogradTest12 main function provides an alternative training pipeline for a three-layer neural network on AutoMPG, using randomized train/test splitting. It tests the autograd engine under variations in data sampling, training strategy, and network depth, while also exercising patience-based early stopping and full regression evaluation on both training and test sets.
runMain scalation.modeling.autograd.AutogradTest.autogradTest12
Attributes
The autogradTest2 main function tests a more complex expression formed by combining several elementary operations, namely element-wise z = (x * y) + (x / y) - y. It evaluates the forward computation and then uses GradCheck.gradCheckAll to verify correct gradient propagation through the composite computation graph. The test also exports the resulting computation graph in DOT format for visualization.
The autogradTest2 main function tests a more complex expression formed by combining several elementary operations, namely element-wise z = (x * y) + (x / y) - y. It evaluates the forward computation and then uses GradCheck.gradCheckAll to verify correct gradient propagation through the composite computation graph. The test also exports the resulting computation graph in DOT format for visualization.
runMain scalation.modeling.autograd.AutogradTest.autogradTest2
Attributes
The autogradTest3 main function tests a broad range of mathematical operations supported by the Autograd system. These include scalar operations (power, square root, logarithm, reciprocal, exponential), global reductions (mean, sum, variance, standard deviation), axis-wise reductions, and extrema extraction. For each operation, the forward result is displayed and GradCheck.gradCheck is applied to verify the correctness of the backward pass.
The autogradTest3 main function tests a broad range of mathematical operations supported by the Autograd system. These include scalar operations (power, square root, logarithm, reciprocal, exponential), global reductions (mean, sum, variance, standard deviation), axis-wise reductions, and extrema extraction. For each operation, the forward result is displayed and GradCheck.gradCheck is applied to verify the correctness of the backward pass.
runMain scalation.modeling.autograd.AutogradTest.autogradTest3
Attributes
The autogradTest4 main function tests a variety of activation functions supported by the Autograd system, including ReLU, Sigmoid, Tanh, GeLU, Softmax, Identity, LeakyReLU, and ELU. Each activation is applied to a representative set of inputs to check behavior across different regions (e.g., saturation, symmetry, negative/positive domains). Gradient checking via GradCheck.gradCheck is performed to ensure correct backward propagation through each nonlinearity. Note that GeLU and Softmax forward values are currently inconsistent with PyTorch and are marked for revision.
The autogradTest4 main function tests a variety of activation functions supported by the Autograd system, including ReLU, Sigmoid, Tanh, GeLU, Softmax, Identity, LeakyReLU, and ELU. Each activation is applied to a representative set of inputs to check behavior across different regions (e.g., saturation, symmetry, negative/positive domains). Gradient checking via GradCheck.gradCheck is performed to ensure correct backward propagation through each nonlinearity. Note that GeLU and Softmax forward values are currently inconsistent with PyTorch and are marked for revision.
runMain scalation.modeling.autograd.AutogradTest.autogradTest4
Attributes
The autogradTest5 main function tests core regression loss functions supported by the Autograd system: SSE, MSE, and MAE. Each loss is evaluated on simple predicted and target tensors, and GradCheck.gradCheck is used to validate correctness of the backward pass. These losses form the basis of most regression models, so ensuring accurate gradients is essential.
The autogradTest5 main function tests core regression loss functions supported by the Autograd system: SSE, MSE, and MAE. Each loss is evaluated on simple predicted and target tensors, and GradCheck.gradCheck is used to validate correctness of the backward pass. These losses form the basis of most regression models, so ensuring accurate gradients is essential.
runMain scalation.modeling.autograd.AutogradTest.autogradTest5
Attributes
The autogradTest6 main function verifies tensor-level linear algebra operations, including transpose, permutation, reshape, slice, concat, dot product, matrix multiplication, and batched matrix multiplication (BMM). For each operation, forward correctness is checked and gradient correctness is validated using GradCheck.gradCheck or gradCheckAll. BMM is also compared against a manually computed expected tensor to ensure alignment with the Autograd system’s shape conventions.
The autogradTest6 main function verifies tensor-level linear algebra operations, including transpose, permutation, reshape, slice, concat, dot product, matrix multiplication, and batched matrix multiplication (BMM). For each operation, forward correctness is checked and gradient correctness is validated using GradCheck.gradCheck or gradCheckAll. BMM is also compared against a manually computed expected tensor to ensure alignment with the Autograd system’s shape conventions.
runMain scalation.modeling.autograd.AutogradTest.autogradTest6
Attributes
The autogradTest7 main function tests autograd through a small two-layer fully connected neural network. A Linear → ReLU → Linear architecture is constructed, and gradients are validated for inputs, weights, and biases using GradCheck.gradCheck. This test ensures end-to-end correctness of forward propagation, backward propagation, and parameter gradient tracking. A computation graph for the network output is also exported for debugging.
The autogradTest7 main function tests autograd through a small two-layer fully connected neural network. A Linear → ReLU → Linear architecture is constructed, and gradients are validated for inputs, weights, and biases using GradCheck.gradCheck. This test ensures end-to-end correctness of forward propagation, backward propagation, and parameter gradient tracking. A computation graph for the network output is also exported for debugging.
runMain scalation.modeling.autograd.AutogradTest.autogradTest7
Attributes
The autogradTest8 main function trains a single-layer neural network on the AutoMPG regression dataset using Autograd. The dataset is standardized, fed through a Linear ~> Identity model, and optimized using stochastic gradient descent. This test validates that the autograd engine supports full training loops, gradient accumulation, parameter updates, and evaluation against regression metrics.
The autogradTest8 main function trains a single-layer neural network on the AutoMPG regression dataset using Autograd. The dataset is standardized, fed through a Linear ~> Identity model, and optimized using stochastic gradient descent. This test validates that the autograd engine supports full training loops, gradient accumulation, parameter updates, and evaluation against regression metrics.
runMain scalation.modeling.autograd.AutogradTest.autogradTest8
Attributes
The autogradTest9 main function trains a two-layer neural network on the AutoMPG regression dataset. The architecture uses a hidden layer with a sigmoid activation followed by a linear output layer. Optimization uses the Adam optimizer, demonstrating compatibility between Autograd parameter graphs and adaptive optimizers. The test evaluates gradient flow, training stability, and final regression quality after rescaling predictions back to the original domain.
The autogradTest9 main function trains a two-layer neural network on the AutoMPG regression dataset. The architecture uses a hidden layer with a sigmoid activation followed by a linear output layer. Optimization uses the Adam optimizer, demonstrating compatibility between Autograd parameter graphs and adaptive optimizers. The test evaluates gradient flow, training stability, and final regression quality after rescaling predictions back to the original domain.
runMain scalation.modeling.autograd.AutogradTest.autogradTest9
Attributes
The autogradTestAll main function sequentially runs the core autograd unit tests (0–7) to verify correctness across basic ops, math operations, activations, losses, tensor algebra, and small neural networks. Longer AutoMPG training tests (8–11) are provided but commented out to avoid extended runtime by default.
The autogradTestAll main function sequentially runs the core autograd unit tests (0–7) to verify correctness across basic ops, math operations, activations, losses, tensor algebra, and small neural networks. Longer AutoMPG training tests (8–11) are provided but commented out to avoid extended runtime by default.
runMain scalation.modeling.autograd.AutogradTest.autogradTestAll