The Fit companion object provides factory methods for assessing quality of fit for standard types of modeling techniques.
Attributes
Members list
Value members
Concrete methods
Return a contrary/starting value, -∞ for maximization, ∞ for minimization
Return a contrary/starting value, -∞ for maximization, ∞ for minimization
Value parameters
- qk
-
the QoF metric index/ordinal value
Attributes
Return the help string that describes the Quality of Fit (QoF) measures provided by the Fit trait. The QoF measures are divided into two groups: general and statistical (that often require degrees of freedom and/or log-likelihoods).
Return the help string that describes the Quality of Fit (QoF) measures provided by the Fit trait. The QoF measures are divided into two groups: general and statistical (that often require degrees of freedom and/or log-likelihoods).
Attributes
- See also
-
en.wikipedia.org/wiki/Coefficient_of_determination
Return the Mean Absolute Error (MAE) for the forecasting model under test.
Return the Mean Absolute Error (MAE) for the forecasting model under test.
Value parameters
- h
-
the forecasting horizon or stride (defaults to 1)
- y
-
the given time-series (must be aligned with the forecast)
- yp
-
the forecasted time-series
Attributes
Return the Mean Absolute Error (MAE) for the Naive Model (simple random walk) with horizon/stride h. For comparison with the above method.
Return the Mean Absolute Error (MAE) for the Naive Model (simple random walk) with horizon/stride h. For comparison with the above method.
Value parameters
- h
-
the forecasting horizon or stride (defaults to 1)
- y
-
the given time-series
Attributes
Return the Mean Absolute Scaled Error (MASE) for the given time-series. It is the ratio of MAE of the forecasting model under test and the MAE of the Naive Model (simple random walk).
Return the Mean Absolute Scaled Error (MASE) for the given time-series. It is the ratio of MAE of the forecasting model under test and the MAE of the Naive Model (simple random walk).
Value parameters
- h
-
the forecasting horizon or stride (defaults to 1)
- y
-
the given time-series (must be aligned with the forecast)
- yp
-
the forecasted time-series
Attributes
Return the Mean Interval Score (MIS) metric which starts with the average prediction interval width and adds a penalty for each true_value y(i) that is outside the prediction interval. Smaller (in absolute value) scores are better.
Return the Mean Interval Score (MIS) metric which starts with the average prediction interval width and adds a penalty for each true_value y(i) that is outside the prediction interval. Smaller (in absolute value) scores are better.
Value parameters
- low_up
-
the (lower, upper) bound vectors used for prediction intervals & @param α the significance level (1 - p_)
- y
-
the given time-series (must be aligned with the interval forecast)
Attributes
- See also
-
huiwenn.github.io/predictive-distributions
arxiv.org/pdf/2005.12881.pdf
search.r-project.org/CRAN/refmans/scoringutils/html/interval_score.html score = (up − low) + α/2 * (low − true_value) ∗ is(true_value < low) + α/2 * (true_value − up) ∗ is(true_value > up)
Return the Prediction Interval Coverage Probability (PICP) metric, i.e., the fraction of actual values inside the prediction interval. While PINC is the nominal/desired coverage probability (1 - α), PICP is the corresponding empirical coverage probability.
Return the Prediction Interval Coverage Probability (PICP) metric, i.e., the fraction of actual values inside the prediction interval. While PINC is the nominal/desired coverage probability (1 - α), PICP is the corresponding empirical coverage probability.
Value parameters
- low_up
-
the (lower, upper) bound vectors used for prediction intervals
- y
-
the given time-series (must be aligned with the interval forecast)
Attributes
Return the Prediction Interval Normalised Average Deviation (PINAD) metric, i.e., the normalized (by range) average deviation outside the prediction interval.
Return the Prediction Interval Normalised Average Deviation (PINAD) metric, i.e., the normalized (by range) average deviation outside the prediction interval.
Value parameters
- low_up
-
the (lower, upper) bound vectors used for prediction intervals
- y
-
the given time-series (must be aligned with the interval forecast)
Attributes
Create a table to store statistics for QoF measures, where each row corresponds to the statistics on a particular QoF measure, e.g., rSq.
Create a table to store statistics for QoF measures, where each row corresponds to the statistics on a particular QoF measure, e.g., rSq.
Attributes
Collect QoF results for a model and return them in a vector.
Collect QoF results for a model and return them in a vector.
Value parameters
- cv_fit
-
the fit array of statistics for cross-validation (upon test sets)
- fit
-
the fit vector with regard to the training set
Attributes
Show the quality of fit measures/metrics for each response/output variable. @ @see FitM.showFitMap
Show the quality of fit measures/metrics for each response/output variable. @ @see FitM.showFitMap
Value parameters
- ftLab
-
the array of QoF labels (defaults to QoF.values.map (_.toString))
- ftMat
-
the matrix of QoF values (qof x var)
Attributes
Tally the current QoF measures into the statistical accumulators.
Tally the current QoF measures into the statistical accumulators.
Value parameters
- qof
-
the current QoF measure vector
- stats
-
the statistics table being updated
Attributes
Return the Weighted Interval Score (WIS) metric, i.e., a weighted average of K prediction intervals each calculated for a different alpha (α) level. WIS approximates the Continuous Ranked Probability Score (CRPS).
Return the Weighted Interval Score (WIS) metric, i.e., a weighted average of K prediction intervals each calculated for a different alpha (α) level. WIS approximates the Continuous Ranked Probability Score (CRPS).
Value parameters
- low_up
-
the (lower, upper) bound vectors used for prediction intervals
- y
-
the given time-series (must be aligned with the interval forecast)
- yp
-
the point prediction mean/median
- α
-
the vector of significance levels (defaults to the K = 11 prediction intervals used by the COVID-19 Forecast Hub)
Attributes
- See also
-
arxiv.org/pdf/2005.12881.pdf
pmc.ncbi.nlm.nih.gov/articles/PMC7880475/pdf/pcbi.1008618.pdf (equation 1)