Skip to content

Commit

Permalink
fix a bunch of Notes
Browse files Browse the repository at this point in the history
  • Loading branch information
nikosbosse committed Oct 30, 2024
1 parent bf566e4 commit 0969dab
Show file tree
Hide file tree
Showing 31 changed files with 78 additions and 82 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Authors@R: c(
email = "[email protected]",
comment = c(ORCID = "0000-0003-0787-087X")))
Description:
scoringutils facilitates the evaluation of forecasts in a convenient
Facilitate the evaluation of forecasts in a convenient
framework based on data.table. It allows user to to check their forecasts
and diagnose issues, to visualise forecasts and missing data, to transform
data before scoring, to handle missing forecasts, to aggregate scores, and
Expand Down
4 changes: 2 additions & 2 deletions R/class-forecast-binary.R
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ score.forecast_binary <- function(forecast, metrics = get_metrics(forecast), ...
#' @param ... unused
#' @return A list of scoring functions.
#' @export
#' @family `get_metrics` functions
#' @family get_metrics functions
#' @keywords handle-metrics
#' @examples
#' get_metrics(example_binary)
Expand Down Expand Up @@ -142,6 +142,6 @@ get_metrics.forecast_binary <- function(x, select = NULL, exclude = NULL, ...) {
#' \item{predicted}{predicted value}
#' }
# nolint start
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe_archive/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
# nolint end
"example_binary"
4 changes: 2 additions & 2 deletions R/class-forecast-nominal.R
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ score.forecast_nominal <- function(forecast, metrics = get_metrics(forecast), ..
#' For nominal forecasts, the default scoring rule is:
#' - "log_score" = [logs_nominal()]
#' @export
#' @family `get_metrics` functions
#' @family get_metrics functions
#' @keywords handle-metrics
#' @examples
#' get_metrics(example_nominal)
Expand Down Expand Up @@ -150,6 +150,6 @@ get_metrics.forecast_nominal <- function(x, select = NULL, exclude = NULL, ...)
#' \item{horizon}{forecast horizon in weeks}
#' }
# nolint start
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe_archive/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
# nolint end
"example_nominal"
4 changes: 2 additions & 2 deletions R/class-forecast-point.R
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ score.forecast_point <- function(forecast, metrics = get_metrics(forecast), ...)
#' @inheritSection illustration-input-metric-binary-point Input format
#' @inheritParams get_metrics.forecast_binary
#' @export
#' @family `get_metrics` functions
#' @family get_metrics functions
#' @keywords handle-metrics
#' @examples
#' get_metrics(example_point, select = "ape")
Expand Down Expand Up @@ -160,6 +160,6 @@ get_metrics.forecast_point <- function(x, select = NULL, exclude = NULL, ...) {
#' \item{horizon}{forecast horizon in weeks}
#' }
# nolint start
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe_archive/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
# nolint end
"example_point"
4 changes: 2 additions & 2 deletions R/class-forecast-quantile.R
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ score.forecast_quantile <- function(forecast, metrics = get_metrics(forecast), .
#' @inheritSection illustration-input-metric-quantile Input format
#' @inheritParams get_metrics.forecast_binary
#' @export
#' @family `get_metrics` functions
#' @family get_metrics functions
#' @keywords handle-metrics
#' @importFrom purrr partial
#' @examples
Expand Down Expand Up @@ -252,6 +252,6 @@ get_pit_histogram.forecast_quantile <- function(forecast, num_bins = NULL,
#' \item{horizon}{forecast horizon in weeks}
#' }
# nolint start
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe_archive/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
# nolint end
"example_quantile"
6 changes: 3 additions & 3 deletions R/class-forecast-sample.R
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ score.forecast_sample <- function(forecast, metrics = get_metrics(forecast), ...
#' @inheritSection illustration-input-metric-sample Input format
#' @inheritParams get_metrics.forecast_binary
#' @export
#' @family `get_metrics` functions
#' @family get_metrics functions
#' @keywords handle-metrics
#' @examples
#' get_metrics(example_sample_continuous, exclude = "mad")
Expand Down Expand Up @@ -238,7 +238,7 @@ get_pit_histogram.forecast_sample <- function(forecast, num_bins = 10,
#' \item{sample_id}{id for the corresponding sample}
#' }
# nolint start
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe_archive/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
# nolint end
"example_sample_continuous"

Expand Down Expand Up @@ -266,6 +266,6 @@ get_pit_histogram.forecast_sample <- function(forecast, num_bins = 10,
#' \item{sample_id}{id for the corresponding sample}
#' }
# nolint start
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe_archive/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/}
# nolint end
"example_sample_discrete"
2 changes: 1 addition & 1 deletion R/class-scores.R
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ assert_scores <- function(scores) {
#' Character vector with the names of the scoring rules that were used
#' for scoring.
#' @keywords handle-metrics
#' @family `get_metrics` functions
#' @family get_metrics functions
#' @export
get_metrics.scores <- function(x, error = FALSE, ...) {
assert_data_frame(x)
Expand Down
16 changes: 7 additions & 9 deletions R/metrics-quantile.R
Original file line number Diff line number Diff line change
Expand Up @@ -556,31 +556,29 @@ interpolate_median <- function(predicted, quantile_level) {
}


#' @title Absolute error of the median (quantile-based version)
#' @description
#' Absolute error of the median (quantile-based version)
#'
#' Compute the absolute error of the median calculated as
#' \deqn{
#' \textrm{abs}(\textrm{observed} - \textrm{median prediction})
#' }{
#' abs(observed - median_prediction)
#' |observed - median prediction|
#' }
#' The median prediction is the predicted value for which quantile_level == 0.5,
#' the function therefore requires 0.5 to be among the quantile levels in
#' `quantile_level`.
#' The median prediction is the predicted value for which quantile_level == 0.5.
#' The function requires 0.5 to be among the quantile levels in `quantile_level`.
#'
#' @inheritParams wis
#' @inheritSection illustration-input-metric-quantile Input format
#' @return Numeric vector of length N with the absolute error of the median.
#' @seealso [ae_median_sample()]
#' @importFrom stats median
#' @importFrom cli cli_warn
#' @keywords metric
#' @examples
#' observed <- rnorm(30, mean = 1:30)
#' predicted_values <- replicate(3, rnorm(30, mean = 1:30))
#' ae_median_quantile(
#' observed, predicted_values, quantile_level = c(0.2, 0.5, 0.8)
#' )
#' @export
#' @keywords metric
ae_median_quantile <- function(observed, predicted, quantile_level) {
assert_input_quantile(observed, predicted, quantile_level)
if (!any(quantile_level == 0.5)) {
Expand Down
14 changes: 6 additions & 8 deletions R/metrics-sample.R
Original file line number Diff line number Diff line change
Expand Up @@ -119,28 +119,26 @@ bias_sample <- function(observed, predicted) {
#'
#' @description
#' Absolute error of the median calculated as
#'
#' \deqn{%
#' \textrm{abs}(\textrm{observevd} - \textrm{median\_prediction})
#' }{%
#' abs(observed - median_prediction)
#' \deqn{
#' |observed - median\_prediction|
#' }
#' where the median prediction is calculated as the median of the predictive
#' samples.
#'
#' @param observed A vector with observed values of size n
#' @param predicted nxN matrix of predictive samples, n (number of rows) being
#' the number of data points and N (number of columns) the number of Monte
#' Carlo samples. Alternatively, `predicted` can just be a vector of size n.
#' @inheritSection illustration-input-metric-sample Input format
#' @return vector with the scoring values
#' @return Numeric vector of length n with the absolute errors of the median.
#' @seealso [ae_median_quantile()]
#' @importFrom stats median
#' @keywords metric
#' @examples
#' observed <- rnorm(30, mean = 1:30)
#' predicted_values <- matrix(rnorm(30, mean = 1:30))
#' ae_median_sample(observed, predicted_values)
#' @export
#' @keywords metric

ae_median_sample <- function(observed, predicted) {
assert_input_sample(observed, predicted)
median_predictions <- apply(
Expand Down
2 changes: 1 addition & 1 deletion R/metrics.R
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ select_metrics <- function(metrics, select = NULL, exclude = NULL) {
#' See [as_forecast()] for more information on `forecast` objects and [score()]
#' for more information on `scores` objects.
#'
#' @family `get_metrics` functions
#' @family get_metrics functions
#' @keywords handle-metrics
#' @export
get_metrics <- function(x, ...) {
Expand Down
7 changes: 4 additions & 3 deletions R/pairwise-comparisons.R
Original file line number Diff line number Diff line change
Expand Up @@ -565,17 +565,18 @@ permutation_test <- function(scores1,
}


#' @title Add relative skill scores based on pairwise comparisons
#' @description
#' Add relative skill scores based on pairwise comparisons
#'
#' Adds a columns with relative skills computed by running
#' pairwise comparisons on the scores.
#' For more information on
#' the computation of relative skill, see [get_pairwise_comparisons()].
#' Relative skill will be calculated for the aggregation level specified in
#' `by`.
#'
#' @inheritParams get_pairwise_comparisons
#' @export
#' @keywords keyword scoring
#' @keywords scoring
add_relative_skill <- function(
scores,
compare = "model",
Expand Down
10 changes: 5 additions & 5 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,17 @@ devel <- length(unclass(package_version(version))[[1]]) > 3
if (devel) {
cat(
"**Note**: ",
"[This documentation](https://epiforecasts.io/scoringutils/dev) refers to the development version of `scoringutils`. ",
"[This documentation](https://epiforecasts.io/scoringutils/dev/) refers to the development version of `scoringutils`. ",
"You can also view the [documentation of the stable version]",
"(https://epiforecasts.io/scoringutils).",
"(https://epiforecasts.io/scoringutils/).",
sep = ""
)
} else {
cat(
"**Note**: ",
"[This documentation](https://epiforecasts.io/scoringutils) refers to the stable version of `scoringutils`. ",
"[This documentation](https://epiforecasts.io/scoringutils/) refers to the stable version of `scoringutils`. ",
"You can also view the [documentation of the development version]",
"(https://epiforecasts.io/scoringutils/dev).",
"(https://epiforecasts.io/scoringutils/dev/).",
sep = ""
)
}
Expand Down Expand Up @@ -163,7 +163,7 @@ The development of `scoringutils` was funded via the Health Protection Research
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->

All contributions to this project are gratefully acknowledged using the [`allcontributors` package](https://github.com/ropenscilabs/allcontributors) following the [all-contributors](https://allcontributors.org) specification. Contributions of any kind are welcome!
All contributions to this project are gratefully acknowledged using the [`allcontributors` package](https://github.com/ropensci/allcontributors) following the [all-contributors](https://allcontributors.org) specification. Contributions of any kind are welcome!

### Code

Expand Down
20 changes: 12 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,11 @@ version](https://img.shields.io/github/r-package/v/epiforecasts/scoringutils)
downloads](http://cranlogs.r-pkg.org/badges/grand-total/scoringutils)](https://cran.r-project.org/package=scoringutils)
<!-- badges: end -->

**Note**: [This documentation](https://epiforecasts.io/scoringutils/dev)
refers to the development version of `scoringutils`. You can also view
the [documentation of the stable
version](https://epiforecasts.io/scoringutils).
**Note**: [This
documentation](https://epiforecasts.io/scoringutils/dev/) refers to the
development version of `scoringutils`. You can also view the
[documentation of the stable
version](https://epiforecasts.io/scoringutils/).

The `scoringutils` package facilitates the process of evaluating
forecasts in R, using a convenient and flexible `data.table`-based
Expand Down Expand Up @@ -253,14 +254,15 @@ Health.
## Contributors

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->

<!-- prettier-ignore-start -->

<!-- markdownlint-disable -->

All contributions to this project are gratefully acknowledged using the
[`allcontributors`
package](https://github.com/ropenscilabs/allcontributors) following the
[all-contributors](https://allcontributors.org) specification.
Contributions of any kind are welcome!
[`allcontributors` package](https://github.com/ropensci/allcontributors)
following the [all-contributors](https://allcontributors.org)
specification. Contributions of any kind are welcome!

### Code

Expand Down Expand Up @@ -291,5 +293,7 @@ Contributions of any kind are welcome!
<a href="https://github.com/epiforecasts/scoringutils/issues?q=is%3Aissue+commenter%3Akathsherratt">kathsherratt</a>

<!-- markdownlint-enable -->

<!-- prettier-ignore-end -->

<!-- ALL-CONTRIBUTORS-LIST:END -->
1 change: 0 additions & 1 deletion man/add_relative_skill.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 3 additions & 6 deletions man/ae_median_quantile.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 5 additions & 6 deletions man/ae_median_sample.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/example_binary.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/example_nominal.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/example_point.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/example_quantile.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/example_sample_continuous.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/example_sample_discrete.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 0969dab

Please sign in to comment.