Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #448: Add back in examples for a few plots #463

Merged
merged 4 commits into from
Jan 4, 2024

Conversation

nikosbosse
Copy link
Contributor

@nikosbosse nikosbosse commented Nov 18, 2023

Closes #448.

Due to previous changes in add_coverage(), several plots that used quantiles or interval ranges. My "solution" at the time was to simply comment the code out. This PR adds it back in and updates the examples.

The PR is currently blocked by the fact that one function that I used in the example is not currently exported to the user. I didn't want to start using ::: in examples, so I suggest addressing that first before merging this PR.

More broadly speaking, there is a question about whether we want to support the function plot_ranges() at all and what should happen with it. I created an issue for that, #462

@nikosbosse nikosbosse added the blocked Something needs to happen before this can proceed label Nov 18, 2023
@nikosbosse nikosbosse marked this pull request as draft November 18, 2023 12:44
@nikosbosse nikosbosse changed the base branch from main to dev November 18, 2023 12:44
@nikosbosse nikosbosse changed the base branch from dev-backup to dev November 18, 2023 15:42
Copy link

codecov bot commented Jan 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a0f23ed) 81.87% compared to head (db6cd16) 81.87%.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #463   +/-   ##
========================================
  Coverage    81.87%   81.87%           
========================================
  Files           20       20           
  Lines         1677     1677           
========================================
  Hits          1373     1373           
  Misses         304      304           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nikosbosse nikosbosse removed the blocked Something needs to happen before this can proceed label Jan 3, 2024
@nikosbosse
Copy link
Contributor Author

Summarising what's happening:
The PR adds examples for a few plotting functions back in that were previously commented out, specifically

  • plot_interval_coverage()
  • plot_quantile_coverage()
  • plot_ranges()

As mentioned above, the example for plot_ranges() is a bit hacky as it makes use of an internal function accessed via :::. In #462 you suggested removing plot_ranges() altogether for now. I have a vague sense of "it's bad to remove existing functionality", but I also see that leaving it in means committing to maintaining it in a backwards-compatible way.
My solution to that internal conflict is: I fixed the example code + left it in for now so you can have another look at the example below. If you don't have a sudden change of heart I'll delete it from this PR (already pasted the code to #462) and just merge the rest.

this is the output after running the example code for plot_ranges():

library(ggplot2)
ex <- example_quantile
ex$interval_range <- scoringutils:::get_range_from_quantile(ex$quantile)
scores <- score(ex, metrics = list("wis" = wis))
scores$range <- scores$interval_range
summarised <- summarise_scores(
  scores,
  by = c("model", "target_type", "range")
)
plot_ranges(summarised, x = "model") +
  facet_wrap(~target_type, scales = "free")

image

@nikosbosse nikosbosse marked this pull request as ready for review January 3, 2024 12:11
@nikosbosse nikosbosse requested a review from seabbs January 3, 2024 12:13
Copy link
Contributor

@seabbs seabbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. See comment for thoughts on plot_ranges/

@seabbs seabbs merged commit c86feb7 into develop Jan 4, 2024
@seabbs seabbs deleted the examples-range-plots branch January 4, 2024 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add examples for plotting functions back in
2 participants