Colin/sanity check #445
Colin/sanity check #445
Conversation
| @@ -39,7 +39,8 @@ Imports: | |||
| renv (>= 0.8.0), | |||
| curl, | |||
| promises, | |||
| digest | |||
| digest, | |||
| cli | |||
There was a problem hiding this comment.
Let's move this to Suggests and check for it before running the main function
| # set global tutorial option which we can use as a basis for hooks | ||
| # (this is so we don't collide with hooks set by the user or | ||
| # by other packages or Rmd output formats) | ||
| knitr::opts_chunk$set(tutorial = TRUE) |
There was a problem hiding this comment.
Let's also restore to the original chunk value to what it was on.exit()
| # Setting learnr options | ||
| learnr::tutorial_options() | ||
|
|
There was a problem hiding this comment.
This is a no-op
| # Setting learnr options | |
| learnr::tutorial_options() |
| # inside the knitr environment | ||
| on.exit({ | ||
| knitr::knit_code$restore() | ||
| }) |
There was a problem hiding this comment.
Never hurts / always safe to use add = TRUE
| }) | |
| }, add = TRUE) |
| # This will register code inside `knitr::knit_code` | ||
| # The results is a list containing all the elements from the Rmd | ||
| # (i.e code + title + yaml) | ||
| res <- knitr:::split_file( |
There was a problem hiding this comment.
Is there a min version for this knitr function?
| setup = { | ||
| paste0(grab_chunk("setup")[[1]]$code, collapse = "\n") | ||
| }, |
There was a problem hiding this comment.
This is currently not used in evaluate_exercise(). Can set to NULL.
| setup = { | |
| paste0(grab_chunk("setup")[[1]]$code, collapse = "\n") | |
| }, | |
| setup = NULL, |
| setup_chunk <- usefull_chunks[["setup"]] | ||
| usefull_chunks[["setup"]] <- NULL |
There was a problem hiding this comment.
Should only need to evaluate this once. Save the result into an environment
| setup_chunk <- usefull_chunks[["setup"]] | |
| usefull_chunks[["setup"]] <- NULL | |
| setup_chunk <- usefull_chunks[["setup"]] | |
| usefull_chunks[["setup"]] <- NULL | |
| setup_chunk_envir <- eval(parse(text = setup_chunk), envir = new.env(parent = globalenv())) |
|
|
||
| res <- evaluate_exercise( | ||
| exercise_blt, | ||
| envir = new.env() |
There was a problem hiding this comment.
Use the previously source 'setup' environment
| envir = new.env() | |
| envir = setup_chunk_envir |
| usefull_chunks[chunk_][[1]]$options | ||
| }, | ||
| engine = { | ||
| usefull_chunks[chunk_][[1]]$enginee %||% "r" |
There was a problem hiding this comment.
| usefull_chunks[chunk_][[1]]$enginee %||% "r" | |
| usefull_chunks[chunk_][[1]]$engine %||% "r" |
| ``` | ||
|
|
||
| ## Flight | ||
|
|
There was a problem hiding this comment.
| ```{r random_chunk} | |
| # should not be checked | |
| 1 + 1 | |
| ``` | |
| } | ||
|
|
||
| # Add the options list to the chunk | ||
| usefull_chunks[[i]]$options <- options |
There was a problem hiding this comment.
Note: do not overwrite useful chunk options permanently. A exercise could be a setup chunk for another exercise.
There was a problem hiding this comment.
To help with this, I think we should abstract out a check_exercise_chunk() method that can be lapplyed over the exercises.
This inner should also have the ability to allow for the caller to pass in their own function value. If an author has a bad merge, and they are using gradethis::grade_code(), all it would be checking is if x == x, not if x is correct.
Both of these functions would serve great purposes. check_exercises() would be a quick and dirty check. check_exercise_chunk() could be used within CI to validate that tutorials work as expected with many different inputs.
Something like...
check_exercise_chunk <- function(
file,
..., # should be ignored
chunks = file_to_chunks(file),
user_code = NULL # or expression
) {
# ....
if (rlang::is_expression(user_code)) {
user_code <- solution_code
}
# ....
# return feedback object
}
# would use the solutions for every exercise
check_exercises("file.Rmd")
# would use the supplied user code to 'ex1' in 'file.Rmd'
check_exercise_chunk("file.Rmd", "ex1", user_code = rlang::expr({
# this is user code
40 + 2
})I would be REALLY happy to have this inside gradethis to validate that gradethis is working as expected with MANY different types of testing documents.
|
|
This is a PR implementing a sanity-check for a Rmd. (WIP for now)