Feedback on application execution

[rpodcast]

With the application now constructed with {golem} and a successful bundling of the app files with {pkglite}, I'd like to request others on our team to verify the application works successfully for them. I have specific details in the vignette vignettes/adrg-prepare.Rmd on how the app was bundled as well as how to unpack and execute the app with the {renv} package environment. To be more specific here are the ways you can help test:

Windows-specific Instructions

The following instructions are lifted from the draft ADRG document. If you already have an installation of R version 4.1.2 or later (as well as RStudio), you can disregard the installation step. The only other requirement is to either clone the contents of this repository or to download the vignettes/ectd/r1pkg.txt bundle file from this repository before proceeding.

Install R and RStudio

If R has not been installed per the instructions detailed in Appendix 1, download and install R 4.1.2 for Windows from https://cran.r-project.org/bin/windows/base/old/4.1.2/R-4.1.2-win.exe. It is also recommended to view the Shiny application within the RStudio IDE. You can download RStudio for Windows by visiting https://www.rstudio.com/products/rstudio/download/#download .

Define Data Files Directory

Create a directory specifically for the ADaM data sets included in the Pilot 1 submission. Copy all of the ADaM data files (ending with extension .xpt) to this directory. For the purposes of these instructions, the directory chosen is C:\pilot1_adam_data.

Install open-source R packages

A minimum set of R packages are required to ensure the Pilot 2 Shiny application files are successfully unpacked and the custom package environment used for the application is replicated correctly. The first packages to install are the remotes and pkglite packages:

install.packages(c(“remotes”, “pkglite”))

Next, install version 0.15.2 of the renv package:

remotes::install_version(“renv”, version = “0.15.2”)

Unpack Shiny Application pkglite bundle

Use the pkglite package to unpack the Shiny application bundle located in the r1pkg.txt text file included in the pilot 2 submission transfer ( m5/datasets/rconsortiumpilot2/analysis/adam/programs/r1pkg.txt ). Take note of the file path where r1pkg.txt is located on your system. For the purposes of these instructions, the location is C:\pilot2_files\r1pkg.txt and run the following R command:

pkglite::unpack(input = “C:/pilot2_files/r1pkg.txt”, output = “C:/pilot2_files”)

After running this command, you should see a new directory pilot2wrappers created within the path specified in the output argument in the previous command.

Initialize R package environment for Shiny application

To bootstrap the customized R package library used for the Shiny application, launch a new R session in the directory where you unpacked the application source files in the previous step. Within the RStudio IDE, create a new project in the pilot2wrappers directory using the following procedure: Select File -> New Project, choose “Existing Directory”, and browse to the pilot2wrappers location on your computer. Then click “Create Project.”

You should see RStudio refresh and start a new session in the specified directory. Within the R console, you will see information from the renv package regarding installation and loading renv into the project. To complete the process of installing the custom R package library for the application, run the following command in the R console:

renv::restore(prompt = FALSE)

The package installation procedure may take a few minutes or longer depending on internet bandwidth.

Update Shiny Application Configuration for Data Location

The Shiny application needs one configuration update in order to import the ADaM data sets in the Pilot 1 submission. Using the directory established in step (2) in these instructions, run the following R command:

pilot2wrappers::set_data_path(“C:\pilot1_adam_data”)

Launch Shiny Application

Method 1: Open the app.R file inside RStudio and click the Run App button within source code window

Method 2: Run the following R command:

golem:::run_dev()

Please add comments to this issue with your feedback. I'll continue to do more testing on my side and add more notes as appropriate.

[lengning]

thanks @rpodcast ! I was able to reproduce the app from this repo!

Several few observations/questions

• after unpacking pilot2wrapper, it seems the app is not running (no dev/run_dev.R)
• I think we will need to submit both pilot1wrappers and pilot2wrappers? Then I guess we may need to name the txt file to another name other than r0pkg.txt? (since we used this file name for pilot1wrappers)
• naive question - do we need to store the xpt files again under inst/adam?

[rpodcast]

Thank you for the comments @lengning !

after unpacking pilot2wrapper, it seems the app is not running (no dev/run_dev.R)

Great catch I will update the bundle to include the dev directory.

I think we will need to submit both pilot1wrappers and pilot2wrappers? Then I guess we may need to name the txt file to another name other than r0pkg.txt? (since we used this file name for pilot1wrappers)

That's a good point and I'd like to discuss that more in our next meeting. Give that the app is meant to accompany the static files from pilot 1, it makes more sense now that I think about it to not only make this bundle have a different name, but also to assume that the reviewer has already unpacked the pilot 1 files before unpacking the pilot 2 bundle and running the app.

do we need to store the xpt files again under inst/adam?

If we can assume the user has unpacked the pilot 1 files, then it is easy for me to set an overall config option within the {golem} infrastructure where the user can specify the directory path of the .xpt data files. Excluding the data sets from the bundle reduces the size dramatically.

[fb-elong]

Thanks for the detail information. Here are my findings while running on RStudio Cloud. The RStudio cloud project has been shared to [email protected] for issue checking.

1. DESCRIPTION file, R version (Update DESCRIPTION #35)
2. Could we be consistent on R version throughout the package?

https://github.com/RConsortium/submissions-wg/blob/02efb6a56d4ae201f56e7e047407924f82fc9e21/renv.lock#L3

submissions-pilot2/vignettes/ectd/r1pkg.txt

Line 1375 in eee7e94

"Version": "4.2.0",

submissions-pilot2/vignettes/adrg-prepare.Rmd

Line 35 in eee7e94

The Shiny application in this pilot was created within R version 4.2.0, and the [`{renv}`](https://rstudio.github.io/renv/index.html) package was used to manage the application dependencies to maintain a reproducible environment. All packages and their associated versions are listed in the table below.

3. (optional) submissions-pilot2.Rproj should be <package-name>.Rproj.

4. the sentence needs to be updated.

   submissions-pilot2/vignettes/adrg-prepare.Rmd

   Line 53 in eee7e94

   The code below assembles the application's code and supporting files into a custom text file using `{pkglite}` and saved into `ectd/r1pk1.txt`. Please note that the size of this text file is approximately 37 megabytes due to the included data sets within the application.

5. After running renv::restore(prompt = FALSE), I get message below but no package are installed that fail reproduce the project.

6. We need to be careful for the ". In the post of this issue, it is not the right character such as install.packages(c(“remotes”, “pkglite”))

• The library is already synchronized with the lockfile.

[fb-elong]

Is there a way to avoid the step below? We should be inclusive for people who are using other editors.

Within the RStudio IDE, create a new project in the pilot2wrappers directory using the following procedure: Select File -> New Project, choose “Existing Directory”, and browse to the pilot2wrappers location on your computer. Then click “Create Project.”

[fb-elong]

The step Define Data Files Directory is not self-contained.

Should we use pilot2-fda repo for dry run that following eCTD folder structure?

https://github.com/RConsortium/submissions-pilot2-to-fda

[fb-elong]

I am able to rerun the app from Windows following the instruction. Below are my findings after running in Windows:

1. The description of footnote in the "Primary table" should be provided. i.e. [1]-[3] next to the pvalue rows.
2. Efficacy table, there is no need for the "*" next to abbreviation.
3. Efficacy table, for the 95%CI column, could we make it wider to fit into one row.
4. Is this part necessary? Pid:12580 Token:8ca82072 in the App Information page?
5. While running the app, I got warning below. Is it possible to address it?

Warning in min > value || max < value :
  'length(x) = 2 > 1' in coercion to 'logical(1)'
Warning in min > value || max < value :
  'length(x) = 2 > 1' in coercion to 'logical(1)'

Below are my session InFo.

R version 4.2.0 (2022-04-22 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8 
[2] LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
[1] shiny_1.7.1          pilot2wrappers_0.3.0

loaded via a namespace (and not attached):
  [1] colorspace_2.0-2     ellipsis_0.3.2       rprojroot_2.0.2     
  [4] estimability_1.3     markdown_1.1         fs_1.5.2            
  [7] rstudioapi_0.13      roxygen2_7.1.2       farver_2.1.0        
 [10] remotes_2.4.2        golem_0.3.1          fansi_1.0.2         
 [13] mvtnorm_1.1-3        xml2_1.3.3           splines_4.2.0       
 [16] cachem_1.0.6         knitr_1.37           config_0.3.1        
 [19] pkgload_1.2.4        jsonlite_1.7.3       teal_0.11.1         
 [22] tippy_0.1.0          readr_2.1.2          compiler_4.2.0      
 [25] emmeans_1.7.2        backports_1.4.1      Matrix_1.3-4        
 [28] assertthat_0.2.1     fastmap_1.1.0        cli_3.1.1           
 [31] later_1.3.0          htmltools_0.5.2      prettyunits_1.1.1   
 [34] tools_4.2.0          gtable_0.3.0         glue_1.6.1          
 [37] dplyr_1.0.7          Rcpp_1.0.8           jquerylib_0.1.4     
 [40] vctrs_0.3.8          teal.data_0.1.1      teal.logger_0.1.0   
 [43] crosstalk_1.2.0      xfun_0.29            stringr_1.4.0       
 [46] ps_1.6.0             brio_1.1.3           testthat_3.1.2      
 [49] mime_0.12            lifecycle_1.0.1      renv_0.15.2         
 [52] scales_1.1.1         hms_1.1.1            promises_1.2.0.1    
 [55] rtables_0.5.1.2      huxtable_5.4.0       yaml_2.2.2          
 [58] teal.slice_0.1.1     teal.transform_0.1.1 ggplot2_3.3.5       
 [61] teal.widgets_0.1.1   sass_0.4.0           stringi_1.7.6       
 [64] desc_1.4.0           checkmate_2.0.0      teal.reporter_0.1.0 
 [67] pkgbuild_1.3.1       attempt_0.3.1        rlang_1.0.0         
 [70] pkgconfig_2.0.3      commonmark_1.7       lattice_0.20-45     
 [73] evaluate_0.14        fontawesome_0.2.2    purrr_0.3.4         
 [76] htmlwidgets_1.5.4    labeling_0.4.2       cowplot_1.1.1       
 [79] processx_3.5.2       tidyselect_1.1.1     here_1.0.1          
 [82] logger_0.2.2         magrittr_2.0.2       R6_2.5.1            
 [85] generics_0.1.2       DBI_1.1.2            pillar_1.6.5        
 [88] haven_2.4.3          withr_2.4.3          survival_3.2-13     
 [91] tibble_3.1.6         reactable_0.2.3      crayon_1.4.2        
 [94] shinyWidgets_0.6.3   utf8_1.2.2           tzdb_0.2.0          
 [97] Tplyr_0.4.4          rmarkdown_2.11       usethis_2.1.5       
[100] grid_4.2.0           reactR_0.4.4         visR_0.2.0          
[103] callr_3.7.0          formatters_0.3.1     forcats_0.5.1       
[106] digest_0.6.29        xtable_1.8-4         tidyr_1.1.4         
[109] httpuv_1.6.5         munsell_0.5.0        dockerfiler_0.1.4   
[112] bslib_0.3.1          shinyjs_2.1.0 

[parmsam-pfizer]

Great job with the instructions, @rpodcast. The setup worked for me. As @fb-elong mentioned, some folks may have problems running the left and right double quotation marks. I had to convert those to quotation marks on my Windows PC.

[robertdevine]

Excellent instructions, @rpodcast. The tests worked in Windows VM using Method 1 and Method 2
launches. The manual instructions provide a basis for secured end-to-end packaging and containerization
automation.

There was a warning during the renv package installation step and several during operation of the launched
Pilot2 Shiny app inside the VM. Despite the console warnings, the plots, data sets, and Shiny app text appeared
to display correctly. Thanks.