Check for QGIS version & path changes; update messages

[florisvdh]

Fixes #94, fixes #109.

Please feel free to suggest improvements.

Checking for QGIS version change in qgisprocess_cache$path when loading cache

When a cache is going to be used, the QGIS version is first compared between the cache version and that of the program in the path defined by the cache. It makes the package take a little more time to load, but this seems worth it, since conflicts are expected (outdated cached algorithms) if a version change at the same location is not captured.

If versions differ, the cache will not be used and a new cache is built. Startup messages make this clear. The reprex below first overwrites an existing cache rds file, in order to mimic an outdated cache upon loading qgisprocess.

# Artificially outdating the cache to mimic version difference
suppressPackageStartupMessages(library(qgisprocess))
cache_data_file <- qgisprocess:::qgisprocess_cache$loaded_from
qgisprocess:::qgisprocess_cache$version
#> [1] "3.26.1-Buenos Aires"
qc <- qgisprocess:::qgisprocess_cache
qc$version <- "3.22.3-Białowieża"
saveRDS(qc, cache_data_file)
detach("package:qgisprocess", unload = TRUE)

# Loading and attaching qgisprocess
library(qgisprocess)
#> QGIS version change detected:
#> - in the qgisprocess cache it was: 3.22.3-Białowieża
#> - while 'qgis_process' is at 3.26.1-Buenos Aires
#> Hence rebuilding cache to reflect this change.
#> Using 'qgis_process' at 'qgis_process'.
#> QGIS version: 3.26.1-Buenos Aires
#> Metadata of 1024 algorithms successfully cached.
#> Run `qgis_algorithms()` to see them.
#> >>> If you need another installed QGIS version, run `qgis_configure()`; see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

^{Created on 2022-08-19 by the reprex package (v2.0.1)}

Further tweaks of messages

• As seen above, startup messaging doesn't advise to run qgis_configure() when a cache has been rebuilt; instead qgis_algorithms() is referred.
• In any case, startup messaging adds a hint (after '>>>') for reconfiguring if another QGIS version is needed.
• If an existing cache was loaded, qgis_configure(use_cached_data = TRUE) is referred for more details:

library(qgisprocess)
#> Using 'qgis_process' at 'qgis_process'.
#> QGIS version: 3.26.1-Buenos Aires
#> Configuration loaded from '~/.cache/R-qgisprocess/cache-0.0.0.9000.rds'
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> If you need another installed QGIS version, run `qgis_configure()`; see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

^{Created on 2022-08-19 by the reprex package (v2.0.1)}

• Upon loading qgisprocess, qgis_configure() is called with argument quiet=TRUE (this was and remains so). Its default is FALSE; if the user calls qgis_configure(use_cached_data = TRUE), the version check is reported:

suppressPackageStartupMessages(library(qgisprocess))
qgis_configure(use_cached_data = TRUE)
#> Checking configuration from '~/.cache/R-qgisprocess/cache-0.0.0.9000.rds'
#> Checking cached QGIS version with version reported by '/usr/bin/qgis_process' ...
#> QGIS versions match! (3.26.1-Buenos Aires)
#> Restoring configuration from '~/.cache/R-qgisprocess/cache-0.0.0.9000.rds'
#> Metadata of 1024 algorithms are present in cache.
#> Run `qgis_algorithms()` to see them.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

^{Created on 2022-08-19 by the reprex package (v2.0.1)}

• If the cache is rebuilt because the qgisprocess.path option or R_QGISPROCESS_PATH environmental variable contain a different path than the cache, this is reported at startup, just like the case of a version change:

options("qgisprocess.path" = "/usr/bin/qgis_process")
library(qgisprocess)
#> The user's qgisprocess.path option or the R_QGISPROCESS_PATH environment variable specify a different qgis_process path (/usr/bin/qgis_process) than the cache did (qgis_process).
#> Hence rebuilding cache to reflect this change.
#> Using 'qgis_process' at '/usr/bin/qgis_process'.
#> QGIS version: 3.26.1-Buenos Aires
#> Metadata of 1024 algorithms successfully cached.
#> Run `qgis_algorithms()` to see them.
#> >>> If you need another installed QGIS version, run `qgis_configure()`; see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

^{Created on 2022-08-19 by the reprex package (v2.0.1)}

• Since upon loading the package using the cache, the qgis_process program as referred by the cached path is queried to do the cache-version check, the program must be available. This is checked first and if this errors, it is reported and reconfiguration is triggered:

# Artificially outdating the cache to mimic (re)moved qgis_process installation
suppressPackageStartupMessages(library(qgisprocess))
cache_data_file <- qgisprocess:::qgisprocess_cache$loaded_from
qgisprocess:::qgisprocess_cache$path
#> [1] "/usr/bin/qgis_process"
qc <- qgisprocess:::qgisprocess_cache
qc$path <- "/bin/qgis_process"
saveRDS(qc, cache_data_file)
detach("package:qgisprocess", unload = TRUE)

# Loading and attaching qgisprocess
library(qgisprocess)
#> Error in value[[3L]](cond) : 
#>   '/bin/qgis_process' (cached path) is not available anymore.
#> Will try to reconfigure qgisprocess and build new cache ...
#> Using 'qgis_process' at 'qgis_process'.
#> QGIS version: 3.26.1-Buenos Aires
#> Metadata of 1024 algorithms successfully cached.
#> Run `qgis_algorithms()` to see them.
#> >>> If you need another installed QGIS version, run `qgis_configure()`; see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

^{Created on 2022-08-19 by the reprex package (v2.0.1)}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23)
#>  os       Linux Mint 20
#>  system   x86_64, linux-gnu
#>  ui       X11
#>  language nl_BE:nl
#>  collate  nl_BE.UTF-8
#>  ctype    nl_BE.UTF-8
#>  tz       Europe/Brussels
#>  date     2022-08-19
#>  pandoc   2.18 @ /usr/lib/rstudio/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  cli           3.3.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  digest        0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  evaluate      0.16       2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi         1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  fs            1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  glue          1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  highr         0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  htmltools     0.5.3      2022-07-18 [1] RSPM (R 4.2.1)
#>  jsonlite      1.8.0      2022-02-22 [1] CRAN (R 4.2.0)
#>  knitr         1.39       2022-04-26 [1] CRAN (R 4.2.0)
#>  lifecycle     1.0.1      2021-09-24 [1] CRAN (R 4.2.0)
#>  magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  pillar        1.8.0      2022-07-18 [1] RSPM (R 4.2.1)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  processx      3.7.0      2022-07-07 [1] RSPM (R 4.2.1)
#>  ps            1.7.1      2022-06-18 [1] RSPM (R 4.2.1)
#>  qgisprocess * 0.0.0.9000 2022-08-19 [1] Github (paleolimbot/qgisprocess@f24e90a)
#>  R6            2.5.1      2021-08-19 [1] CRAN (R 4.2.0)
#>  rappdirs      0.3.3      2021-01-31 [1] CRAN (R 4.2.0)
#>  reprex        2.0.1      2021-08-05 [1] CRAN (R 4.2.0)
#>  rlang         1.0.4      2022-07-12 [1] RSPM (R 4.2.1)
#>  rmarkdown     2.14       2022-04-25 [1] CRAN (R 4.2.0)
#>  rstudioapi    0.13       2020-11-12 [1] CRAN (R 4.2.0)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi       1.7.8      2022-07-11 [1] RSPM (R 4.2.1)
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 4.2.0)
#>  tibble        3.1.8      2022-07-22 [1] RSPM (R 4.2.1)
#>  utf8          1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs         0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr         2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun          0.32       2022-08-10 [1] CRAN (R 4.2.1)
#>  yaml          2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#> 
#>  [1] /home/floris/lib/R/library
#>  [2] /usr/local/lib/R/site-library
#>  [3] /usr/lib/R/site-library
#>  [4] /usr/lib/R/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[paleolimbot]

Cacheing is complicated! I did a local checkout and I can confirm that this works on MacOS (although on MacOS the path changes so linux is a better check as you've done above. Thanks!

[florisvdh]

170dfbd avoids the odd message: Using 'qgis_process' at 'qgis_process'. It means qgis_process is called from the system PATH. Although it seems most preferable to actually communicate the full path here (more specifically: have that returned by qgis_path()), I couldn't find a single cross-platform R solution to get that (in Unixes one would run e.g. system(which(qgis_process)) and get /usr/bin/qgis_process in my case). I opted for an easier route below, just for the message, but if you have a simple answer to get the full path, let me know.

The silly aspect in below reprex is that switching from PATH to /usr/bin/qgis_process triggers a reconfiguration, while it is 2 times the same (the cached path is qgis_process in the PATH case). But anyway, performing this type of trivial changes is a rare event in practice and the consequences are not harmful. BTW this behaviour is unrelated to this PR.

library(qgisprocess)
#> Using 'qgis_process' in the system PATH.
#> QGIS version: 3.26.1-Buenos Aires
#> Configuration loaded from '~/.cache/R-qgisprocess/cache-0.0.0.9000.rds'
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> If you need another installed QGIS version, run `qgis_configure()`;
#>     see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.
detach("package:qgisprocess", unload = TRUE)

oldopt <- options("qgisprocess.path" = "/usr/bin/qgis_process")
library(qgisprocess)
#> The user's qgisprocess.path option or the R_QGISPROCESS_PATH environment variable specify a different qgis_process path (/usr/bin/qgis_process) than the cache did (qgis_process).
#> Hence rebuilding cache to reflect this change.
#> Using 'qgis_process' at '/usr/bin/qgis_process'.
#> QGIS version: 3.26.1-Buenos Aires
#> Metadata of 1024 algorithms successfully cached.
#> Run `qgis_algorithms()` to see them.
#> >>> If you need another installed QGIS version, run `qgis_configure()`;
#>     see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

detach("package:qgisprocess", unload = TRUE)

options(oldopt)
suppressPackageStartupMessages(
  library(qgisprocess) # keeps using /usr/bin/qgis_process since that works
  )
qgis_configure() # reconfigures to using PATH
#> getOption('qgisprocess.path') was not found.
#> Sys.getenv('R_QGISPROCESS_PATH') was not found.
#> Trying 'qgis_process' on PATH...
#> Success!
#> QGIS version: 3.26.1-Buenos Aires
#> Saving configuration to '~/.cache/R-qgisprocess/cache-0.0.0.9000.rds'
#> Metadata of 1024 algorithms queried and stored in cache.
#> Run `qgis_algorithms()` to see them.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

^{Created on 2022-08-19 by the reprex package (v2.0.1)}

[florisvdh]

I had been postponing the merge since I wondered what is the possible overhead of checking the version of the cached qgis_process location in Windows & MacOS, considering that the filepaths usually contain a version number in these systems, hence qgis_process should be never overwritten.

It turns out that at least the OSGeo4W installation (Windows) has a version-agnostic path, so it's worthwile to have the same procedure in place in Windows.

We could however drop this check in the case of MacOS, on condition that qgis_process will always be in a version specific path. @paleolimbot how do you see this?

Further, about the version-specific filepaths in Windows & MacOS:

• if a newer QGIS is installed and the old one is kept, qgisprocess will rely on the user to reconfigure, but the new startup message now indicates this.
• Having a newer version will not be affected by the version check of this PR, since it only checks the cached installation, without searching for newer ones.
• The latter could be an idea however: notify with something as 'You have a newer QGIS version available. To use it, run qgis_configure()'. Any preferences about this?

[paleolimbot]

I appreciate the concern, although I think the overhead of one additional call is OK (I barely noticed). The main thing that takes a long time is all generating a cache of all the algorithms and that's still skipped opportunistically. I think it's better to be safe up front!

[florisvdh]

Thanks. Further, the idea of notifying that a newer installed version is available but not in use could be tricky since it would rely on regexes to extract version numbers from different possible paths in Win & Mac, although it's doable (but is it worth the effort?). As an alternative, individual calls to each installed qgis_process to get the versions would generate further overhead or perhaps a source of new problems. For now we can leave this to the user I guess, given the message about 'if you need another version'. At least detecting version updates will be automated in Linux & OSGeo4W!

EDIT: there appear to be cases in MacOS where the path is version-agnostic too, e.g. the path outputted in #68 (comment).