Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Action Repair #144

Merged
merged 77 commits into from
Apr 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
28da363
Adding myself as an author
njlyon0 Mar 21, 2024
6808dca
Re-documenting to create documentation for new functions and update r…
njlyon0 Mar 21, 2024
827dfe0
Fixed file paths, silenced a title mistmatch issue, and renamed chunk…
njlyon0 Mar 21, 2024
1b95e04
Moving exploratory stuff to a dedicated subfolder that has its own RE…
njlyon0 Mar 21, 2024
da1791f
Alphabetized the dependencies (easier to quickly scan through)
njlyon0 Mar 21, 2024
9699a02
Adding XML as an official dependency
njlyon0 Mar 21, 2024
1c93ed2
Replacing deprecated `tidyr::spread` with `tidyr::pivot_wider`
njlyon0 Mar 21, 2024
b61fda8
Removing rlang as a dependency (warning returned that we don't actual…
njlyon0 Mar 21, 2024
748ea84
Squelching 'visible bindings' NOTE
njlyon0 Mar 21, 2024
388df57
Fixing issue with non-unique vignette names
njlyon0 Mar 21, 2024
ba130ad
Fixing typo in a folder name to be ignored by package build
njlyon0 Mar 21, 2024
9bec4c8
Specifying implied roxygen fields
njlyon0 Mar 21, 2024
0f29b13
`check_version` changes: specified arguments, added comments, and cle…
njlyon0 Mar 21, 2024
e261e2e
Adding 'internal' keyword to EML/ISO download helper functions
njlyon0 Mar 21, 2024
ad1e0d9
Successfully rebuilt pkgdown locally
njlyon0 Mar 21, 2024
4f02281
First pass at updating R-CMD check GitHub action YAML
njlyon0 Mar 21, 2024
f29e1fe
Experimenting with GitHub action for pkgdown build on PR (if works, c…
njlyon0 Mar 21, 2024
d071be4
Attempting update of pkgdown site built GitHub action
njlyon0 Mar 21, 2024
e80aaff
Tweaking dependency location for pkgdown build
njlyon0 Mar 21, 2024
00b34da
Deleting existing pkgdown docs / GitHub action to try a clean reset o…
njlyon0 Mar 21, 2024
246fb8a
Attempting to reset pkgdown GitHub action with `usethis::use_pkgdown_…
njlyon0 Mar 21, 2024
13e3f7f
Cleaning up/organizing gitignore (same content ignored just visually …
njlyon0 Mar 21, 2024
b621b26
Putting old pkgdown YAML content back (ruled it out as source of GitH…
njlyon0 Mar 21, 2024
ca0be62
Supressing vignette title/YAML title mismatch warning
njlyon0 Mar 21, 2024
9af1d96
Renaming two inconsistently formatted vignette files
njlyon0 Mar 21, 2024
85376c2
Standardized vignette names to remove warning in pkgdown build about …
njlyon0 Mar 21, 2024
888133c
Fixed a `gsub` that was removing the file extension from certain files
njlyon0 Mar 22, 2024
5509d99
`download_d1_data` contained a hard coded warning that could not be a…
njlyon0 Mar 22, 2024
402d38d
EML download function lacked a parameter it required
njlyon0 Mar 22, 2024
fbc5803
Adding needed parameter to EML download subfunction
njlyon0 Mar 22, 2024
f8c79ca
Added a comment in the `check_version` test
njlyon0 Apr 10, 2024
dcd1335
Testing possible solve to `download_EML_data` failure to identify ID …
njlyon0 Apr 10, 2024
298c6af
Fixed a warning about previously downloaded data (makes sense to just…
njlyon0 Apr 10, 2024
4d7a654
Cleaned up old directory name assembly
njlyon0 Apr 10, 2024
a9d8e9b
Added argument names to `download_d1_data` test
njlyon0 Apr 10, 2024
03def6d
Attempting work around on `download_EML` and `download_ISO` to avoid …
njlyon0 Apr 10, 2024
ad9e0f2
Added more alternates for various places particular dataset ID can be…
njlyon0 Apr 10, 2024
f3a2b10
Caught/removed superseded data download chunk that was causing errors…
njlyon0 Apr 10, 2024
1f2cd7e
Fixing incorrect `testhat` behavior expectation
njlyon0 Apr 10, 2024
2f2291f
Simplifying success message
njlyon0 Apr 10, 2024
64bf51e
Cleaning up message line breaks and defining arguments in test cases …
njlyon0 Apr 10, 2024
104bdc3
Added useful comment and an error for data without metadata
njlyon0 Apr 10, 2024
6b9378c
Fixed more incorrect expectations in `download_d1_data` and specified…
njlyon0 Apr 10, 2024
17a6d34
Tweaking to handle EDI data (hopefully) without issue
njlyon0 Apr 10, 2024
14c5135
Removing more unrealistic tests and specifying more places where warn…
njlyon0 Apr 10, 2024
e839ae5
Fixing Julien's email as the `metajam` maintainer
njlyon0 Apr 10, 2024
bf8ab7e
Removing more unrealistic `download_d1_data` tests
njlyon0 Apr 10, 2024
400aba5
Issue with `...d1_data_pkg` function testing was just an obsoleted id…
njlyon0 Apr 10, 2024
3dc9a57
Removed unreasonable tests in `download_EML_data` test script
njlyon0 Apr 10, 2024
c4e3032
Removed bad test for `download_ISO_data` function
njlyon0 Apr 10, 2024
03b608d
Commented out `read_d1_files` and reworked interior to be more stream…
njlyon0 Apr 10, 2024
ba8b02c
Tweaked `read_d1_files` tests to reflect new status of function
njlyon0 Apr 10, 2024
b2d79c2
Removing commented-out/superseded internal mechanics of `read_d1_files`
njlyon0 Apr 10, 2024
b7d8894
Adding some empty lines to the bottom of a script
njlyon0 Apr 10, 2024
0bbd55f
Reordering/streamlining the `DESCRIPTION` and re-documenting the pack…
njlyon0 Apr 10, 2024
fef70ca
Simplifying pkgdown YAML to rule out sources of GHA error
njlyon0 Apr 10, 2024
ff45dba
Renaming vignette files to be machine sorted in desired order
njlyon0 Apr 10, 2024
0b26d89
Updating R-CMD check GitHub action with `use_github_action(name = "ch…
njlyon0 Apr 10, 2024
8a9699e
Changing package build badge source on README
njlyon0 Apr 10, 2024
b1819e3
Removing orphaned package suggestion (`udunits2`) as I am unable to f…
njlyon0 Apr 10, 2024
6208b68
Adding `udunits2` pack in as a suggestion because we get a different …
njlyon0 Apr 10, 2024
eb25f31
Restructuring vignette article 3 to avoid the `udunits2` package so t…
njlyon0 Apr 10, 2024
0a3073b
Removed `udunits2` as a dependency again but this time I think I've a…
njlyon0 Apr 10, 2024
467531b
Adding system requirements to `DESCRIPTION` per Matt Jones' advice
njlyon0 Apr 11, 2024
359b2e5
Adding an R version dependency
njlyon0 Apr 11, 2024
bebf92c
Reducing R version dependency to exactly match `redland` version numb…
njlyon0 Apr 11, 2024
f13375d
Adding a change to GHA YAMLs per this issue's recommendation: https:/…
njlyon0 Apr 11, 2024
e677968
Trying to change how Linux handles dependencies
njlyon0 Apr 11, 2024
df950cf
Reordering special forced install packages (to match order of appeara…
njlyon0 Apr 11, 2024
7892830
Removing `libcurl4-openssl-dev` as that seems to have conflicted with…
njlyon0 Apr 11, 2024
6a17c65
Replacing `file.remove` with `unlink` to (hopefully) avoid error in W…
njlyon0 Apr 11, 2024
29cbcc0
Fixing obsoleted DOI in `get_pkg_ids`
njlyon0 Apr 11, 2024
a235b90
Changing `read_d1_files` function to avoid a hard-coded OS-specific s…
njlyon0 Apr 11, 2024
78922f2
Tweaking `paste0`-based assembly of user-defined reading function wit…
njlyon0 Apr 11, 2024
fe05006
Finding and resolving other OS-specific hard-coding of file path slas…
njlyon0 Apr 11, 2024
10fc577
Removing periods from created folder elements (hopefully fixes weird …
njlyon0 Apr 11, 2024
c0f8a63
Reworking OS-agnostic file path parsing inside of `read_d1_files`
njlyon0 Apr 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
^Meta$
^doc$
^docs$
^dev$
^_pkgdown\.yml$
^.*\.Rproj$
^\.Rproj\.user$
Expand Down
63 changes: 41 additions & 22 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
@@ -1,35 +1,54 @@
# For help debugging build failures open an issue on the RStudio community with the 'github-actions' tag.
# https://community.rstudio.com/new-topic?category=Package%20development&tags=github-actions
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches:
- dev
- main
branches: [main, master]
pull_request:
branches:
- dev
- main
branches: [main, master]

name: R-CMD-check

jobs:
R-CMD-check:
runs-on: macOS-latest
runs-on: ${{ matrix.config.os }}

name: ${{ matrix.config.os }} (${{ matrix.config.r }})

strategy:
fail-fast: false
matrix:
config:
- {os: macos-latest, r: 'release'}
- {os: windows-latest, r: 'release'}
- {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'}
- {os: ubuntu-latest, r: 'release'}
- {os: ubuntu-latest, r: 'oldrel-1'}

env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes
PKG_SYSREQS: false

steps:
- uses: actions/checkout@v2
- uses: r-lib/actions/setup-pandoc@v1
- uses: r-lib/actions/setup-r@v1
- uses: actions/checkout@v4

- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.config.r }}
http-user-agent: ${{ matrix.config.http-user-agent }}
use-public-rspm: true
- name: Install udunits
run: brew install udunits
- name: Install dependencies
run: |
install.packages(c("remotes", "rcmdcheck"))
remotes::install_deps(dependencies = TRUE)
shell: Rscript {0}
- name: Check
run: rcmdcheck::rcmdcheck(args = "--no-manual", error_on = "error")
shell: Rscript {0}

- name: dependencies on Linux
if: runner.os == 'Linux'
run: sudo apt-get install -y libcurl4-gnutls-dev libicu-dev libjq-dev libnode-dev librdf0-dev libssl-dev libxml2-dev make pandoc

- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: any::rcmdcheck
needs: check

- uses: r-lib/actions/check-r-package@v2
with:
upload-snapshots: true
42 changes: 29 additions & 13 deletions .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Workflow derived from https://github.com/r-lib/actions/tree/master/examples
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches: [main, master]
pull_request:
branches: [main, master]
release:
types: [published]
workflow_dispatch:
Expand All @@ -12,27 +14,41 @@ name: pkgdown
jobs:
pkgdown:
runs-on: ubuntu-latest
# Only restrict concurrency for non-PR jobs
concurrency:
group: pkgdown-${{ github.event_name != 'pull_request' || github.run_id }}
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes
PKG_SYSREQS: false
permissions:
contents: write
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- uses: r-lib/actions/setup-pandoc@v1
- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v1
- uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true

- name: Install udunit2
run: sudo apt-get install libudunits2-dev
- name: dependencies on Linux
if: runner.os == 'Linux'
run: sudo apt-get install -y make libicu-dev libxml2-dev libssl-dev pandoc librdf0-dev libnode-dev libcurl4-gnutls-dev

- uses: r-lib/actions/setup-r-dependencies@v1
- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: pkgdown
extra-packages: any::pkgdown, local::.
needs: website

- name: Deploy package
run: |
git config --local user.name "$GITHUB_ACTOR"
git config --local user.email "[email protected]"
Rscript -e 'pkgdown::deploy_to_branch(github_pages = (branch == "gh-pages"), new_process = FALSE)'
- name: Build site
run: pkgdown::build_site_github_pages(new_process = FALSE, install = FALSE)
shell: Rscript {0}

- name: Deploy to GitHub pages 🚀
if: github.event_name != 'pull_request'
uses: JamesIves/[email protected]
with:
clean: false
branch: gh-pages
folder: docs
14 changes: 11 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,17 @@
Meta/
doc/
# Background R stuff
.Rproj.user
.Rhistory
.RData
.Ruserdata
.httr-oauth
.DS_Store

# R project
*.Rproj

# Certain folders
Meta/
doc/
docs/

# Ignore Mac file
.DS_Store
26 changes: 16 additions & 10 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ Package: metajam
Type: Package
Title: Easily Download Data and Metadata from 'DataONE'
Version: 0.2.3
Date: 2020-10-14
Authors@R: c(
person("Julien", "Brun",
email = "[email protected]",
Expand All @@ -15,6 +14,11 @@ Authors@R: c(
person("Mitchell", "Maier",
comment= c(ORCID = "0000-0001-6955-0535"),
role = "aut"),
person("Nicholas", "Lyon",
email = "[email protected]",
comment = c(ORCID = "0000-0003-3905-1078",
"https://njlyon0.github.io/"),
role = "ctb"),
person("Nathan", "Hwangbo",
email = "[email protected]",
comment= c(ORCID = "0000-0002-3777-0821"),
Expand All @@ -32,13 +36,19 @@ Authors@R: c(
comment = c(ORCID = "0000-0002-6388-0901"),
role = "ctb")
)
Maintainer: Julien Brun <brun@nceas.ucsb.edu>
Maintainer: Julien Brun <jb160@ucsb.edu>
Description: A set of tools to foster the development of reproducible analytical workflow by simplifying the download of data and
metadata from 'DataONE' (<https://www.dataone.org>) and easily importing this information into R.
License: Apache License (== 2.0)
URL: https://github.com/nceas/metajam,
https://nceas.github.io/metajam/
Encoding: UTF-8
Language: en-US
RoxygenNote: 7.2.3
SystemRequirements: Mac OSX: redland (>= 1.0.14) ; Linux: librdf0 (>= 1.0.14),
librdf0-dev (>= 1.0.14)
URL: https://nceas.github.io/metajam/
BugReports: https://github.com/nceas/metajam/issues
Depends:
R (>= 3.1.1)
Imports:
dataone,
dplyr,
Expand All @@ -51,13 +61,9 @@ Imports:
stringr,
tibble,
tidyr,
rlang
XML
Suggests:
testthat,
knitr,
rmarkdown,
udunits2
Encoding: UTF-8
testthat
VignetteBuilder: knitr
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.1
2 changes: 1 addition & 1 deletion NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ importFrom(dataone,query)
importFrom(emld,as_emld)
importFrom(lubridate,ymd_hms)
importFrom(purrr,"%||%")
importFrom(rlang,.data)
importFrom(stats,setNames)
importFrom(stringr,str_detect)
importFrom(stringr,str_extract)
importFrom(stringr,str_trim)
importFrom(tibble,enframe)
importFrom(tidyr,pivot_wider)
importFrom(tidyr,spread)
importFrom(tools,file_path_sans_ext)
importFrom(utils,URLdecode)
42 changes: 19 additions & 23 deletions R/check_version.R
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#' Check PID version
#' @title Check PID version
#'
#' This function takes an identifier and checks to see if it has been obsoleted.
#' @description This function takes an identifier and checks to see if it has been obsoleted.
#'
#' @param pid (character) The persistent identifier of a data, metadata, or resource map object on a DataONE member node.
#' @param formatType (character) Optional. The format type to return (one of data, metadata, or resource).
#' @param formatType (character) Optional. The format type to return (one of 'data', 'metadata', or 'resource').
#'
#' @return (data.frame) A data frame of object version PIDs and related information.
#'
Expand Down Expand Up @@ -32,28 +32,28 @@

check_version <- function(pid, formatType = NULL) {

if (!all(is.character(pid), all(nchar(pid) > 0))) {
stop("Argument 'pids' must be character class with non-zero number of characters.")
}
if (!is.null(formatType) && !all(is.character(formatType), length(formatType) == 1, formatType %in% c("data", "metadata", "resource"))) {
# Error out for missing pids
if(all(is.character(pid), nchar(pid) > 0) != TRUE)
stop("Argument 'pid' must be character class with non-zero number of characters.")

# Error out for missing / unsupported `formatType` specification
if(is.null(formatType) != TRUE & all(is.character(formatType), length(formatType) == 1, formatType %in% c("data", "metadata", "resource")) != TRUE)
stop("Argument 'formatType' should either be NULL or one of 'data', 'metadata', or 'resource'.")
}

# Query DataONE for the specified `pid`
while (nchar(pid) > 5) {
results <- suppressMessages(
dataone::query(dataone::CNode(),
list(q = sprintf('identifier:"%s"', pid),
fl = "identifier, dateUploaded, formatType, obsoletedBy, resourceMap"),
dataone::query(x = dataone::CNode(),
solrQuery = list(q = sprintf('identifier:"%s"', pid),
fl = "identifier, dateUploaded, formatType, obsoletedBy, resourceMap"),
as = "data.frame")
)
#if results is null or empty dataframe, remove part of the URI
# If results is null or empty dataframe, remove part of the URI
if (is.null(results) || nrow(results) == 0) {
pid <- gsub("^[^/=]+[/=]*", "", pid)
pid <- gsub(pattern = "^[^/=]+[/=]*", replacement = "", x = pid)

} else {
#what to do if multiple are returned
break
}
# If multiple are returned, break
} else { break }
}

# filter out extra types (resource map/etc with similar pid)
Expand All @@ -66,16 +66,12 @@ check_version <- function(pid, formatType = NULL) {
stop("No matching identifiers were found.")
} else if (nrow(results) == 1) {
if (is.null(results$obsoletedBy) || is.na(results$obsoletedBy)) {
message("\n",
results$identifier,
"\nis the latest version for identifier\n",
pid)
message(results$identifier, " is the latest version for the provided persistent identifier")
} else {
warning("The identifier has been obsoleted by ", results$obsoletedBy)
}
} else {
warning("Several identifiers are associated with ", pid)
}

return(results)
}
return(results) }
Loading