Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check found NA's in TADA_FlagContinuousData's TADA.ContinuousData.Flag #484

Open
8 tasks
cristinamullin opened this issue Jun 12, 2024 · 1 comment · May be fixed by #549
Open
8 tasks

Check found NA's in TADA_FlagContinuousData's TADA.ContinuousData.Flag #484

cristinamullin opened this issue Jun 12, 2024 · 1 comment · May be fixed by #549

Comments

@cristinamullin
Copy link
Collaborator

Describe the bug

TADA_FlagContinuousData sometimes populates TADA.ContinuousData.Flag with all NA's.

══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-ResultFlagsIndependent.R:67:3'): No NA's in independent flag columns ──
Error in `if (!is.na(noncont.data$ActivityStartDateTime[i])) {
    info_match <- which(noncont.data$TADA.LatitudeMeasure == 
        noncont.data$TADA.LatitudeMeasure[i] & noncont.data$TADA.LongitudeMeasure == 
        noncont.data$TADA.LongitudeMeasure[i] & noncont.data$OrganizationIdentifier == 
        noncont.data$OrganizationIdentifier[i] & noncont.data$TADA.ComparableDataIdentifier == 
        noncont.data$TADA.ComparableDataIdentifier[i] & ((noncont.data$TADA.ActivityDepthHeightMeasure.MeasureValue == 
        noncont.data$TADA.ActivityDepthHeightMeasure.MeasureValue[i]) | 
        (is.na(noncont.data$TADA.ActivityDepthHeightMeasure.MeasureValue) & 
            is.na(noncont.data$TADA.ActivityDepthHeightMeasure.MeasureValue[i]))) & 
        ((noncont.data$TADA.ResultDepthHeightMeasure.MeasureValue == 
            noncont.data$TADA.ResultDepthHeightMeasure.MeasureValue[i]) | 
            (is.na(noncont.data$TADA.ResultDepthHeightMeasure.MeasureValue) & 
                is.na(noncont.data$TADA.ResultDepthHeightMeasure.MeasureValue[i]))) & 
        ((noncont.data$TADA.ActivityTopDepthHeightMeasure.MeasureValue == 
            noncont.data$TADA.ActivityTopDepthHeightMeasure.MeasureValue[i]) | 
            (is.na(noncont.data$TADA.ActivityTopDepthHeightMeasure.MeasureValue) & 
                is.na(noncont.data$TADA.ActivityTopDepthHeightMeasure.MeasureValue[i]))) & 
        ((noncont.data$TADA.ActivityBottomDepthHeightMeasure.MeasureValue == 
            noncont.data$TADA.ActivityBottomDepthHeightMeasure.MeasureValue[i]) | 
            (is.na(noncont.data$TADA.ActivityBottomDepthHeightMeasure.MeasureValue) & 
                is.na(noncont.data$TADA.ActivityBottomDepthHeightMeasure.MeasureValue[i]))) & 
        ((noncont.data$ActivityRelativeDepthName == noncont.data$ActivityRelativeDepthName[i]) | 
            (is.na(noncont.data$ActivityRelativeDepthName) & 
                is.na(noncont.data$ActivityRelativeDepthName[i]))))
    time_diff <- abs(difftime(noncont.data$ActivityStartDateTime[i], 
        noncont.data$ActivityStartDateTime[info_match], units = "hours"))
    within_window <- info_match[time_diff <= time_difference]
    info_match <- intersect(info_match, within_window)
    if (length(info_match) >= 1) {
        noncont.data$TADA.ContinuousData.Flag[info_match] <- "Continuous"
    }
}`: argument is of length zero
Backtrace:
    ▆
 1. └─EPATADA::TADA_FlagContinuousData(testdat, clean = FALSE, flaggedonly = FALSE) at test-ResultFlagsIndependent.R:67:3

Expected behavior

The flag field should not include NA's.

Reminders for TADA contributors addressing this issue

Bug fixes should include all the following work:

  • Create or edit the function/code.

  • Document all code using line/inline and/or multi-line/block comments
    to describe what is does.

  • Create or edit tests in tests/testthat folder to help prevent and/or
    troubleshoot potential future issues.

  • Create or edit the function documentation. Include working
    examples.

  • Update or add the new functionality to the appropriate vignette
    (or create new one).

  • If function/code edits made as part of this issue impact other
    functions in the package or functionality in the shiny app, ensure
    those are updated as well.

  • Run TADA_UpdateAllRefs(), TADA_UpdateExampleData(), styler::style_pkg(),
    devtools::document(), and devtools::check() and address any new notes or
    issues before creating a pull request.

  • Run more robust check for releases: devtools::check(manual = TRUE,
    remote = TRUE, incoming = TRUE)
@cristinamullin
Copy link
Collaborator Author

Here are a few example retrieval calls from this workflow run that might reproduce this error:
https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106583316

$statecode 
[1479](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1481)[1] "US:47" 
[1480](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1482) 
[1481](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1483)$startDate 
[1482](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1484)[1] "2007-12-06" 
[1483](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1485) 
[1484](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1486)$endDate 
[1485](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1487)[1] "2007-12-11"

[1] "US:56" 
[1440](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1442) 
[1441](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1443)$startDate 
[1442](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1444)[1] "2021-07-31" 
[1443](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1445) 
[1444](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1446)$endDate 
[1445](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1447)[1] "2021-08-03"

$statecode 
[1395](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1397)[1] "US:49" 
[1396](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1398) 
[1397](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1399)$startDate 
[1398](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1400)[1] "2023-01-01" 
[1399](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1401) 
[1400](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1402)$characteristicName 
[1401](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1403)[1] "Ammonia" "Nitrate" "Nitrogen" 
[1402](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1404) 
[1403](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1405)$organization 
[1404](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1406)[1] "USGS-UT" 
[1405](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1407) 
[1406](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1408)$endDate 
[1407](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1409)[1] "2023-03-01"

startDate endDate statecode 
[1355](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1357)"2013-03-20" "2013-03-21" "CO" 
[1356](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1358)[1] "Downloading WQP query results. This may take some time depending upon the query size." 
[1357](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1359)$statecode 
[1358](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1360)[1] "US:08" 
[1359](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1361) 
[1360](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1362)$startDate 
[1361](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1363)[1] "2013-03-20" 
[1362](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1364) 
[1363](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1365)$endDate 
[1364](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1366)[1] "2013-03-21"


$statecode 
[1314](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1316)[1] "US:20" 
[1315](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1317) 
[1316](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1318)$startDate 
[1317](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1319)[1] "2021-01-01" 
[1318](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1320) 
[1319](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1321)$characteristicName 
[1320](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1322)[1] "Phosphorus" "Nitrate" 
[1321](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1323) 
[1322](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1324)$endDate 
[1323](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1325)[1] "2022-01-01"


$startDate 
[1280](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1282)[1] "2005-12-01" 
[1281](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1283) 
[1282](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1284)$endDate 
[1283](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1285)[1] "2005-12-02"


$startDate 
[1244](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1246)[1] "2011-04-28" 
[1245](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1247) 
[1246](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1248)$endDate 
[1247](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1249)[1] "2011-04-29"

$statecode 
[1202](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1204)[1] "US:22" 
[1203](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1205) 
[1204](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1206)$startDate 
[1205](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1207)[1] "2020-06-17" 
[1206](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1208) 
[1207](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1209)$endDate 
[1208](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1210)[1] "2020-06-18"

$statecode 
[1161](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1163)[1] "US:02" 
[1162](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1164) 
[1163](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1165)$startDate 
[1164](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1166)[1] "2014-03-15" 
[1165](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1167) 
[1166](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1168)$endDate 
[1167](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1169)[1] "2014-03-16"


$statecode 
[1114](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1116)[1] "US:49" 
[1115](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1117) 
[1116](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1118)$startDate 
[1117](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1119)[1] "2022-06-13" 
[1118](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1120) 
[1119](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1121)$characteristicName 
[1120](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1122)[1] "Nitrate" "Copper" 
[1121](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1123) 
[1122](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1124)$sampleMedia 
[1123](https://github.com/USEPA/EPATADA/actions/runs/9475373423/job/26106582834#step:6:1125)[1] "Water"

@hillarymarler hillarymarler linked a pull request Nov 29, 2024 that will close this issue
Update continuous data flagging #549
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update continuous data flagging USEPA/EPATADA
1 participant
@cristinamullin