Skip to content

Commit

Permalink
Update expected frequencies for functional tests
Browse files Browse the repository at this point in the history 
Updates the specific values of expected frequencies for functional
tests, since the corrected pivot calculation logic produces new pivots
and correspondingly new frequency values per pivot. One nice side effect
of the new pivot calculations is that they produce the correct number of
pivots in our functional test of relative min and max date values that
had previously failed in a flaky manner. The corrected pivot logic
reveals that the "flaky" behavior of that test was actually the correct
behavior: we should always get 3 pivots when the max date is 6 months
ago, the min date is 12 months ago, and the pivot interval is 3 months.
That we previously only got 2 pivots from these relative dates most
times of the year reflects the invalid pivot calculation logic from a
different perspective. This commit reactivates that previously
commented-out test.

Fixes #963
  • Loading branch information
@huddlej
huddlej committed Jan 9, 2023
1 parent 7e9f8fe commit 6a3d7c0
Show file tree
Hide file tree
Showing 5 changed files with 146 additions and 137 deletions.
90 changes: 45 additions & 45 deletions tests/builds/zika/auspice/zika_tip-frequencies.json
View file
Original file line number Diff line number Diff line change
@@ -1,92 +1,92 @@
{
"BRA/2016/FC_6706": {
"frequencies": [
0.022579,
0.011312,
0.206983,
0.102282
0.022368,
0.011759,
0.207309,
0.102551
]
},
"COL/FLR_00008/2015": {
"frequencies": [
0.243453,
0.208443,
0.008645,
0.010659
0.244555,
0.204597,
0.008456,
0.010737
]
},
"Colombia/2016/ZC204Se": {
"frequencies": [
0.126056,
0.231597,
0.014167,
0.017099
0.12489,
0.234574,
0.013671,
0.017245
]
},
"DOM/2016/BB_0183": {
"frequencies": [
0.017888,
0.009342,
0.183671,
0.148759
0.017736,
0.009645,
0.185763,
0.148494
]
},
"EcEs062_16": {
"frequencies": [
0.018981,
0.009763,
0.190994,
0.134398
0.018817,
0.010092,
0.192704,
0.134289
]
},
"HND/2016/HU_ME59": {
"frequencies": [
0.009484,
0.006254,
0.090552,
0.457824
0.009425,
0.006444,
0.093589,
0.456546
]
},
"PAN/CDC_259359_V1_V3/2015": {
"frequencies": [
0.224832,
0.214575,
0.008963,
0.011176
0.225577,
0.211235,
0.008761,
0.011258
]
},
"PRVABC59": {
"frequencies": [
0.243453,
0.208443,
0.008645,
0.010659
0.244555,
0.204597,
0.008456,
0.010737
]
},
"VEN/UF_1/2016": {
"frequencies": [
0.030662,
0.016483,
0.206983,
0.070196
0.030336,
0.017436,
0.204461,
0.07079
]
},
"ZKC2/2016": {
"frequencies": [
0.062612,
0.083787,
0.080395,
0.036947
0.06174,
0.089622,
0.076833,
0.037353
]
},
"generated_by": {
"program": "augur",
"version": "7.0.2"
"version": "19.2.0"
},
"pivots": [
2015.75,
2016.0,
2016.25,
2016.5
2015.7521,
2016.0041,
2016.2527,
2016.5014
]
}
44 changes: 21 additions & 23 deletions tests/functional/frequencies.t
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,26 +34,24 @@ Pivots get calculated from the requested date range.
Calculate KDE-based tip frequencies for a time period with relative dates.
Testing relative dates deterministically from the shell is tricky.
To keep these tests simple and avoid freezing the system date to specific values, this test checks the logical consistency of the requested relative dates and pivot interval.
With a minimum date of 1 year (12 months) ago, a maximum date of 6 months ago, and a pivot interval of 3 months, we expect only 2 pivots in the final output.
Since the test data are much older than the time period requested, all strains will always have frequencies of 0 for the 2 requested pivots.
As long as we always calculate 2 pivots with frequencies of 0 for all strains, we can ignore the actual pivot values calculated for the relative dates in the diff below.

TODO: un-comment this test, which is failing on 2022-06-01 with 3 pivot points: https://github.com/nextstrain/augur/runs/6694014041

$ ${AUGUR} frequencies \
> --method kde \
> --tree "frequencies/tree.nwk" \
> --metadata "frequencies/metadata.tsv" \
> --pivot-interval 3 \
> --min-date 1Y \
> --max-date 6M \
> --output "$TMP/tip-frequencies.json" > /dev/null

$ python3 "$TESTDIR/../../scripts/diff_jsons.py" \
> --exclude-paths "root['generated_by']['version']" "root['pivots']" -- \
> "frequencies/zika_tip-frequencies_with_relative_dates.json" \
> "$TMP/tip-frequencies.json"
{}
$ rm -f "$TMP/tip-frequencies.json"

$ popd > /dev/null
With a minimum date of 12 months ago, a maximum date of 6 months ago, and a pivot interval of 3 months, we always expect 3 pivots in the final output corresponding to the end date (always included), the 3-month pivot before that end date, and the start date (also should always be included).
Since the test data are much older than the time period requested, all strains will always have frequencies of 0 for the 3 requested pivots.
As long as we always calculate 3 pivots with frequencies of 0 for all strains, we can ignore the actual pivot values calculated for the relative dates in the diff below.

$ ${AUGUR} frequencies \
> --method kde \
> --tree "frequencies/tree.nwk" \
> --metadata "frequencies/metadata.tsv" \
> --pivot-interval 3 \
> --min-date 12M \
> --max-date 6M \
> --output "$TMP/tip-frequencies.json" > /dev/null

$ python3 "$TESTDIR/../../scripts/diff_jsons.py" \
> --exclude-paths "root['generated_by']['version']" "root['pivots']" -- \
> "frequencies/zika_tip-frequencies_with_relative_dates.json" \
> "$TMP/tip-frequencies.json"
{}
$ rm -f "$TMP/tip-frequencies.json"

$ popd > /dev/null
90 changes: 45 additions & 45 deletions tests/functional/frequencies/zika_tip-frequencies.json
View file
Original file line number Diff line number Diff line change
@@ -1,92 +1,92 @@
{
"BRA/2016/FC_6706": {
"frequencies": [
0.022579,
0.011312,
0.206983,
0.102282
0.022368,
0.011759,
0.207309,
0.102551
]
},
"COL/FLR_00008/2015": {
"frequencies": [
0.243453,
0.208443,
0.008645,
0.010659
0.244555,
0.204597,
0.008456,
0.010737
]
},
"Colombia/2016/ZC204Se": {
"frequencies": [
0.126056,
0.231597,
0.014167,
0.017099
0.12489,
0.234574,
0.013671,
0.017245
]
},
"DOM/2016/BB_0183": {
"frequencies": [
0.017888,
0.009342,
0.183671,
0.148759
0.017736,
0.009645,
0.185763,
0.148494
]
},
"EcEs062_16": {
"frequencies": [
0.018981,
0.009763,
0.190994,
0.134398
0.018817,
0.010092,
0.192704,
0.134289
]
},
"HND/2016/HU_ME59": {
"frequencies": [
0.009484,
0.006254,
0.090552,
0.457824
0.009425,
0.006444,
0.093589,
0.456546
]
},
"PAN/CDC_259359_V1_V3/2015": {
"frequencies": [
0.224832,
0.214575,
0.008963,
0.011176
0.225577,
0.211235,
0.008761,
0.011258
]
},
"PRVABC59": {
"frequencies": [
0.243453,
0.208443,
0.008645,
0.010659
0.244555,
0.204597,
0.008456,
0.010737
]
},
"VEN/UF_1/2016": {
"frequencies": [
0.030662,
0.016483,
0.206983,
0.070196
0.030336,
0.017436,
0.204461,
0.07079
]
},
"ZKC2/2016": {
"frequencies": [
0.062612,
0.083787,
0.080395,
0.036947
0.06174,
0.089622,
0.076833,
0.037353
]
},
"generated_by": {
"program": "augur",
"version": "7.0.2"
"version": "19.2.0"
},
"pivots": [
2015.75,
2016.0,
2016.25,
2016.5
2015.7521,
2016.0041,
2016.2527,
2016.5014
]
}
42 changes: 21 additions & 21 deletions tests/functional/frequencies/zika_tip-frequencies_with_fixed_dates.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@
},
"COL/FLR_00008/2015": {
"frequencies": [
0.342553,
0.340179,
0.337787,
0.342054,
0.330097
0.34254,
0.340199,
0.337813,
0.34201,
0.329984
]
},
"Colombia/2016/ZC204Se": {
Expand Down Expand Up @@ -55,20 +55,20 @@
},
"PAN/CDC_259359_V1_V3/2015": {
"frequencies": [
0.314894,
0.319642,
0.324426,
0.315891,
0.339807
0.314921,
0.319602,
0.324374,
0.31598,
0.340033
]
},
"PRVABC59": {
"frequencies": [
0.342553,
0.340179,
0.337787,
0.342054,
0.330097
0.34254,
0.340199,
0.337813,
0.34201,
0.329984
]
},
"VEN/UF_1/2016": {
Expand All @@ -91,13 +91,13 @@
},
"generated_by": {
"program": "augur",
"version": "14.1.0"
"version": "19.2.0"
},
"pivots": [
2015.0,
2015.25,
2015.5,
2015.75,
2016.0
2015.0014,
2015.2479,
2015.4973,
2015.7493,
2016.0014
]
}
Loading

0 comments on commit 6a3d7c0

Please sign in to comment.