-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add variational method accuracy tests to test_accuracy.py
#265
Comments
I think we are in a good place to add a few of these tests, right @nspope ? |
Sure -- but what are we testing, exactly? If it's statistical accuracy then I don't think unit tests are the right place for that -- we want a separate set of benchmarks that uses data of a reasonable size. |
The idea was simply to check if changes accidentally created any major regressions (I don't think they should). I agree that larger-scale accuracy tests should be in the unit tests. I think that's what #379 is about |
I think a few very simple tests with tree sequences with (say) 1, 2, and 5 trees of 2 and 20 samples would probably be fine. But maybe you think it's not worth it? |
Not worth it, IMO. I'd rather have a benchmark suite that could be run whenever there's a change to the low-level stuff. |
Once we have a reasonable variational method working, we should add a test or two to
test_accuracy.py
so that we pick up any regressions. Do you think you could do this @nspope ?The text was updated successfully, but these errors were encountered: