Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add variational method accuracy tests to test_accuracy.py #265

Open
hyanwong opened this issue Jun 13, 2023 · 5 comments
Open

Add variational method accuracy tests to test_accuracy.py #265

hyanwong opened this issue Jun 13, 2023 · 5 comments

Comments

@hyanwong
Copy link
Member

Once we have a reasonable variational method working, we should add a test or two to test_accuracy.py so that we pick up any regressions. Do you think you could do this @nspope ?

@hyanwong
Copy link
Member Author

hyanwong commented Jun 4, 2024

I think we are in a good place to add a few of these tests, right @nspope ?

@hyanwong hyanwong mentioned this issue Jun 4, 2024
15 tasks
@nspope
Copy link
Contributor

nspope commented Jun 4, 2024

Sure -- but what are we testing, exactly? If it's statistical accuracy then I don't think unit tests are the right place for that -- we want a separate set of benchmarks that uses data of a reasonable size.

@hyanwong
Copy link
Member Author

hyanwong commented Jun 4, 2024

The idea was simply to check if changes accidentally created any major regressions (I don't think they should). I agree that larger-scale accuracy tests should be in the unit tests. I think that's what #379 is about

@hyanwong
Copy link
Member Author

hyanwong commented Jun 4, 2024

I think a few very simple tests with tree sequences with (say) 1, 2, and 5 trees of 2 and 20 samples would probably be fine. But maybe you think it's not worth it?

@nspope
Copy link
Contributor

nspope commented Jun 4, 2024

Not worth it, IMO. I'd rather have a benchmark suite that could be run whenever there's a change to the low-level stuff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants