Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create basic tests for query strategy #41

Merged
merged 2 commits into from
Oct 10, 2024
Merged

Conversation

jeremykubica
Copy link
Collaborator

Create tests for the random and uncertainty based sampling methods. Does not create anything for QBD because I'm still trying to understand how those should work.

Copy link

github-actions bot commented Oct 10, 2024

Before [051369d] After [6f9f52b] Ratio Benchmark (Parameter)
failed failed n/a benchmarks.peakmem_learn_loop('KNN')
failed failed n/a benchmarks.peakmem_learn_loop('RandomForest')
172±2ms 175±3ms 1.02 benchmarks.time_learn_loop('KNN', 'UncSampling')
2.62±0.01s 2.67±0.04s 1.02 benchmarks.time_learn_loop('RandomForest', 'RandomSampling')
170±0.9ms 170±0.8ms 1.00 benchmarks.time_learn_loop('KNN', 'RandomSampling')
2.67±0.01s 2.66±0.01s 1.00 benchmarks.time_learn_loop('RandomForest', 'UncSampling')
139±1ms 138±1ms 0.99 benchmarks.time_feature_creation

Click here to view all benchmarks.

[1.0, 0.0, 0.0], # 0.0
[0.5, 0.5, 0.0], # 0.693
[1.0/3.0, 1.0/3.0, 1.0/3.0], # 1.098
[0.5, 0.0, 5.0], # 0.693
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not totally sure what this function is doing, but should this line be [0.5, 0.0, 0.5] instead of[0.5, 0.0, 5.0]?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think I understand now. Shouldn't change the results, but would be good to fix

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I correct this.

@AmandaWasserman AmandaWasserman merged commit c160aed into main Oct 10, 2024
7 checks passed
@AmandaWasserman AmandaWasserman deleted the test_query_strategies branch October 10, 2024 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants