New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can you share how budget works ? #8

Open

lcrmorin opened this issue Oct 2, 2024 · 1 comment

lcrmorin commented Oct 2, 2024 •

edited

Loading

I am trying to understand this library. Notably how budget scale runtime:

How does run time (measured in real time) scale with budget ?
When benchmarking it seems you go from 56x speed up to 41x speed up when you go from budget 1 to budget 2.1. This speed up look insane compared to lgbm that is considered very fast. Can you clarify how is this speed up obtained ? (it seems you compare to 100 lgbm runs)

Contributor

deadsoul44 commented Oct 2, 2024

Hello,

Run time vs. budget depends on the data but here are the rough results for California Housing data:

budget=1.0 -> wall time=10 sec
budget=1.5 -> wall time=20 sec
budget=2.1 -> wall time=80 sec

Yes, it is compared to 100 hyperparameter optimization iterations. The budget is adjusted to achieve the same test metric.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment