-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Import is slow #26
Comments
I think that they best approach here should be to move away from pydantic and use vanilla dataclasses. Validation is at the static type level, which should be fine. Perhaps pydantic's dataclass decorator is much faster than the full BaseModel construction... |
Seems like 99ms -> 26ms |
Update to pydantic 2.9.1 https://docs.pydantic.dev/latest/changelog/#performance An initial pass was taken at improving import performance.
Still reports 300ms, unchanged.
TBH, after investigating type validation with vanilla dataclasses, I realize that it's all runtime metaprogramming under the hood which make me think that import time optimizations are going to be hard to come by. What's necessary is that python sees that the files are unchanged and then doesn't run inspections - as in, the models are already generated and compiled - as in python uses static types at runtime. |
This isn't a huge deal, but I've been on a mission to improve python import time performance lately since it's noticeable (and embarrassing!) in CLI applications.
You can see that the initialization of these Pydantic models is taking quite a bit.
Run each command or script at least once before measuring to give Python a chance to compile.
My terminal records ~300ms just to do this:
Here's the import times:
Pydantic is looking at ways of improving this, so we should stay tuned and follow any suggestions: pydantic/pydantic#6748
As of writing this, smp is using Pydantic 2.8.2.
If I sprinkle
defer_build=True
on everything, it seems to do what it says it does:Net, it does improve the speed quite a bit, but the reported 46ms for image_management is still pretty wild!
The text was updated successfully, but these errors were encountered: