Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import is slow #26

Open
JPHutchins opened this issue Jul 17, 2024 · 4 comments
Open

Import is slow #26

JPHutchins opened this issue Jul 17, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@JPHutchins
Copy link
Owner

This isn't a huge deal, but I've been on a mission to improve python import time performance lately since it's noticeable (and embarrassing!) in CLI applications.

You can see that the initialization of these Pydantic models is taking quite a bit.

Run each command or script at least once before measuring to give Python a chance to compile.

My terminal records ~300ms just to do this:

python -c "from smp import image_management"

Here's the import times:

python -X importtime -c "from smp import image_management"

import time: self [us] | cumulative | imported package
import time:       194 |        194 | winreg
import time:       271 |        271 |   _io
import time:        36 |         36 |   marshal
import time:       132 |        132 |   nt
import time:       739 |       1176 | _frozen_importlib_external
import time:       454 |        454 |   time
import time:       215 |        669 | zipimport
import time:        45 |         45 |     _codecs
import time:       533 |        578 |   codecs
import time:      1227 |       1227 |   encodings.aliases
import time:      2022 |       3826 | encodings
import time:      1018 |       1018 | encodings.utf_8
import time:       646 |        646 | encodings.cp1252
import time:        93 |         93 | _signal
import time:        67 |         67 |     _abc
import time:       258 |        324 |   abc
import time:       328 |        651 | io
import time:        94 |         94 |       _stat
import time:       139 |        232 |     stat
import time:      1371 |       1371 |     _collections_abc
import time:        94 |         94 |       genericpath
import time:       100 |        100 |       _winapi
import time:       242 |        436 |     ntpath
import time:       505 |       2543 |   os
import time:        66 |         66 |   _sitebuiltins
import time:       473 |        473 |   encodings.utf_8_sig
import time:       437 |        437 |     __future__
import time:       853 |       1290 |   _virtualenv
import time:       402 |        402 |   sitecustomize
import time:      2771 |       7543 | site
import time:       875 |        875 | smp
import time:       608 |        608 |     types
import time:        80 |         80 |       _operator
import time:      1016 |       1095 |     operator
import time:       143 |        143 |         itertools
import time:       538 |        538 |         keyword
import time:      1025 |       1025 |         reprlib
import time:       153 |        153 |         _collections
import time:      1429 |       3287 |       collections
import time:        55 |         55 |       _functools
import time:      1656 |       4997 |     functools
import time:      2361 |       9060 |   enum
import time:       747 |        747 |     collections.abc
import time:       758 |        758 |     copyreg
import time:      1242 |       1242 |     contextlib
import time:        62 |         62 |         _sre
import time:       457 |        457 |           re._constants
import time:       742 |       1199 |         re._parser
import time:       364 |        364 |         re._casefix
import time:       861 |       2485 |       re._compiler
import time:      1240 |       3725 |     re
import time:       666 |        666 |     warnings
import time:        39 |         39 |     _typing
import time:      4490 |      11665 |   typing
import time:       226 |        226 |       pydantic.version
import time:       559 |        785 |     pydantic._migration
import time:       702 |        702 |             _ast
import time:      2315 |       3016 |           ast
import time:        35 |         35 |               _opcode
import time:       795 |        830 |             opcode
import time:      1445 |       2274 |           dis
import time:      1374 |       1374 |             importlib
import time:       149 |       1523 |           importlib.machinery
import time:       437 |        437 |               token
import time:        90 |         90 |               _tokenize
import time:      1585 |       2111 |             tokenize
import time:       865 |       2976 |           linecache
import time:      1491 |       1491 |             _weakrefset
import time:      1825 |       3315 |           weakref
import time:      2848 |      15950 |         inspect
import time:      2827 |       2827 |         _socket
import time:      1697 |      20473 |       typing_extensions
import time:       685 |      21158 |     pydantic.errors
import time:      1219 |      23161 |   pydantic
import time:       567 |        567 |   copy
import time:       261 |        261 |         _datetime
import time:       560 |        821 |       datetime
import time:      1304 |       2124 |     pydantic_core._pydantic_core
import time:       769 |        769 |           numbers
import time:      1660 |       2428 |         _decimal
import time:       537 |       2964 |       decimal
import time:      6550 |       9514 |     pydantic_core.core_schema
import time:      1068 |      12705 |   pydantic_core
import time:       818 |        818 |   pydantic._internal
import time:      2004 |       2004 |       dataclasses
import time:       569 |        569 |       pydantic._internal._internal_dataclass
import time:      1263 |       3834 |     pydantic.aliases
import time:       818 |        818 |     pydantic.config
import time:       301 |        301 |     pydantic.warnings
import time:       793 |       5745 |   pydantic._internal._config
import time:       322 |        322 |         pydantic._internal._typing_extra
import time:       322 |        643 |       pydantic._internal._repr
import time:       949 |       1591 |     pydantic._internal._core_utils
import time:      3427 |       5018 |   pydantic._internal._decorators
import time:      1376 |       1376 |       textwrap
import time:       717 |       2093 |     pydantic._internal._docs_extraction
import time:       839 |       2931 |   pydantic._internal._fields
import time:      1167 |       1167 |   pydantic._internal._forward_ref
import time:       105 |        105 |       _contextvars
import time:       984 |       1088 |     contextvars
import time:      1614 |       1614 |     pydantic._internal._utils
import time:      1256 |       3958 |   pydantic._internal._generics
import time:       634 |        634 |       pydantic.plugin
import time:       494 |       1127 |     pydantic.plugin._schema_validator
import time:       718 |       1845 |   pydantic._internal._mock_val_ser
import time:       249 |        249 |       pydantic.annotated_handlers
import time:        56 |         56 |         math
import time:       349 |        349 |         pydantic._internal._core_metadata
import time:       321 |        321 |         pydantic._internal._schema_generation_shared
import time:      1811 |       2536 |       pydantic.json_schema
import time:       331 |        331 |       pydantic._internal._discriminated_union
import time:       334 |        334 |       pydantic._internal._known_annotated_metadata
import time:      1319 |       4768 |     pydantic._internal._generate_schema
import time:       261 |        261 |     pydantic._internal._signature
import time:       681 |        681 |     pydantic._internal._validate_call
import time:       829 |       6537 |   pydantic._internal._model_construction
import time:       936 |        936 |         traceback
import time:        50 |         50 |           _string
import time:      2254 |       2304 |         string
import time:       983 |        983 |         threading
import time:        36 |         36 |         atexit
import time:      2356 |       6611 |       logging
import time:        65 |         65 |             _struct
import time:       574 |        639 |           struct
import time:      1055 |       1055 |           cbor2._types
import time:       769 |       2461 |         cbor2._decoder
import time:       423 |        423 |         cbor2._encoder
import time:      1290 |       1290 |         _cbor2
import time:       900 |       5073 |       cbor2
import time:      2215 |       2215 |       smp.header
import time:       697 |        697 |       smp.exceptions
import time:      7986 |       7986 |         annotated_types
import time:      1551 |       1551 |           ipaddress
import time:       595 |       2145 |         pydantic._internal._validators
import time:        64 |         64 |             binascii
import time:       741 |        804 |           base64
import time:       347 |        347 |                 posix
import time:       190 |        537 |               posixpath
import time:       623 |       1159 |             fnmatch
import time:       154 |        154 |             errno
import time:      1045 |       1045 |               urllib
import time:      3476 |       4520 |             urllib.parse
import time:      2447 |       8278 |           pathlib
import time:       859 |        859 |             _uuid
import time:       919 |       1777 |           uuid
import time:      5563 |      16421 |         pydantic.types
import time:      3177 |      29727 |       pydantic.fields
import time:      1639 |       1639 |       pydantic._internal._std_types_schema
import time:       318 |        318 |         pydantic._internal._dataclasses
import time:       429 |        746 |       pydantic.dataclasses
import time:       260 |        260 |             _csv
import time:      1597 |       1857 |           csv
import time:      1371 |       1371 |           email
import time:       392 |        392 |               importlib._abc
import time:       274 |        666 |             importlib.util
import time:       225 |        225 |               zlib
import time:       439 |        439 |                 _compression
import time:       952 |        952 |                 _bz2
import time:       600 |       1990 |               bz2
import time:       826 |        826 |                 _lzma
import time:       709 |       1535 |               lzma
import time:      1830 |       5579 |             shutil
import time:       419 |        419 |               zipfile._path.glob
import time:       774 |       1192 |             zipfile._path
import time:      2039 |       9474 |           zipfile
import time:       437 |        437 |               quopri
import time:       654 |        654 |                     _bisect
import time:       558 |       1212 |                   bisect
import time:       103 |        103 |                   _random
import time:       110 |        110 |                   _sha2
import time:      1078 |       2501 |                 random
import time:       825 |        825 |                     select
import time:      1205 |       2030 |                   selectors
import time:      2896 |       4926 |                 socket
import time:        53 |         53 |                       _locale
import time:      1451 |       1504 |                     locale
import time:      1500 |       3003 |                   calendar
import time:       762 |       3765 |                 email._parseaddr
import time:       405 |        405 |                   email.base64mime
import time:       558 |        558 |                   email.quoprimime
import time:      1024 |       1024 |                   email.errors
import time:       859 |        859 |                   email.encoders
import time:       732 |       3576 |                 email.charset
import time:       994 |      15760 |               email.utils
import time:       625 |        625 |                 email.header
import time:       519 |       1143 |               email._policybase
import time:       469 |        469 |               email._encoded_words
import time:       424 |        424 |               email.iterators
import time:      1059 |      19291 |             email.message
import time:       337 |        337 |               importlib.metadata._functools
import time:       998 |       1335 |             importlib.metadata._text
import time:       694 |      21319 |           importlib.metadata._adapters
import time:       564 |        564 |           importlib.metadata._meta
import time:       446 |        446 |           importlib.metadata._collections
import time:       695 |        695 |           importlib.metadata._itertools
import time:       677 |        677 |                 tempfile
import time:       548 |        548 |                 importlib.resources.abc
import time:       480 |        480 |                 importlib.resources._adapters
import time:       962 |       2666 |               importlib.resources._common
import time:       922 |        922 |               importlib.resources._legacy
import time:       627 |       4214 |             importlib.resources
import time:       903 |       5116 |           importlib.abc
import time:      2414 |      43252 |         importlib.metadata
import time:       730 |      43982 |       pydantic.plugin._loader
import time:     36638 |     127326 |     smp.message
import time:      5955 |     133280 |   smp.error
import time:     40081 |     258531 | smp.image_management

Pydantic is looking at ways of improving this, so we should stay tuned and follow any suggestions: pydantic/pydantic#6748

As of writing this, smp is using Pydantic 2.8.2.

If I sprinkle defer_build=True on everything, it seems to do what it says it does:

import time:      3300 |      41647 |     smp.message
import time:      1917 |      43564 |   smp.error
import time:     46152 |     220726 | smp.image_management

Net, it does improve the speed quite a bit, but the reported 46ms for image_management is still pretty wild!

@JPHutchins JPHutchins added the enhancement New feature or request label Jul 17, 2024
@JPHutchins
Copy link
Owner Author

I think that they best approach here should be to move away from pydantic and use vanilla dataclasses. Validation is at the static type level, which should be fine. Perhaps pydantic's dataclass decorator is much faster than the full BaseModel construction...

@JPHutchins
Copy link
Owner Author

Yeah, header, which uses vanilla dataclasses looks OK:

python -X importtime -c "from smp import header"                                 󰯸 smp
import time: self [us] | cumulative | imported package
import time:       211 |        211 | winreg
import time:       199 |        199 |   _io
import time:        43 |         43 |   marshal
import time:       148 |        148 |   nt
import time:       427 |        815 | _frozen_importlib_external
import time:       761 |        761 |   time
import time:       223 |        983 | zipimport
import time:       122 |        122 |     _codecs
import time:       644 |        765 |   codecs
import time:       941 |        941 |   encodings.aliases
import time:      1859 |       3564 | encodings
import time:       417 |        417 | encodings.utf_8
import time:       576 |        576 | encodings.cp1252
import time:       170 |        170 | _signal
import time:        82 |         82 |     _abc
import time:       316 |        398 |   abc
import time:       277 |        675 | io
import time:        46 |         46 |       _stat
import time:       185 |        231 |     stat
import time:      1076 |       1076 |     _collections_abc
import time:        33 |         33 |       genericpath
import time:        85 |         85 |       _winapi
import time:       129 |        246 |     ntpath
import time:       328 |       1880 |   os
import time:        63 |         63 |   _sitebuiltins
import time:      1021 |       1021 |   encodings.utf_8_sig
import time:       408 |        408 |     __future__
import time:       852 |       1259 |   _virtualenv
import time:       359 |        359 |   sitecustomize
import time:      2549 |       7128 | site
import time:       610 |        610 | smp
import time:        63 |         63 |     _struct
import time:       434 |        496 |   struct
import time:       577 |        577 |         types
import time:        66 |         66 |           _operator
import time:      1416 |       1482 |         operator
import time:       105 |        105 |             itertools
import time:       998 |        998 |             keyword
import time:       503 |        503 |             reprlib
import time:       152 |        152 |             _collections
import time:      1444 |       3200 |           collections
import time:        54 |         54 |           _functools
import time:      1159 |       4412 |         functools
import time:      1504 |       7973 |       enum
import time:        56 |         56 |         _sre
import time:       481 |        481 |           re._constants
import time:       545 |       1026 |         re._parser
import time:       517 |        517 |         re._casefix
import time:       771 |       2369 |       re._compiler
import time:       701 |        701 |       copyreg
import time:      1358 |      12399 |     re
import time:       455 |        455 |         _weakrefset
import time:      1207 |       1662 |       weakref
import time:       908 |       2570 |     copy
import time:      1699 |       1699 |         _ast
import time:      1321 |       1321 |         contextlib
import time:      2274 |       5294 |       ast
import time:        63 |         63 |           _opcode
import time:      1041 |       1104 |         opcode
import time:      2006 |       3110 |       dis
import time:       544 |        544 |       collections.abc
import time:       570 |        570 |           warnings
import time:       526 |       1095 |         importlib
import time:        94 |       1189 |       importlib.machinery
import time:       426 |        426 |           token
import time:        40 |         40 |           _tokenize
import time:      1425 |       1890 |         tokenize
import time:       832 |       2722 |       linecache
import time:      3045 |      15901 |     inspect
import time:      1173 |      32042 |   dataclasses
import time:        36 |         36 |     _typing
import time:      3137 |       3173 |   typing
import time:      3428 |       3428 |     _socket
import time:      2160 |       5587 |   typing_extensions
import time:      3272 |      44569 | smp.header
image

Using pydantic's dataclasses is much slower again

python -X importtime -c "from smp import header"                                 󰯸 smp
import time: self [us] | cumulative | imported package
import time:       198 |        198 | winreg
import time:       615 |        615 |   _io
import time:        92 |         92 |   marshal
import time:       374 |        374 |   nt
import time:       740 |       1821 | _frozen_importlib_external
import time:       439 |        439 |   time
import time:       234 |        672 | zipimport
import time:        57 |         57 |     _codecs
import time:       504 |        561 |   codecs
import time:       850 |        850 |   encodings.aliases
import time:      1631 |       3041 | encodings
import time:       405 |        405 | encodings.utf_8
import time:       958 |        958 | encodings.cp1252
import time:        93 |         93 | _signal
import time:        67 |         67 |     _abc
import time:       279 |        346 |   abc
import time:       322 |        668 | io
import time:        55 |         55 |       _stat
import time:       121 |        175 |     stat
import time:       481 |        481 |     _collections_abc
import time:        33 |         33 |       genericpath
import time:        78 |         78 |       _winapi
import time:       127 |        237 |     ntpath
import time:       314 |       1205 |   os
import time:        61 |         61 |   _sitebuiltins
import time:       703 |        703 |   encodings.utf_8_sig
import time:       432 |        432 |     __future__
import time:      1412 |       1843 |   _virtualenv
import time:       368 |        368 |   sitecustomize
import time:      2146 |       6324 | site
import time:       292 |        292 | smp
import time:        81 |         81 |     _struct
import time:      1063 |       1143 |   struct
import time:       543 |        543 |     types
import time:        65 |         65 |       _operator
import time:       570 |        635 |     operator
import time:        90 |         90 |         itertools
import time:      1060 |       1060 |         keyword
import time:       450 |        450 |         reprlib
import time:        65 |         65 |         _collections
import time:      1227 |       2889 |       collections
import time:        55 |         55 |       _functools
import time:       928 |       3871 |     functools
import time:      2122 |       7170 |   enum
import time:      1000 |       1000 |     collections.abc
import time:       473 |        473 |     copyreg
import time:       692 |        692 |     contextlib
import time:        63 |         63 |         _sre
import time:       712 |        712 |           re._constants
import time:      1395 |       2107 |         re._parser
import time:       679 |        679 |         re._casefix
import time:       787 |       3634 |       re._compiler
import time:      1364 |       4998 |     re
import time:       527 |        527 |     warnings
import time:        36 |         36 |     _typing
import time:      3720 |      11443 |   typing
import time:       262 |        262 |         pydantic.version
import time:       666 |        927 |       pydantic._migration
import time:      1278 |       1278 |               _ast
import time:      2436 |       3713 |             ast
import time:        84 |         84 |                 _opcode
import time:      1130 |       1214 |               opcode
import time:      1864 |       3078 |             dis
import time:       542 |        542 |               importlib
import time:       121 |        662 |             importlib.machinery
import time:       456 |        456 |                 token
import time:        45 |         45 |                 _tokenize
import time:      1476 |       1976 |               tokenize
import time:      1142 |       3117 |             linecache
import time:       491 |        491 |               _weakrefset
import time:      1292 |       1782 |             weakref
import time:      3196 |      15546 |           inspect
import time:      3251 |       3251 |           _socket
import time:      2453 |      21249 |         typing_extensions
import time:       782 |      22031 |       pydantic.errors
import time:      1404 |      24361 |     pydantic
import time:       539 |        539 |       copy
import time:      1995 |       2533 |     dataclasses
import time:       979 |        979 |     pydantic._internal
import time:       111 |        111 |             _datetime
import time:       670 |        780 |           datetime
import time:      1087 |       1867 |         pydantic_core._pydantic_core
import time:       661 |        661 |               numbers
import time:      1304 |       1964 |             _decimal
import time:       666 |       2629 |           decimal
import time:      6392 |       9020 |         pydantic_core.core_schema
import time:      2573 |      13459 |       pydantic_core
import time:       223 |        223 |         pydantic._internal._internal_dataclass
import time:      1205 |       1428 |       pydantic.aliases
import time:      1686 |       1686 |       pydantic.config
import time:       929 |        929 |       pydantic.warnings
import time:      1107 |      18606 |     pydantic._internal._config
import time:       495 |        495 |           pydantic._internal._typing_extra
import time:       415 |        909 |         pydantic._internal._repr
import time:      1043 |       1951 |       pydantic._internal._core_utils
import time:      4404 |       6355 |     pydantic._internal._decorators
import time:        69 |         69 |           math
import time:      6693 |       6762 |         annotated_types
import time:      1318 |       1318 |           ipaddress
import time:       586 |       1904 |         pydantic._internal._validators
import time:        80 |         80 |             binascii
import time:       892 |        972 |           base64
import time:       308 |        308 |                 posix
import time:       142 |        449 |               posixpath
import time:       587 |       1036 |             fnmatch
import time:       146 |        146 |             errno
import time:       613 |        613 |               urllib
import time:      1723 |       2335 |             urllib.parse
import time:      1560 |       5076 |           pathlib
import time:       906 |        906 |             _uuid
import time:      1560 |       2466 |           uuid
import time:       929 |        929 |               textwrap
import time:       310 |       1238 |             pydantic._internal._docs_extraction
import time:       439 |       1676 |           pydantic._internal._fields
import time:      1393 |       1393 |           pydantic._internal._utils
import time:       469 |        469 |           pydantic.annotated_handlers
import time:       342 |        342 |             pydantic._internal._core_metadata
import time:       574 |        574 |                 pydantic.plugin
import time:       546 |       1120 |               pydantic.plugin._schema_validator
import time:       437 |       1557 |             pydantic._internal._mock_val_ser
import time:       279 |        279 |             pydantic._internal._schema_generation_shared
import time:      2088 |       4265 |           pydantic.json_schema
import time:      5612 |      21926 |         pydantic.types
import time:        37 |         37 |             _contextvars
import time:       624 |        660 |           contextvars
import time:       882 |        882 |           pydantic._internal._forward_ref
import time:      1186 |       2727 |         pydantic._internal._generics
import time:      2692 |      36009 |       pydantic.fields
import time:      1428 |       1428 |         pydantic._internal._discriminated_union
import time:       422 |        422 |         pydantic._internal._known_annotated_metadata
import time:      3317 |       5166 |       pydantic._internal._generate_schema
import time:       533 |        533 |       pydantic._internal._signature
import time:      1120 |      42827 |     pydantic._internal._dataclasses
import time:      1304 |      96961 |   pydantic.dataclasses
import time:       255 |        255 |       pydantic._internal._validate_call
import time:       576 |        830 |     pydantic._internal._model_construction
import time:      1165 |       1994 |   pydantic.main
import time:      1582 |       1582 |   pydantic._internal._std_types_schema
import time:        71 |         71 |         _csv
import time:       811 |        881 |       csv
import time:       787 |        787 |       email
import time:       364 |        364 |           importlib._abc
import time:       162 |        526 |         importlib.util
import time:       142 |        142 |           zlib
import time:       453 |        453 |             _compression
import time:       846 |        846 |             _bz2
import time:       705 |       2003 |           bz2
import time:       651 |        651 |             _lzma
import time:       814 |       1464 |           lzma
import time:      1253 |       4860 |         shutil
import time:      1490 |       1490 |         threading
import time:       443 |        443 |           zipfile._path.glob
import time:       874 |       1317 |         zipfile._path
import time:      1735 |       9926 |       zipfile
import time:       521 |        521 |           quopri
import time:        33 |         33 |                 _bisect
import time:       469 |        501 |               bisect
import time:        67 |         67 |               _random
import time:        51 |         51 |               _sha2
import time:      1195 |       1813 |             random
import time:       718 |        718 |                 select
import time:      1219 |       1936 |               selectors
import time:      1696 |       3632 |             socket
import time:        47 |         47 |                   _locale
import time:      1447 |       1494 |                 locale
import time:       877 |       2370 |               calendar
import time:       487 |       2857 |             email._parseaddr
import time:       320 |        320 |               email.base64mime
import time:        30 |         30 |                   _string
import time:       695 |        724 |                 string
import time:       538 |       1262 |               email.quoprimime
import time:      1212 |       1212 |               email.errors
import time:       843 |        843 |               email.encoders
import time:       621 |       4257 |             email.charset
import time:      1344 |      13901 |           email.utils
import time:       989 |        989 |             email.header
import time:       996 |       1984 |           email._policybase
import time:       536 |        536 |           email._encoded_words
import time:       406 |        406 |           email.iterators
import time:      1991 |      19337 |         email.message
import time:       351 |        351 |           importlib.metadata._functools
import time:       527 |        877 |         importlib.metadata._text
import time:      1641 |      21854 |       importlib.metadata._adapters
import time:       762 |        762 |       importlib.metadata._meta
import time:       511 |        511 |       importlib.metadata._collections
import time:       383 |        383 |       importlib.metadata._itertools
import time:       666 |        666 |             tempfile
import time:      2160 |       2160 |             importlib.resources.abc
import time:       503 |        503 |             importlib.resources._adapters
import time:      1126 |       4455 |           importlib.resources._common
import time:       903 |        903 |           importlib.resources._legacy
import time:       960 |       6317 |         importlib.resources
import time:      1361 |       7678 |       importlib.abc
import time:      2983 |      45760 |     importlib.metadata
import time:       583 |      46343 |   pydantic.plugin._loader
import time:     31302 |     197936 | smp.header
image

@JPHutchins
Copy link
Owner Author

Seems like 99ms -> 26ms

@JPHutchins
Copy link
Owner Author

Update to pydantic 2.9.1

https://docs.pydantic.dev/latest/changelog/#performance

An initial pass was taken at improving import performance.

python -c "from smp import image_management"

Still reports 300ms, unchanged.

import time: self [us] | cumulative | imported package
import time:       221 |        221 | winreg
import time:       420 |        420 |   _io
import time:        45 |         45 |   marshal
import time:       202 |        202 |   nt
import time:       432 |       1098 | _frozen_importlib_external
import time:       415 |        415 |   time
import time:       105 |        519 | zipimport
import time:        66 |         66 |     _codecs
import time:       547 |        613 |   codecs
import time:       690 |        690 |   encodings.aliases
import time:      1501 |       2802 | encodings
import time:       950 |        950 | encodings.utf_8
import time:       544 |        544 | encodings.cp1252
import time:       109 |        109 | _signal
import time:        30 |         30 |     _abc
import time:       128 |        157 |   abc
import time:       164 |        321 | io
import time:        48 |         48 |       _stat
import time:        78 |        126 |     stat
import time:       542 |        542 |     _collections_abc
import time:        70 |         70 |       genericpath
import time:        87 |         87 |       _winapi
import time:       172 |        328 |     ntpath
import time:       291 |       1286 |   os
import time:        64 |         64 |   _sitebuiltins
import time:       494 |        494 |   encodings.utf_8_sig
import time:       407 |        407 |     __future__
import time:      1572 |       1978 |   _virtualenv
import time:       361 |        361 |   sitecustomize
import time:      2010 |       6191 | site
import time:       609 |        609 | smp
import time:       488 |        488 |     types
import time:        67 |         67 |       _operator
import time:       644 |        711 |     operator
import time:       134 |        134 |         itertools
import time:       385 |        385 |         keyword
import time:       890 |        890 |         reprlib
import time:       151 |        151 |         _collections
import time:      1268 |       2827 |       collections
import time:        73 |         73 |       _functools
import time:      1543 |       4442 |     functools
import time:      1487 |       7127 |   enum
import time:       515 |        515 |     collections.abc
import time:       451 |        451 |     copyreg
import time:       649 |        649 |     contextlib
import time:        70 |         70 |         _sre
import time:       553 |        553 |           re._constants
import time:      1414 |       1967 |         re._parser
import time:       335 |        335 |         re._casefix
import time:       952 |       3323 |       re._compiler
import time:       759 |       4081 |     re
import time:       547 |        547 |     warnings
import time:        32 |         32 |     _typing
import time:      3164 |       9437 |   typing
import time:       455 |        455 |     importlib
import time:       218 |        218 |       pydantic.version
import time:       580 |        798 |     pydantic._migration
import time:       633 |        633 |             _ast
import time:      1787 |       2420 |           ast
import time:        99 |         99 |               _opcode
import time:      1045 |       1144 |             opcode
import time:      1649 |       2792 |           dis
import time:        72 |         72 |           importlib.machinery
import time:       419 |        419 |               token
import time:        33 |         33 |               _tokenize
import time:      1531 |       1983 |             tokenize
import time:       540 |       2522 |           linecache
import time:       430 |        430 |             _weakrefset
import time:       968 |       1398 |           weakref
import time:      2660 |      11862 |         inspect
import time:      2643 |       2643 |         _socket
import time:      2615 |      17119 |       typing_extensions
import time:       860 |      17978 |     pydantic.errors
import time:       817 |      20046 |   pydantic
import time:       811 |        811 |   copy
import time:       174 |        174 |         _datetime
import time:       482 |        655 |       datetime
import time:       913 |       1568 |     pydantic_core._pydantic_core
import time:       626 |        626 |           numbers
import time:      1591 |       2216 |         _decimal
import time:       542 |       2757 |       decimal
import time:      6336 |       9093 |     pydantic_core.core_schema
import time:      1255 |      11915 |   pydantic_core
import time:       337 |        337 |   pydantic._internal
import time:      1387 |       1387 |       dataclasses
import time:       559 |        559 |       pydantic._internal._internal_dataclass
import time:      1600 |       3545 |     pydantic.aliases
import time:      1314 |       1314 |     pydantic.config
import time:       396 |        396 |     pydantic.warnings
import time:       829 |       6083 |   pydantic._internal._config
import time:       955 |        955 |                 threading
import time:      1789 |       2743 |               sysconfig
import time:      1097 |       3840 |             zoneinfo._tzpath
import time:        74 |         74 |                 _struct
import time:       532 |        606 |               struct
import time:       583 |       1188 |             zoneinfo._common
import time:       702 |        702 |             _zoneinfo
import time:       697 |       6426 |           zoneinfo
import time:       479 |       6905 |         pydantic._internal._typing_extra
import time:       393 |       7298 |       pydantic._internal._repr
import time:       873 |       8170 |     pydantic._internal._core_utils
import time:      4456 |      12625 |   pydantic._internal._decorators
import time:       943 |        943 |       textwrap
import time:       384 |       1327 |     pydantic._internal._docs_extraction
import time:       389 |        389 |     pydantic._internal._import_utils
import time:       544 |       2259 |   pydantic._internal._fields
import time:       619 |        619 |   pydantic._internal._forward_ref
import time:        41 |         41 |       _contextvars
import time:       506 |        546 |     contextvars
import time:       838 |        838 |     pydantic._internal._utils
import time:       662 |       2045 |   pydantic._internal._generics
import time:       672 |        672 |       pydantic.plugin
import time:      1060 |       1732 |     pydantic.plugin._schema_validator
import time:       760 |       2492 |   pydantic._internal._mock_val_ser
import time:       262 |        262 |             posix
import time:       137 |        398 |           posixpath
import time:       488 |        886 |         fnmatch
import time:       127 |        127 |         errno
import time:       700 |        700 |           urllib
import time:       196 |        196 |           math
import time:      1702 |       1702 |           ipaddress
import time:      2063 |       4659 |         urllib.parse
import time:      1196 |       6866 |       pathlib
import time:       788 |        788 |         _uuid
import time:       794 |       1582 |       uuid
import time:       300 |        300 |       pydantic.annotated_handlers
import time:       335 |        335 |         pydantic._internal._core_metadata
import time:      3677 |       4012 |       pydantic.functional_validators
import time:       343 |        343 |         pydantic._internal._schema_generation_shared
import time:      2096 |       2439 |       pydantic.json_schema
import time:       340 |        340 |       pydantic._internal._discriminated_union
import time:       345 |        345 |       pydantic._internal._known_annotated_metadata
import time:      1733 |      17614 |     pydantic._internal._generate_schema
import time:       622 |        622 |     pydantic._internal._signature
import time:       267 |        267 |     pydantic._internal._validate_call
import time:      1195 |      19698 |   pydantic._internal._model_construction
import time:       823 |        823 |         traceback
import time:        34 |         34 |           _string
import time:      1242 |       1275 |         string
import time:       112 |        112 |         atexit
import time:      4167 |       6376 |       logging
import time:       549 |        549 |           cbor2._types
import time:       877 |       1425 |         cbor2._decoder
import time:       775 |        775 |         cbor2._encoder
import time:      1614 |       1614 |         _cbor2
import time:      1563 |       5376 |       cbor2
import time:      1975 |       1975 |       smp.header
import time:       657 |        657 |       smp.exceptions
import time:      5697 |       5697 |         annotated_types
import time:       887 |        887 |         pydantic._internal._validators
import time:        75 |         75 |             binascii
import time:       958 |       1033 |           base64
import time:      5414 |       6446 |         pydantic.types
import time:      2851 |      15880 |       pydantic.fields
import time:       239 |        239 |         pydantic._internal._serializers
import time:      2380 |       2618 |       pydantic._internal._std_types_schema
import time:       725 |        725 |         pydantic._internal._dataclasses
import time:      1434 |       2159 |       pydantic.dataclasses
import time:        87 |         87 |             _csv
import time:       808 |        895 |           csv
import time:      1007 |       1007 |           email
import time:       339 |        339 |               importlib._abc
import time:       202 |        540 |             importlib.util
import time:       162 |        162 |               zlib
import time:       498 |        498 |                 _compression
import time:      2157 |       2157 |                 _bz2
import time:      1479 |       4134 |               bz2
import time:       659 |        659 |                 _lzma
import time:       970 |       1629 |               lzma
import time:      3221 |       9144 |             shutil
import time:       451 |        451 |               zipfile._path.glob
import time:      1843 |       2294 |             zipfile._path
import time:      1751 |      13728 |           zipfile
import time:       415 |        415 |               quopri
import time:        33 |         33 |                     _bisect
import time:       554 |        586 |                   bisect
import time:        43 |         43 |                   _random
import time:        35 |         35 |                   _sha2
import time:       707 |       1371 |                 random
import time:       540 |        540 |                     select
import time:       701 |       1241 |                   selectors
import time:      1188 |       2429 |                 socket
import time:        59 |         59 |                       _locale
import time:      1582 |       1640 |                     locale
import time:      1324 |       2964 |                   calendar
import time:       648 |       3612 |                 email._parseaddr
import time:       389 |        389 |                   email.base64mime
import time:       548 |        548 |                   email.quoprimime
import time:       613 |        613 |                   email.errors
import time:       383 |        383 |                   email.encoders
import time:       899 |       2830 |                 email.charset
import time:      1190 |      11430 |               email.utils
import time:       732 |        732 |                 email.header
import time:       771 |       1503 |               email._policybase
import time:       520 |        520 |               email._encoded_words
import time:       371 |        371 |               email.iterators
import time:      1203 |      15440 |             email.message
import time:       322 |        322 |               importlib.metadata._functools
import time:       448 |        770 |             importlib.metadata._text
import time:       718 |      16927 |           importlib.metadata._adapters
import time:      1029 |       1029 |           importlib.metadata._meta
import time:       742 |        742 |           importlib.metadata._collections
import time:       850 |        850 |           importlib.metadata._itertools
import time:       855 |        855 |                 tempfile
import time:       663 |        663 |                 importlib.resources.abc
import time:       967 |        967 |                 importlib.resources._adapters
import time:      1220 |       3704 |               importlib.resources._common
import time:       730 |        730 |               importlib.resources._legacy
import time:       545 |       4978 |             importlib.resources
import time:      1191 |       6169 |           importlib.abc
import time:      2394 |      43735 |         importlib.metadata
import time:       506 |      44240 |       pydantic.plugin._loader
import time:     43848 |     123126 |     smp.message
import time:      5489 |     128615 |   smp.error
import time:     25858 |     249961 | smp.image_management
image

TBH, after investigating type validation with vanilla dataclasses, I realize that it's all runtime metaprogramming under the hood which make me think that import time optimizations are going to be hard to come by. What's necessary is that python sees that the files are unchanged and then doesn't run inspections - as in, the models are already generated and compiled - as in python uses static types at runtime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant