add benchmark

[vincentsarago]

No description provided.

[jonhealy1]

A small difference maybe. What if it ran with larger items? It probably wouldn't make much difference.

[vincentsarago]

WoW 🤯

well this is the same as thomas-maschler#2, in fact validation was never called 🤦

[vincentsarago]

🥳

------------------------ benchmark 'Collection': 1 tests ------------------------
Name (time in us)                         Min         Max        Mean      Median
---------------------------------------------------------------------------------
test_benchmark_collection[False]     938.8680  1,843.4210  1,042.7988  1,007.7470
---------------------------------------------------------------------------------

----- benchmark 'Collection With Model validation': 1 tests ------
Name (time in ms)                      Min     Max    Mean  Median
------------------------------------------------------------------
test_benchmark_collection[True]     1.3901  2.2407  1.5117  1.5085
------------------------------------------------------------------

----------------- benchmark 'Collections': 1 tests -----------------
Name (time in ms)                        Min     Max    Mean  Median
--------------------------------------------------------------------
test_benchmark_collections[False]     1.8438  2.6199  1.9471  1.9282
--------------------------------------------------------------------

------ benchmark 'Collections With Model validation': 1 tests ------
Name (time in ms)                       Min      Max    Mean  Median
--------------------------------------------------------------------
test_benchmark_collections[True]     5.5668  41.8152  5.9273  5.6902
--------------------------------------------------------------------

------------------------------------------ benchmark 'Items': 7 tests ------------------------------------------
Name (time in ms)                        Min                 Max               Mean             Median          
----------------------------------------------------------------------------------------------------------------
test_benchmark_items[False-1]         1.0510 (1.0)        2.1373 (1.0)       1.2407 (1.0)       1.2498 (1.0)    
test_benchmark_items[False-10]        1.9332 (1.84)       3.0217 (1.41)      2.0737 (1.67)      2.0594 (1.65)   
test_benchmark_items[False-50]        5.6753 (5.40)      11.0111 (5.15)      6.0548 (4.88)      5.9042 (4.72)   
test_benchmark_items[False-100]      10.5232 (10.01)     11.3931 (5.33)     10.7932 (8.70)     10.7663 (8.61)   
test_benchmark_items[False-200]      19.6394 (18.69)     20.6484 (9.66)     20.0561 (16.16)    20.0372 (16.03)  
test_benchmark_items[False-250]      23.9003 (22.74)     63.0864 (29.52)    25.5576 (20.60)    24.6302 (19.71)  
test_benchmark_items[False-1000]     92.2480 (87.77)    129.9291 (60.79)    99.1982 (79.95)    93.6968 (74.97)  
----------------------------------------------------------------------------------------------------------------

-------------------------------- benchmark 'Items With Model validation': 7 tests --------------------------------
Name (time in ms)                        Min                 Max                Mean              Median          
------------------------------------------------------------------------------------------------------------------
test_benchmark_items[True-1]          1.6814 (1.0)        2.7930 (1.0)        1.8697 (1.0)        1.8621 (1.0)    
test_benchmark_items[True-10]         5.6303 (3.35)      33.6163 (12.04)      6.0987 (3.26)       5.8813 (3.16)   
test_benchmark_items[True-50]        22.9453 (13.65)     27.3397 (9.79)      23.9438 (12.81)     23.4351 (12.59)  
test_benchmark_items[True-100]       44.1020 (26.23)     84.6257 (30.30)     48.6900 (26.04)     45.5900 (24.48)  
test_benchmark_items[True-200]       87.0463 (51.77)    131.5226 (47.09)     95.5188 (51.09)     90.6731 (48.69)  
test_benchmark_items[True-250]      107.6794 (64.04)    142.4899 (51.02)    114.9167 (61.46)    111.0771 (59.65)  
test_benchmark_items[True-1000]     434.4064 (258.36)   486.4296 (174.16)   463.5648 (247.94)   478.4742 (256.96) 
------------------------------------------------------------------------------------------------------------------

[vincentsarago]

This PR also fix #461

[vincentsarago]

@jonhealy1 @gadomski could you approve this PR 🙏

[jonhealy1]

This is really cool