Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

⚡️(api) optimize create/bulk dynamic endpoints number of db queries #297

Merged
merged 1 commit into from
Dec 17, 2024
Merged

⚡️(api) optimize create/bulk dynamic endpoints number of db queries #297

merged 1 commit into from
Dec 17, 2024

Conversation

jmaupetit
Copy link
Member

Purpose

As the most accessed endpoints, dynamic create/bulk operations needed a carefull attention to ensure no extra database queries are performed.

Proposal

We removed 1 extra database query to fetch created object identifier and 3 extra database queries for bulk endpoints.

@jmaupetit jmaupetit self-assigned this Dec 16, 2024
@jmaupetit jmaupetit added critical improvement needs review API Item related to the API service labels Dec 16, 2024
jmaupetit
jmaupetit commented Dec 16, 2024
View reviewed changes
src/api/qualicharge/db.py Outdated Show resolved Hide resolved
@jmaupetit jmaupetit force-pushed the api-remove-useless-db-queries branch from b3e6038 to 68b483d Compare December 16, 2024 18:13
@github-actions GitHub Actions
Copy link

Current benchmark

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% git timestamp version
GET /auth/whoami 38 0 440.0 451.6330085789512 2.2144729999808987 1197.9447119999804 160.0 1.3235127301100806 0.0 450 530 640 790 1000 1200 1200 1200 1200 1200 1200 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
POST /dynamique/session/ 32 0 720.0 791.771145875 10.052814999994553 2309.777383999972 45.0 1.1145370358821731 0.0 790 1200 1300 1400 1600 1800 2300 2300 2300 2300 2300 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
POST /dynamique/session/bulk 30 0 780.0 812.6999823333355 18.311499000049025 1783.2183229999714 411.0 1.0448784711395374 0.0 890 950 1100 1500 1600 1700 1800 1800 1800 1800 1800 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
GET /dynamique/status/ 30 0 1200.0 1178.4414062666694 170.01447299998063 2306.387309999991 285.0 1.0448784711395374 0.0 1200 1500 1700 1800 1900 2000 2300 2300 2300 2300 2300 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
POST /dynamique/status/ 32 0 510.0 691.4028621875018 37.22537999999531 2315.316773999996 45.0 1.1145370358821731 0.0 630 930 960 1000 1500 1800 2300 2300 2300 2300 2300 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3 30 0 810.0 941.1651696999892 10.24198099997875 2165.151180999999 268.7333333333333 1.0448784711395374 0.0 920 1400 1600 1800 2000 2100 2200 2200 2200 2200 2200 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3/history 26 0 690.0 871.6466560384613 29.960189999997056 1934.1037589999812 82229.92307692308 0.9055613416542656 0.0 790 1100 1500 1600 1700 1900 1900 1900 1900 1900 1900 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
POST /dynamique/status/bulk 42 0 530.0 782.2741126428594 35.087188999966656 2431.1078979999934 411.0 1.4628298595953522 0.0 530 930 1200 1200 1700 1800 2400 2400 2400 2400 2400 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
POST /statique/ 31 12 450.0 589.8809734516025 27.57343199999696 2546.3330669999777 59.38709677419355 1.0797077535108552 0.4179513884558149 450 550 900 930 1200 2000 2500 2500 2500 2500 2500 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
GET /statique/?limit=10 29 0 740.0 802.2577660000015 35.75355300000638 2309.209084999964 13826.586206896553 1.0100491887682197 0.0 740 930 970 1400 1600 1700 2300 2300 2300 2300 2300 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
GET /statique/?limit=100 32 0 920.0 1048.1999084062447 228.45750699997325 1952.786064999998 129006.59375 1.1145370358821731 0.0 960 1200 1600 1600 1800 1900 2000 2000 2000 2000 2000 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
PUT /statique/FR073EP7QU5233 27 0 620.0 792.5674889259307 45.44811599998866 2556.943251000007 1371.0 0.9403906240255836 0.0 620 760 960 1100 1600 2300 2600 2600 2600 2600 2600 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
GET /statique/FRALLEGO002006P3 22 0 560.0 761.0163946818138 24.24311299995452 2159.8611279999886 1202.0 0.7662442121689941 0.0 600 890 950 1500 2000 2000 2200 2200 2200 2200 2200 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
POST /statique/bulk 30 0 960.0 1144.1375240999928 370.3286139999591 2175.6267260000186 1112.5333333333333 1.0448784711395374 0.0 1100 1500 1700 1800 2000 2000 2200 2200 2200 2200 2200 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0
Aggregated 431 12 710.0 825.223053800463 2.2144729999808987 2556.943251000007 15825.99071925754 15.01142070203802 0.4179513884558149 710 960 1200 1400 1700 1900 2200 2300 2600 2600 2600 26e99ae 2024-12-16 18:14:04.667567+00:00 0.16.0

Comparison with the latest previous benchmark

A lower (negative) value means the current version performs better than the previous one.

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s
GET /auth/whoami
POST /dynamique/session/
POST /dynamique/session/bulk
GET /dynamique/status/
POST /dynamique/status/
GET /dynamique/status/FRALLEGO002006P3
GET /dynamique/status/FRALLEGO002006P3/history
POST /dynamique/status/bulk
POST /statique/
GET /statique/?limit=10
GET /statique/?limit=100
PUT /statique/FR073EP7QU5233
GET /statique/FRALLEGO002006P3
POST /statique/bulk
Aggregated

@github-actions GitHub Actions
Copy link

Current benchmark

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% git timestamp version
GET /auth/whoami 35 0 480.0 568.9031972000017 2.7678619999846887 1879.3194499999688 160.0 1.222349144475485 0.0 480 810 850 920 1200 1500 1900 1900 1900 1900 1900 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
POST /dynamique/session/ 29 0 600.0 708.6760590344836 9.882734000029814 1778.4961119999707 45.0 1.0128035768511163 0.0 600 900 1100 1400 1700 1800 1800 1800 1800 1800 1800 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
POST /dynamique/session/bulk 37 0 480.0 588.4907387297268 11.888124999984484 2152.911832999962 411.0 1.2921976670169415 0.0 480 610 850 980 1400 1600 2200 2200 2200 2200 2200 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
GET /dynamique/status/ 25 0 670.0 918.8381530800052 13.829880999992383 2658.641040999953 273.68 0.8731065317682036 0.0 670 990 1300 1900 2000 2000 2700 2700 2700 2700 2700 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
POST /dynamique/status/ 31 0 680.0 776.0432803548324 13.723333999962506 2378.483276999986 45.0 1.0826520993925726 0.0 680 900 990 1000 1400 1700 2400 2400 2400 2400 2400 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3 29 0 450.0 604.3977238275897 24.836484999980257 2604.971384999999 283.0 1.0128035768511163 0.0 450 560 650 990 1500 1900 2600 2600 2600 2600 2600 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3/history 32 0 660.0 850.9882670625045 11.537173000021994 2657.6773470000035 54877.25 1.117576360663301 0.0 670 930 1300 1300 1900 2400 2700 2700 2700 2700 2700 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
POST /dynamique/status/bulk 31 0 620.0 877.3006492903235 30.109864999985803 2323.3163499999705 411.0 1.0826520993925726 0.0 620 1100 1200 1300 1600 2200 2300 2300 2300 2300 2300 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
POST /statique/ 33 16 530.0 707.8477084242433 27.91936600004874 2658.610044 55.21212121212121 1.1525006219340288 0.5587881803316503 530 850 880 890 1600 1800 2700 2700 2700 2700 2700 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
GET /statique/?limit=10 38 0 670.0 703.8440663684199 18.06840300002932 2605.608713000038 13826.78947368421 1.3271219282876696 0.0 680 760 830 980 1700 1800 2600 2600 2600 2600 2600 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
GET /statique/?limit=100 39 0 720.0 907.6920417435942 132.89635500001395 2655.6403620000424 129006.79487179487 1.3620461895583975 0.0 720 890 1200 1200 2200 2500 2700 2700 2700 2700 2700 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
PUT /statique/FR073EP7QU5233 43 0 530.0 710.3538865581407 24.905112999988432 2376.12494299998 1371.0 1.5017432346413104 0.0 530 890 1200 1300 1600 1700 2400 2400 2400 2400 2400 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
GET /statique/FRALLEGO002006P3 39 0 530.0 666.871666000001 12.239892000025064 2223.3230329999856 1202.0 1.3620461895583975 0.0 530 670 890 1300 1700 2200 2200 2200 2200 2200 2200 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
POST /statique/bulk 29 0 1100.0 1219.8021795172465 392.4293350000312 2604.519663000019 1112.7241379310344 1.0128035768511163 0.0 1100 1600 1600 1900 2200 2500 2600 2600 2600 2600 2600 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0
Aggregated 470 16 610.0 763.0928999148953 2.7678619999846887 2658.641040999953 15965.902127659574 16.41440279724223 0.5587881803316503 610 860 1100 1300 1700 2000 2400 2600 2700 2700 2700 08736b4 2024-12-16 18:16:22.165790+00:00 0.16.0

Comparison with the latest previous benchmark

A lower (negative) value means the current version performs better than the previous one.

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s
GET /auth/whoami
POST /dynamique/session/
POST /dynamique/session/bulk
GET /dynamique/status/
POST /dynamique/status/
GET /dynamique/status/FRALLEGO002006P3
GET /dynamique/status/FRALLEGO002006P3/history
POST /dynamique/status/bulk
POST /statique/
GET /statique/?limit=10
GET /statique/?limit=100
PUT /statique/FR073EP7QU5233
GET /statique/FRALLEGO002006P3
POST /statique/bulk
Aggregated

@jmaupetit
⚡️(api) optimize create/bulk dynamic endpoints number of db queries
542afeb 
As the most accessed endpoints, dynamic create/bulk operations needed a
carefull attention to ensure no extra database queries were performed.
We removed 1 extra database query to fetch created object identifier
and 3 extra database queries for bulk endpoints.
@jmaupetit jmaupetit force-pushed the api-remove-useless-db-queries branch from 68b483d to 542afeb Compare December 16, 2024 20:31
@github-actions GitHub Actions
Copy link

Current benchmark

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% git timestamp version
GET /auth/whoami 44 0 410.0 610.9551770000018 1.9286439999746108 1715.6083140000078 160.0 1.5308301906415518 0.0 430 930 1100 1300 1400 1600 1700 1700 1700 1700 1700 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
POST /dynamique/session/ 36 0 510.0 705.6647863888878 26.69272700001102 2450.0011340000183 45.0 1.252497428706724 0.0 540 690 1300 1300 1700 2100 2500 2500 2500 2500 2500 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
POST /dynamique/session/bulk 39 0 670.0 838.7908238717949 10.305424999984323 2051.6033859999825 411.0 1.3568722144322845 0.0 670 790 1300 1700 1800 2000 2100 2100 2100 2100 2100 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
GET /dynamique/status/ 31 0 780.0 935.4557251290316 25.090032999997906 2050.04076900002 275.8709677419355 1.078539452497457 0.0 780 1300 1600 1700 2000 2000 2100 2100 2100 2100 2100 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
POST /dynamique/status/ 28 0 480.0 728.7657052499991 28.530851999988727 2391.1035350000134 45.0 0.9741646667718966 0.0 550 790 1300 1300 1700 2000 2400 2400 2400 2400 2400 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3 33 0 500.0 667.877862484848 10.48431199998845 2061.213389000017 276.5151515151515 1.1481226429811635 0.0 500 750 880 1100 1700 2000 2100 2100 2100 2100 2100 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3/history 39 0 720.0 753.8148906666657 7.242919999981723 2030.158 42728.28205128205 1.3568722144322845 0.0 720 870 1300 1300 1600 1700 2000 2000 2000 2000 2000 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
POST /dynamique/status/bulk 42 0 690.0 861.464326523807 13.273861999977044 2089.576638000012 411.0 1.4612470001578448 0.0 690 1100 1400 1400 1900 2000 2100 2100 2100 2100 2100 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
POST /statique/ 32 17 400.0 515.5006549375 12.127723000020296 2015.3260879999948 57.46875 1.1133310477393104 0.5914571191115087 460 710 770 790 1100 1800 2000 2000 2000 2000 2000 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
GET /statique/?limit=10 17 0 310.0 561.3525545882362 16.26074600000038 1836.9097409999997 13826.529411764706 0.5914571191115087 0.0 310 530 810 1000 1700 1800 1800 1800 1800 1800 1800 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
GET /statique/?limit=100 19 0 710.0 859.867219421053 137.10602599999788 1751.4745889999972 129006.57894736844 0.6610403095952155 0.0 710 1200 1400 1600 1600 1800 1800 1800 1800 1800 1800 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
PUT /statique/FR073EP7QU5233 32 0 550.0 738.3641292187518 61.24146600001268 2031.712174999996 1371.0 1.1133310477393104 0.0 580 710 1000 1300 1600 2000 2000 2000 2000 2000 2000 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
GET /statique/FRALLEGO002006P3 39 0 630.0 688.9590830512822 13.831656999997222 1814.4918909999888 1202.0 1.3568722144322845 0.0 630 830 1100 1300 1400 1700 1800 1800 1800 1800 1800 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
POST /statique/bulk 36 0 1100.0 1258.637016749996 366.5307440000163 2455.1267390000075 1115.1944444444443 1.252497428706724 0.0 1300 1600 1800 1800 2100 2100 2500 2500 2500 2500 2500 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0
Aggregated 467 17 640.0 771.3235586702345 1.9286439999746108 2455.1267390000075 9734.910064239828 16.24767497794556 0.5914571191115087 640 870 1300 1400 1700 2000 2100 2100 2500 2500 2500 e3279c6 2024-12-16 20:34:44.807707+00:00 0.16.0

Comparison with the latest previous benchmark

A lower (negative) value means the current version performs better than the previous one.

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s
GET /auth/whoami
POST /dynamique/session/
POST /dynamique/session/bulk
GET /dynamique/status/
POST /dynamique/status/
GET /dynamique/status/FRALLEGO002006P3
GET /dynamique/status/FRALLEGO002006P3/history
POST /dynamique/status/bulk
POST /statique/
GET /statique/?limit=10
GET /statique/?limit=100
PUT /statique/FR073EP7QU5233
GET /statique/FRALLEGO002006P3
POST /statique/bulk
Aggregated

@jmaupetit jmaupetit requested a review from ssorin December 16, 2024 21:19
@jmaupetit jmaupetit merged commit 0d31e77 into main Dec 17, 2024
15 checks passed
@jmaupetit jmaupetit deleted the api-remove-useless-db-queries branch December 17, 2024 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ssorin ssorin ssorin approved these changes

Assignees

@jmaupetit jmaupetit

Labels
API Item related to the API service critical improvement needs review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jmaupetit @ssorin