Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: migrate database enum from name to value (2) #353

Merged
merged 1 commit into from
Jan 23, 2025

Conversation

jmaupetit
Copy link
Member

@jmaupetit jmaupetit commented Jan 23, 2025

Purpose

By default, python-derived enum fields are stored using the enum name instead of its value in database. This forces us to map the corresponding value when exporting data.

Proposal

Now that we moved legacy enum (with keys) to a varchar column, we update enum keys to their corresponding values and switch back columns to the new enum (with values).

This is the second step of the database migration (for the first step, see #351).

We also had to fix previous migration in #354

@jmaupetit jmaupetit self-assigned this Jan 23, 2025
@jmaupetit jmaupetit added improvement schema Discussions around data schemas API Item related to the API service labels Jan 23, 2025
@jmaupetit jmaupetit force-pushed the api-migrate-enums-2 branch 2 times, most recently from 614fc2d to a38cbbd Compare January 23, 2025 14:50
Now that we moved legacy enum (with keys) to a varchar column, we update
enum keys to their corresponding values and switch back columns to the
new enum (with values).
@jmaupetit jmaupetit force-pushed the api-migrate-enums-2 branch from a38cbbd to 9ba6fea Compare January 23, 2025 16:21
@MTES-MCT MTES-MCT deleted a comment from github-actions bot Jan 23, 2025
@MTES-MCT MTES-MCT deleted a comment from github-actions bot Jan 23, 2025
Copy link

Current benchmark

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% git timestamp version
GET /auth/whoami 45 0 310.0 467.326571911112 2.1773980000148185 2174.4312110000124 160.0 1.552083196879066 0.0 310 630 770 780 1300 1500 2200 2200 2200 2200 2200 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
POST /dynamique/session/ 45 0 570.0 762.4709530666653 7.571269000010261 2161.679575999983 45.0 1.552083196879066 0.0 570 930 1100 1600 1900 2000 2200 2200 2200 2200 2200 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
POST /dynamique/session/bulk 27 0 350.0 541.9506219259283 7.866140000004407 2618.8640169999926 411.0 0.9312499181274396 0.0 350 460 880 930 1400 1500 2600 2600 2600 2600 2600 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
GET /dynamique/status/ 28 0 500.0 608.5601375714306 10.770476000004692 2090.312830000016 274.8928571428572 0.9657406558358632 0.0 510 660 1000 1000 1100 1600 2100 2100 2100 2100 2100 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
POST /dynamique/status/ 34 0 660.0 753.1631261470587 29.24436499998251 2162.426018000019 45.0 1.1726850820864054 0.0 660 930 1100 1200 1600 1900 2200 2200 2200 2200 2200 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3 31 0 650.0 826.6643912580665 9.561242000017955 2621.362545000011 283.0 1.0692128689611344 0.0 650 850 1100 1200 1900 2200 2600 2600 2600 2600 2600 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3/history 40 0 370.0 531.1750773500037 9.13653000000636 2191.222605000007 51758.0 1.3796295083369476 0.0 370 620 690 850 1400 1700 2200 2200 2200 2200 2200 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
POST /dynamique/status/bulk 41 0 370.0 634.5246387804902 9.995212999996284 2596.9381429999885 411.0 1.4141202460453712 0.0 370 650 840 1100 1500 1900 2600 2600 2600 2600 2600 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
POST /statique/ 35 16 530.0 753.952377142858 21.70059200000196 2598.8233809999883 57.82857142857143 1.2071758197948292 0.551851803334779 530 860 1400 1400 1900 2400 2600 2600 2600 2600 2600 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
GET /statique/?limit=10 32 0 500.0 778.0239140624996 59.68511800000442 2411.220032000017 13826.625 1.103703606669558 0.0 510 870 1300 1300 1800 1900 2400 2400 2400 2400 2400 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
GET /statique/?limit=100 34 0 830.0 989.7364075882376 145.1397330000077 2128.2990770000083 129006.64705882352 1.1726850820864054 0.0 940 1300 1400 1900 1900 2100 2100 2100 2100 2100 2100 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
PUT /statique/FR073EP7QU5233 36 0 660.0 832.5862391666645 30.187712000014244 2619.5266519999905 1371.0 1.2416665575032528 0.0 710 930 1300 1300 1900 2400 2600 2600 2600 2600 2600 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
GET /statique/FRALLEGO002006P3 17 0 470.0 539.593147235295 29.92717899999775 2003.0420870000116 1202.0 0.5863425410432027 0.0 470 650 740 790 1100 2000 2000 2000 2000 2000 2000 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
POST /statique/bulk 36 0 1100.0 1174.0408669444455 344.271550000002 2597.458222 1124.1666666666667 1.2416665575032528 0.0 1100 1300 1700 1800 2000 2400 2600 2600 2600 2600 2600 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0
Aggregated 481 16 540.0 731.5128702723498 2.1773980000148185 2621.362545000011 14691.174636174635 16.590044837751794 0.551851803334779 540 810 1100 1300 1800 2000 2400 2600 2600 2600 2600 c9ba821 2025-01-23 16:24:30.062774+00:00 0.16.0

Comparison with the latest previous benchmark

A lower (negative) value means the current version performs better than the previous one.

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s
GET /auth/whoami
POST /dynamique/session/
POST /dynamique/session/bulk
GET /dynamique/status/
POST /dynamique/status/
GET /dynamique/status/FRALLEGO002006P3
GET /dynamique/status/FRALLEGO002006P3/history
POST /dynamique/status/bulk
POST /statique/
GET /statique/?limit=10
GET /statique/?limit=100
PUT /statique/FR073EP7QU5233
GET /statique/FRALLEGO002006P3
POST /statique/bulk
Aggregated

@jmaupetit jmaupetit merged commit 739cc3d into main Jan 23, 2025
14 of 15 checks passed
@jmaupetit jmaupetit deleted the api-migrate-enums-2 branch January 23, 2025 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Item related to the API service improvement schema Discussions around data schemas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant