Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add statique materialized view #305

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jmaupetit
Copy link
Member

Purpose

Having a SQL materialized view for "statique" data would ease recurrent database queries joining the same tables frequently. Moreover this should speed up database queries and response time for API endpoints using them.

Proposal

  • add sqlalchemy_utils dependency to ease materialized view management
  • add statique materialized view
  • test the materialized view
  • integrate a refresh strategy

@jmaupetit jmaupetit self-assigned this Dec 18, 2024
@jmaupetit jmaupetit marked this pull request as draft December 18, 2024 18:31
@jmaupetit jmaupetit added critical improvement WIP API Item related to the API service labels Dec 18, 2024
Copy link

Current benchmark

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% git timestamp version
GET /auth/whoami 26 0 400.0 515.718413346151 2.056316000022207 1214.9964719999955 160.0 1.0189333668455802 0.0 420 770 870 1100 1200 1200 1200 1200 1200 1200 1200 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
POST /dynamique/session/ 29 0 790.0 681.737442827586 10.328525000005584 1484.992591000008 45.0 1.1365026014816089 0.0 790 830 990 1200 1400 1500 1500 1500 1500 1500 1500 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
POST /dynamique/session/bulk 17 0 650.0 740.7971410000066 7.775741000017433 1910.784860999996 411.0 0.6662256629374949 0.0 650 870 1200 1400 1600 1900 1900 1900 1900 1900 1900 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
GET /dynamique/status/ 28 0 750.0 716.55433414286 20.307404000021734 1911.5672039999936 285.0 1.0973128566029329 0.0 830 990 1100 1100 1200 1200 1900 1900 1900 1900 1900 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
POST /dynamique/status/ 34 0 800.0 785.3022494117653 7.849969000005785 1661.8876889999967 45.0 1.3324513258749895 0.0 830 870 1100 1300 1500 1600 1700 1700 1700 1700 1700 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3 30 0 810.0 758.761375200001 12.608385999982374 1471.665807000022 275.8666666666667 1.175692346360285 0.0 830 870 940 1200 1400 1400 1500 1500 1500 1500 1500 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
GET /dynamique/status/FRALLEGO002006P3/history 34 0 820.0 811.818878647058 9.800361000003477 1786.0387789999947 41581.94117647059 1.3324513258749895 0.0 820 910 1100 1200 1700 1800 1800 1800 1800 1800 1800 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
POST /dynamique/status/bulk 22 0 540.0 621.1746135000002 14.838013000002093 1662.4327170000015 411.0 0.8621743873308757 0.0 620 810 880 1100 1300 1400 1700 1700 1700 1700 1700 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
POST /statique/ 23 9 600.0 639.8570041739135 21.20237700000871 1680.279635000005 61.08695652173913 0.9013641322095518 0.3527077039080855 600 840 980 1100 1400 1700 1700 1700 1700 1700 1700 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
GET /statique/?limit=10 24 0 830.0 822.7763993749994 20.146393000004537 1903.0624000000105 13826.583333333334 0.940553877088228 0.0 840 870 1200 1500 1600 1700 1900 1900 1900 1900 1900 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
GET /statique/?limit=100 25 0 850.0 841.2271899999995 160.07827699999666 1908.0001359999983 129006.6 0.979743621966904 0.0 850 940 1100 1200 1400 1600 1900 1900 1900 1900 1900 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
PUT /statique/FR073EP7QU5233 32 0 840.0 803.9745506875012 25.570134999981065 2002.932936999997 1371.0 1.2540718361176373 0.0 860 1100 1200 1200 1300 1900 2000 2000 2000 2000 2000 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
GET /statique/FRALLEGO002006P3 25 0 810.0 702.7399620400003 40.88732499999992 1606.7217929999913 1202.0 0.979743621966904 0.0 810 880 890 1200 1400 1500 1600 1600 1600 1600 1600 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
POST /statique/bulk 27 0 880.0 1010.3842450740756 327.29155699999524 1929.828604000022 1120.4444444444443 1.0581231117242564 0.0 880 1100 1300 1400 1700 1800 1900 1900 1900 1900 1900 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0
Aggregated 376 9 800.0 751.6942078989364 2.056316000022207 2002.932936999997 13605.446808510638 14.73534407438224 0.3527077039080855 810 880 1100 1200 1400 1600 1800 1900 2000 2000 2000 4ccf1ae 2024-12-18 18:34:52.777111+00:00 0.16.0

Comparison with the latest previous benchmark

A lower (negative) value means the current version performs better than the previous one.

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s
GET /auth/whoami
POST /dynamique/session/
POST /dynamique/session/bulk
GET /dynamique/status/
POST /dynamique/status/
GET /dynamique/status/FRALLEGO002006P3
GET /dynamique/status/FRALLEGO002006P3/history
POST /dynamique/status/bulk
POST /statique/
GET /statique/?limit=10
GET /statique/?limit=100
PUT /statique/FR073EP7QU5233
GET /statique/FRALLEGO002006P3
POST /statique/bulk
Aggregated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Item related to the API service critical improvement WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant