Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Optimise ID Generation #114

Open
lawrenceadams opened this issue Jan 28, 2025 · 0 comments
Open

refactor: Optimise ID Generation #114

lawrenceadams opened this issue Jan 28, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@lawrenceadams
Copy link
Collaborator

Currently there are ID generating methods at play in several models which are expensive requiring ordering of IDs (and hence lots of table scans); for little/no upside as not deterministic.

We should simplify and unify the approach across all models.


Performance on "big Synthea" continues to improve with #111 - at least for me! Feel free to test it yourselves by checking out my branch and giving it a run :)

btw @lawrenceadams I am thinking we might want to get rid of all the order by columns in our row_number ID assignments. it's a drag on performance with little benefit since it's rarely (if ever) actually deterministic. i think there's a whole separate discussion worth having on the best way to approach ID assignment (which can extend into topics like support for incremental runs). thoughts?

Originally posted by @katy-sadowski in #91

@lawrenceadams lawrenceadams added the enhancement New feature or request label Jan 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant