Add StatiqueImporter and related import-statique CLI command
#139
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jmaupetit
force-pushed
the
add-parquet-import
branch
from
7a9beae to
ad6d0da
Compare
jmaupetit
force-pushed
the
add-parquet-import
branch
from
ad6d0da to
cca2993
Compare
jmaupetit
force-pushed
the
add-parquet-import
branch
from
dbf0cfe to
4b222e8
Compare
jmaupetit
changed the title
StatiqueImporter and related import-statique CLI command
|
J'ai un peu regardé mais uniquement sur quelques points (pas fondamentaux) liés à pandas (cf au dessus). Sur le reste, il faudrait que je passe du temps pour bien comprendre (mais je ne pense pas que ce soit très utile). Pour les temps de réponse, je ne sais pas quelles sont les parties les plus consommatrices sur lesquelles il faudrait creuser (si ça vaut la peine de gagner du temps). |
|
@loco-philippe merci pour ta review, j'ai appliqué tous les changements demandés. 🙏 |
We designed an efficient ORM-free Pandas-based Statique importer. It is supposed to be 10x faster than using ORM-based routines as implemented in /statique/bulk endpoint.
jmaupetit
force-pushed
the
add-parquet-import
branch
from
fab969b to
34942da
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
We need a more efficient way to massively import Statique data.
Proposal
qualicharge import-staticcommandusepostponed to a further PR (this one is already too big)save_statiques_from_dataframein/statique/bulkendpoint?