Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimise future parallel strategy #345

Open
maxheld83 opened this issue Dec 2, 2021 · 3 comments
Open

optimise future parallel strategy #345

maxheld83 opened this issue Dec 2, 2021 · 3 comments
Labels
performance

Comments

@maxheld83
Copy link
Contributor

we might actually use more processes than CPUs, because a lot of the work isn't CPU bound (crossref api).
Should also leave some free for shiny though.
Also need to worry about the topology of the strategy.
@maxheld83 maxheld83 moved this to Todo in max's scrum(ish) Dec 2, 2021
@maxheld83
Copy link
Contributor Author

also need to benchmark the performance

@maxheld83
Copy link
Contributor Author

also need to (integration?) test this

@maxheld83
Copy link
Contributor Author

and the strategy should (largely) be set per platform (i.e. via env vars on azure), not in pkg

@maxheld83 maxheld83 moved this from Todo to In Progress in max's scrum(ish) Dec 7, 2021
maxheld83 added a commit that referenced this issue Dec 8, 2021
@maxheld83
only define plan at runtime #345
f8cd888
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance
Projects
max's scrum(ish)
Status: In Progress
Milestone
No milestone
Development

No branches or pull requests
1 participant
@maxheld83