Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speed up formatting & lint scripts via parallelism #330

Open
andyk opened this issue Mar 20, 2022 · 1 comment
Open

speed up formatting & lint scripts via parallelism #330

andyk opened this issue Mar 20, 2022 · 1 comment

Comments

@andyk
Copy link
Contributor

andyk commented Mar 20, 2022

Our formatting requirements for PRs are pretty stringent, specifically that we check that black and isort have been run. This leads to having the formatting script on my inner development/debugging loop (i.e. re-running between small changes while debugging a PR), and so I'm feeling the pain of the 30sec+ runtime of the lint and formatting scripts.

Can we trivially use (thread or process) parallelism to speed these up?

@nickjalbert
Copy link
Contributor

Yeah! Probably pretty straightforward to parallelize.

One trick that helps sometimes is that you can pass file path(s) to the script and it will only run against those files (e.g. if you've only modified a small number of files). For example:

python scripts/format_code.py agentos/component.py agentos/run.py
python scripts/lint_code.py agentos/component.py agentos/run.py

@andyk andyk removed this from Sprint 0.2.0 Mar 22, 2022
@andyk andyk moved this to Todo in Sprint 0.2.2 Mar 22, 2022
@andyk andyk removed this from Sprint 0.2.2 Jun 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants