Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: introduce promiseall-no-unbounded-parallelism linter rule #31592

Merged
merged 21 commits into from
Oct 4, 2024

Conversation

rix0rrr
Copy link
Contributor

@rix0rrr rix0rrr commented Sep 30, 2024

Since JavaScript is single-threaded, Promise.all() will only ever be used for I/O-bound tasks. However, I/O-parallelism isn't free either. Every async I/O-performing task launched will consume some FDs, and their amount is limited. If the amount of promises launched is a function of the input the program runs on, the system FDs might be exhausted. Some concurrency limit must be introduced.

This linter rule exists to remind the programmer of that fact. It triggers on every Promise.all() invocation and cannot be resolved; the only solution is to think about it, and then silence this rule as proof that you thought about it.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@rix0rrr rix0rrr requested a review from a team September 30, 2024 09:19
@aws-cdk-automation aws-cdk-automation requested a review from a team September 30, 2024 09:20
@github-actions github-actions bot added the p2 label Sep 30, 2024
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Sep 30, 2024
@rix0rrr rix0rrr force-pushed the huijbers/no-unbounded-parallelism branch from 0285df4 to 50a07c9 Compare September 30, 2024 09:22
Since JavaScript is single-threaded, `Promise.all()` will only ever be used for
I/O-bound tasks. However, I/O-parallelism isn't free either. Every async I/O-performing
task launched will consume some FDs, and their amount is limited. If the amount
of promises launched is a function of the input the program runs on, the system
FDs might be exhausted. Some concurrency limit must be introduced.

This linter rule exists to remind the programmer of that fact. It triggers
on every `Promise.all()` invocation and cannot be resolved; the only solution
is to think about it, and then silence this rule as proof that you thought about it.
@rix0rrr rix0rrr force-pushed the huijbers/no-unbounded-parallelism branch from 50a07c9 to a04c4c3 Compare September 30, 2024 09:26
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@aws-cdk-automation aws-cdk-automation added the pr/needs-cli-test-run This PR needs CLI tests run against it. label Sep 30, 2024
@rix0rrr rix0rrr self-assigned this Sep 30, 2024
@sumupitchayan sumupitchayan added pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested pr/needs-cli-test-run This PR needs CLI tests run against it. and removed pr/needs-cli-test-run This PR needs CLI tests run against it. pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested labels Oct 3, 2024
@aws-cdk-automation
Copy link
Collaborator

➡️ PR build request submitted to test-main-pipeline ⬅️

A maintainer must now check the pipeline and add the pr-linter/cli-integ-tested label once the pipeline succeeds.

@rix0rrr rix0rrr added the pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested label Oct 4, 2024
@aws-cdk-automation aws-cdk-automation dismissed their stale review October 4, 2024 08:40

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-cli-test-run This PR needs CLI tests run against it. label Oct 4, 2024
…s-cdk into huijbers/no-unbounded-parallelism
@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Oct 4, 2024
Copy link
Contributor

mergify bot commented Oct 4, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Oct 4, 2024
Copy link
Contributor

mergify bot commented Oct 4, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 324163f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit a37b27e into main Oct 4, 2024
11 of 12 checks passed
Copy link
Contributor

mergify bot commented Oct 4, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot deleted the huijbers/no-unbounded-parallelism branch October 4, 2024 14:22
Copy link

github-actions bot commented Oct 4, 2024

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
contribution/core This is a PR that came from AWS. p2 pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants