Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ETL-671] Update to use older node version on aws actions #124

Merged
merged 9 commits into from
Jul 10, 2024

Conversation

rxu17
Copy link
Contributor

@rxu17 rxu17 commented Jul 6, 2024

Purpose: This is a draft PR because this is just a temp. solution. There is a known discrepancy between the glibc version used by the aws glue docker images (2.26) and glibc version used by all of the AWS actions versionswhich is expecting >= 2.27. Until the aws glue docker images are updated to use Node js version 20 (which uses glibc >= 2.27), this is the discovered workaround. See blog post for more info. This discussion gives a good view into the issue itself: actions/checkout#1809 (comment)

This solution is still temp. because ideally aws updates their aws glue docker images to use node js 20.

See JIRA ticket comments for more detailed updates on what was tried

Testing: Our GH actions for aws credentials now works without error

@@ -14,6 +14,7 @@ env:
PROD_INPUT_BUCKET: recover-input-data
PROD_INTERMEDIATE_BUCKET: recover-intermediate-data
INTEGRATION_TEST_NUM_EXPORTS: 28
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Does it make sense to update to newer versions or is that a harder lift?

Copy link
Contributor Author

@rxu17 rxu17 Jul 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the problem is even the newest versions of the aws glue docker containers (glue 4.0) haven't been updated to use node 20 hence they all have node 16 / the glibc version 2.26 that the aws actions cannot accept because the aws actions use node 20 / glibc version >= 2.27.

The above would only be a temp. solution though. A more long term solution (until the above gets fixed) i was thinking of and mentioned in the discussion is by building Node.js statically so it doesn't matter what glibc version is used by the docker container. Not sure how viable this is yet so going to try it.

Copy link

sonarqubecloud bot commented Jul 9, 2024

@rxu17 rxu17 marked this pull request as ready for review July 9, 2024 20:37
@rxu17 rxu17 requested a review from a team as a code owner July 9, 2024 20:37
@rxu17 rxu17 requested a review from philerooski July 10, 2024 07:30
@rxu17 rxu17 merged commit 29fe072 into main Jul 10, 2024
17 checks passed
@rxu17 rxu17 deleted the etl-671-update-glibc branch July 10, 2024 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants