build(deps): Bump actions/upload-artifact from 3 to 4 #2307
Conversation
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?s=60&v=4){kind=small}
dependabot
bot
force-pushed
the
dependabot/github_actions/actions/upload-artifact-4
branch
from
e707855 to
d63d4b0
Compare
dependabot
bot
force-pushed
the
dependabot/github_actions/actions/upload-artifact-4
branch
from
d63d4b0 to
909eb40
Compare
dependabot
bot
force-pushed
the
dependabot/github_actions/actions/upload-artifact-4
branch
from
909eb40 to
ad70b46
Compare
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v3...v4) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>
dependabot
bot
force-pushed
the
dependabot/github_actions/actions/upload-artifact-4
branch
from
ad70b46 to
fc36fc4
Compare
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v3...v4) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>
Bumps [geekyeggo/delete-artifact](https://github.com/geekyeggo/delete-artifact) from 2 to 5. - [Release notes](https://github.com/geekyeggo/delete-artifact/releases) - [Changelog](https://github.com/GeekyEggo/delete-artifact/blob/main/CHANGELOG.md) - [Commits](GeekyEggo/delete-artifact@v2...v5) --- updated-dependencies: - dependency-name: geekyeggo/delete-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>
The "Sync Labels" GitHub Actions workflow is configured to allow the use of multiple shared label configuration files. This is done by using a job matrix in the GitHub Actions workflow to download each of the files from the source repository in a parallel GitHub Actions workflow job. A GitHub Actions workflow artifact was used to transfer the generated files between sequential jobs in the workflow. The "actions/upload-artifact" and "actions/download-artifact" actions are used for this purpose. Previously, a single artifact was used for the transfer of all the shared label configuration files, with each of the parallel jobs uploading its own generated files to that artifact. However, support for uploading multiple times to a single artifact was dropped in version 4.0.0 of the "actions/upload-artifact" action. So it is now necessary to use a dedicated artifact for each of the builds. These can be downloaded in aggregate by using the artifact name globbing and merging features which were introduced in version 4.1.0 of the "actions/download-artifact" action.
The build workflow produces binaries for a range of target hosts. This is done by using a job matrix in the GitHub Actions workflow that produces each build in a parallel job. GitHub Actions workflow artifacts are used to transfer the generated files between sequential jobs in the workflow. The "actions/upload-artifact" action is used for this purpose. Previously, a single artifact was used for this purpose, with each of the parallel jobs uploading its own generated files to that artifact. However, support for uploading multiple times to a single artifact was dropped in version 4.0.0 of the "actions/upload-artifact" action. So it is now necessary to use a dedicated artifact for each of the builds. These can be downloaded in aggregate by using the artifact name globbing and merging features which were introduced in version 4.1.0 of the "actions/download-artifact" action.
Background
==========
Shared Library Dependencies
---------------------------
The Linux build of Arduino IDE has dynamic linkage against the libstdc++ and glibc shared libraries. This results in
it having a dependency on the version of the libraries that happens to be present in the environment it is built in.
Although newer versions of the shared libraries are compatible with executables linked against an older version, the
reverse is not true. This means that building Arduino IDE on a Linux machine with a recent distro version installed
causes the IDE to error on startup for users who have a distro with older versions of the dependencies.
For example, if Arduino IDE were built on a machine with version 3.4.33 of libstdc++, then attempting to run it on a
machine with an older version of libstdc++ would fail with an error like:
```
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.33' not found (required by /home/foo/arduino-ide/resources/app/lib/backend/native/nsfw.node)
```
Likewise, if Arduino IDE were built on a machine with version 2.39 of glibc, then attempting to run it on a machine
with an older version of glibc would fail with an error like:
```
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.39' not found (required by /home/foo/arduino-ide/resources/app/node_modules/nsfw/build/Release/nsfw.node)
```
Build Machine Requirements
--------------------------
The IDE builds distributed by Arduino should be compatible with a reasonable range of Linux distribution versions. In
order to achieve this, the builds must be performed in a machine with an older version of the shared libraries. The
shared libraries are part of the Linux distro, and installing a different version is not feasible. So this imposes a
maximum limit on the build machine's distro version.
The distributed builds are generated via a GitHub Actions workflow. The most simple approach is to run the build in the
machine of the GitHub-hosted runners provided for each operating system. However, GitHub provides a limited range of
operating system versions in their runners, and removes the older versions as newer versions are added. This means that
building in the GitHub-hosted runner machine would not allow for the desired range of Linux distro version
compatibility. For this reason, the Linux build is performed in a Docker container that provides an older version of
Ubuntu.
The same situation of incompatibility with Linux distro versions that have a version of the shared library dependencies
older than the version present on the build machine occurs for several of the tools and frameworks used by the build
process (e.g., Node.js, Python). In this case, the tables are turned as we are now the user rather than the distributor
and so are at the mercy of the Linux distro version compatibility range provided by the distributor. So this imposes a
minimum limit on the build machine's distro version.
Although several of the dependencies used by the standard build system have dependencies on versions of glibc higher
than the version 2.27 present in Ubuntu 18.04, it was possible to use this distro version in the Linux build container
by using alternative distributions and/or versions of these dependencies.
Workflow Artifacts
------------------
The build workflow uses GitHub actions workflow artifacts to transfer the files generated by the build job to subsequent
jobs in the workflow. The "actions/upload-artifact" action is used for this purpose.
Problem
=======
GitHub is dropping support for the workflow artifacts produced by the version 3.x of the "actions/upload-artifact"
action that was previously used by the build job. So the action version used in the build workflow was updated to the
current version 4.x. This version of the action uses a newer version of the Node.js runtime (20). Unfortunately the the
Node.js 20 runtime used by the action has a dependency on glibc version 2.28, which causes the Linux build job to fail
after the update of the "actions/upload-artifact" action:
```
Run actions/upload-artifact@v4
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
```
Unlike the other dependencies of the build process, it is no longer possible to work around this incompatibility by
continuing to use the older compatible version of the "actions/upload-artifact" action. It is also impossible to replace
the incompatible Node.js 20.x distribution used by the action, since it comes from the read-only file system of the
runner image. Likewise, it is not possible to configure or force the action to use a Node.js installation at a different
path on the runner machine.
Resolution
==========
Compatibility with the new version of the "actions/upload-artifact" action is attained by updating the version of Linux
in the build container to 18.10, which is the oldest version that has glibc 2.28. The presence of a newer glibc version
in the container also makes it compatible with several other dependencies of the build process, meaning the code in the
Dockerfile and workflow for working around the incompatibilities of Ubuntu 18.04 can be removed.
Consequences
============
Unfortunately this means the loss of compatibility of the Linux Arduino IDE builds with distros that use glibc 2.27
(e.g., Ubuntu 18.04). User of those distros will now find that Arduino IDE fails to start with an error like:
```
Error: node-loader:
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/foo/arduino-ide/resources/app/lib/backend/native/pty.node)
at 85467 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:2766)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 23571 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3374073)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 55444 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3369761)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 24290 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1780542)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 43416 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1770138)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
```
There was a problem hiding this comment.
The v4 bump of the actions/upload-artifact action proposed by this PR has an unfortunate side effect. I'll provide a summary below. There are some additional details in the commit message of 99df637 (#2307).
Background
Need to Immediately Bump the Artifact Actions
GitHub is dropping support for the different format of artifacts generated by the pre-4.x versions of the action:
https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/
Starting December 5, 2024, GitHub Actions customers will no longer be able to use v3 of actions/upload-artifact
[...]
attempting to use a version of the actions after the deprecation date will result in a workflow failure
Incompatibility Between New Action Version and Linux Build Container
Starting from version 4.0.0, the actions/upload-artifact action is executed using the Node.js 20 runtime (actions/upload-artifact#448). The build of that runtime used by the action has a dependency on version 2.28 of the glibc shared library, while the container used by the build workflow for the Linux build of Arduino IDE has glibc 2.27.
This causes the Linux build job to fail after the update of the action:
Run actions/upload-artifact@v4
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
Impact of Updating Linux Build Container
The only solution I could find for the incompatibility of actions/upload-artifact@v4 with the Linux build container was to update the Linux distro used in the container to one that has glibc 2.28 (Ubuntu 18.10).
I have implemented that in 99df637 (#2307) (which I pushed to this PR along with other commits for the non-impactful adjustments required to accommodate the action bump).
The reason we perform the Linux build in a container with an older distro version (#2253) is because we want to support the use of Arduino IDE with a reasonable range of Linux distro versions.
The Linux build of Arduino IDE has dynamic linkage against some shared libraries. This results in it having a dependency on the version of the libraries that happens to be present in the environment it is built in. So the version of these libraries installed in the machine in which the build is performed determines the application's compatibility with older distro versions.
So updating the container to using Ubuntu 18.10 will cause the loss of compatibility of Arduino IDE with Linux distro versions that use glibc 2.27 (e.g., Ubuntu 18.04). User of those distros will find that Arduino IDE fails to start with an error like:
Error: node-loader:
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/foo/arduino-ide/resources/app/lib/backend/native/pty.node)
at 85467 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:2766)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 23571 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3374073)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 55444 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3369761)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 24290 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1780542)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 43416 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1770138)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
When there were some previous instances of a temporary loss of compatibility with older distro versions, it did impact users (e.g., #2018, #259).
Unfortunately I haven't found any other way forward, but I would like to solicit opinions/ideas from Tooling Team members before merging this PR.
Mitigation
Communication
The loss of support for some older Linux distro versions must be clearly communicated via the release notes and release announcement for the subsequent release.
Final Compatible Release
Ideally we would make a release of Arduino IDE before the compatibility regression is introduced by the merge of this PR. This would allow the impacted users who can't/won't upgrade their OS and so are stuck at the last compatible IDE version to at least benefit from all the development work up to this point. This would be especially beneficial since 2.3.3 has some significant regressions (#2562, #2516) which have been fixed since the 2.3.3 release.
However, the release would have to happen before the date when the current version of the build workflow will stop working per GitHub's deprecation notice: 2024-12-05.
Notes for Reviewers
The build workflow runs fail spuriously even after the changes I pushed to make it compatible with the new action version:
https://github.com/arduino/arduino-ide/actions/runs/11985563031/job/33417625788?pr=2307#step:4:117
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
This is expected. The reason for the failure is that the Linux build is using the image built from the version of the Dockerfile in the main branch, and so the container still has Ubuntu 18.04. The new image based on the updates to the Dockerfile I propose here will only be published once the PR is merged into the main branch (the publishing is performed automatically). After that, subsequent runs of the workflow will use the new image and should pass.
I performed a demonstration release with the changes proposed here in my fork (using an image I published under my account):
https://github.com/per1234/arduino-ide/actions/runs/11976699348
https://github.com/per1234/arduino-ide/releases/tag/0.0.0-rc.48
per1234
requested review from
ubidefeo,
giacomocusinato,
umbynos and
davegarthsimpson
There was a problem hiding this comment.
Ideally we would make a release of Arduino IDE before the compatibility regression is introduced by the merge of this PR
Unfortunately this is no longer possible. The reason is that, starting earlier today, GitHub is forcing actions that specify the Node 16 runtime to use Node 20 instead. That broke the build workflow's compatibility with the Ubuntu 18.04 Linux build container, even without the changes proposed in this PR:
https://github.com/arduino/arduino-ide/actions/runs/12050901556/job/33600783618#step:5:1
Run actions/checkout@v3
/usr/bin/docker exec 79b28bbec4b91d51299893c77b93af952833388b64649b4e26f604c01531aa19 sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
So there is now no reason to delay the merge of this PR.
per1234
deleted the
dependabot/github_actions/actions/upload-artifact-4
branch
Bumps actions/upload-artifact from 3 to 4.
Release notes
Sourced from actions/upload-artifact's releases.
Commits
c7d193fMerge pull request #466 from actions/v4-beta13131bblicensed cache4a6c273Merge branch 'main' into v4-betaf391bb9Merge pull request #465 from actions/robherley/v4-documentation9653d03Apply suggestions from code review875b630add limitations sectionecb2146add compression example5e7604ftrim some repeated infod6437d0naming1b56155s/v4-beta/v4/gYou can trigger a rebase of this PR by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot mergewill merge this PR after your CI passes on it@dependabot squash and mergewill squash and merge this PR after your CI passes on it@dependabot cancel mergewill cancel a previously requested merge and block automerging@dependabot reopenwill reopen this PR if it is closed@dependabot closewill close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)