Openwrt 23.05 #83
Merged
Openwrt 23.05 #83
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Drop redundant generare ccache hash job as that can be done by integrated github expressions to generate an hash. The only change is that the integrated way generate a sha256 hash instead of an md5 sum. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 457f6b0)
Package external toolchain after correct build. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit ce2e7c5)
Add option to configure container to use for build test. By default the tools container is used if no option is provided. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 803b011)
Add checks to test if toolchain container can be used. This is to handle case of new target or migration of any sort. If the toolchain container can't be found, the tools container is used instead. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 23a5c71)
Build and push container with external toolchain embedded in the container image. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit e1370cd)
Add support to use container included external toolchain and skip redownloading external sdk for each test. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 0fe5776)
Due to problem with migrating from master to main as the default branch and downstream project still requiring the master branch to be present, we currently have for push events double CI runs, one for main and one for master. To solve this ignore any push event to the master branch for every workflow that react on push events. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit f5a5ce8)
Split caching ccache in separate restore and save jobs to always refresh the ccache across different runs. Currently if a key is restored, cache is not saved resulting in a less useful ccache that benefits from multiple runs. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 6321361)
Github Actions cache doesn't permit to overwrite cache if it does already exist. As a trick to refresh and have fresh ccache pool, delete the ccache cache if it does exist with the help of Github REST API. An additional permission is needed to access this API. Add this permittion to each user of the build workflow. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 203cc0a)
Add option to disable use of ccache. This can be useful for some sensible test that should not use ccache as they can cause side effects of any sort. (example Coverity Scan) Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit b9a41c1)
Add new input to define custom ccache cache type. This is useful to use a different ccache cache for some special workflow that may do more test than simple kernel compilation. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 07b52a8)
CCache cache is currently broken due to a funny bug in ccache compiler type detection. It seems ccache compiler type detection is very fragile and with the use of external toolchain doesn't correctly detect the type. The type detected is set to other instead of gcc resulting in ccache complaining for unsupported compiler options. To handle this problem, force the compiler type to gcc to make ccache correctly work and speedup compilation. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit ae7b053)
Disable ccache usage for coverity workflow as it may cause side effect in the produced bins. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 2129ee1)
Limit ccache cache save/delete only on push events. Saving ccache cache for pull request will result in bloat and refreshing ccache is not possible due to security measure on enforcing read permission on pull_request events. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit ff66a7c)
Add support for getting ccache cache from S3. ccache is archieved in a tar and downloaded from S3 Cloud Storage. For push events, ccache is then uplodaed back to S3 to refresh and have a ccache cache always fresh. An additional workflow is added to upload files to an S3 Cloud Storage from artifacts uplodaed to github. The minio tool is used to upload files to S3. If the ccache can't be downloaded from s3, we fallback to github cache system. Also limit s3 upload to the openwrt repository since external fork won't have (obviously) the required secrtes to upload data to the S3 Cloud Storage. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit ebbc806)
Test each subtarget on push events to improve testing and to refresh ccache of each subtarget. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 5bafc43)
Its being used by buildbot workers, adds g++-multilib to fix node cross-compilation from a 64-bit build machine to 32-bit host. References: openwrt/buildbot#7 Signed-off-by: Petr Štetiar <[email protected]> (cherry picked from commit 5677841)
CDNs are known to ship outdated or corrupted files, if it unpacks correctly, it necessarily doesn't mean, that we're using the desired content. So lets fix it by checking the tarball as well. I'm adding GPG checking explicitly, its not needed, but just double checking, that everything is working as expected on build infrastructure. Signed-off-by: Petr Štetiar <[email protected]> (cherry picked from commit 95dde52)
Add support to label-kernel for compiling testing kernel version and check patches. To trigger this special build appent :testing to the normal label. Example: - ci:kernel:ipq806x:generic:testing Test will fail if the requested target doesn't have a defined kernel testing version. Also add support for testing all target and subtarget. To trigger this some special pattern are added: - ci:kernel:all:all Trigger test for all target and subtarget - ci:kernel:all:first Trigger test for all target and the first subtarget in alphabetical order for the target. With these special case :testing can also be used and every target and subtarget that supports kernel testing version will be selected: - ci:kernel:all:all:testing Trigger test for all target and subtarget that have a kernel testing version defined. - ci:kernel:all:first:testing Trigger test for all target and the first subtarget in alphabetical order for the target that, if they have a kernel testing version defined. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 218deba)
Now that we build also core packages, we need more host tools. Compile all of them to reduce compile time on other actions. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit de9955a)
Migrate each workflow to use reusable workflow from dedicated repo to skip pushing CI related commits to openwrt and better track versioning of CI workflow. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 38cc091)
Drop unused reusable workflow and dockerfiles now that we moved them to a dedicated repository. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 14293dd)
Generilize S3 secret keys and rename to make them not platform specific. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit f98dc5a)
Provide new required secret for S3 endpoint and bucket name to permit an easier migration to new services. Signed-off-by: Christian Marangi <[email protected]> (cherry picked from commit 881235c)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks for your contribution to OpenWrt!
To help keep the codebase consistent and readable,
and to help people review your contribution,
we ask you to follow the rules you find in the wiki at this link
https://openwrt.org/submitting-patches
Please remove this message before posting the pull request.