From a65004566fd14af9805d1d9afcd3665e1ed069d4 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 7 Nov 2023 15:58:00 -0500 Subject: [PATCH] Change flag --platform to use --target, and adding a field to override the path for binaries in buildpack.toml Signed-off-by: Juan Bustamante --- text/0000-multiarch-builders-and-package.md | 119 ++++++++++---------- 1 file changed, 61 insertions(+), 58 deletions(-) diff --git a/text/0000-multiarch-builders-and-package.md b/text/0000-multiarch-builders-and-package.md index aeaa88d5f..3bcd0cd7e 100644 --- a/text/0000-multiarch-builders-and-package.md +++ b/text/0000-multiarch-builders-and-package.md @@ -181,7 +181,7 @@ pack builder create my-jammy-builder --config ./builder.toml \ --publish ``` -In this case, because `targets` are specified and `--multi-arch` is being used `pack` will follow the builder creation process for **each provided platform**, +In this case, because `targets` are specified and `--multi-arch` is being used `pack` will follow the builder creation process for **each provided target**, pulling the correct (based on os/arch) buildpacks, build and run images and creating different builders images that are exported and combined into an [image index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) @@ -201,15 +201,15 @@ As a quick summary, our current process to create a buildpack package involves: We propose: - Deprecate the `platform.os` field from [package.toml](https://buildpacks.io/docs/reference/config/package-config/). It will be removed after two pack releases with the new feature - When `platform.os` is present in [package.toml](https://buildpacks.io/docs/reference/config/package-config/), throw a warning messages indicating the field will be removed -and `--platform` flag must be used -- When `platform.os` is not present [package.toml](https://buildpacks.io/docs/reference/config/package-config/) and `--platform` flag is not used, throw a warning messages indicating -a new `--platform` flag is available +and `--target` flag must be used +- When `platform.os` is not present [package.toml](https://buildpacks.io/docs/reference/config/package-config/) and `--target` flag is not used, throw a warning messages indicating +a new `--target` flag is available - Keep doing our current process to package a buildpack ### To improve user experience We propose: -- Add a new `--platform` flag with format `[os][/arch][/variant]:[name@version]` to build for a particular target, once the `platform.os` field is removed, +- Add a new `--target` flag with format `[os][/arch][/variant]:[name@version]` to build for a particular target, once the `platform.os` field is removed, this will be the way for end-users to specify the platform for which they want to create single OCI artifact. > **Note** > We want to avoid duplicated information, we expect to have `targets` with all the platform supported by a buildpack @@ -237,6 +237,9 @@ this flag must be used in conjunction with `--publish` or `--format file` flags > **Note** > For cross-compile buildpacks like Paketo, it looks easy to add a step to their Makefile to compile and separate the binaries following this structure. +In case, this folder structure is not suitable for Buildpack Authors, a new `path` attribute can be included into the `target` section of the `buildpack.toml` to specify +where the binaries are located in the filesystem. + Based on the [RFC-0096](https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md) we replaced the concept of `stacks` in `buildpack.toml` by a `target` which looks like: ```toml @@ -244,6 +247,7 @@ Based on the [RFC-0096](https://github.com/buildpacks/rfcs/blob/main/text/0096-r os = "" arch = "" variant = "" +path = "" #optional [[targets.distributions]] name = "" versions = [""] @@ -278,16 +282,16 @@ In these cases, the expected output will be similar to: ```bash pack buildpack package --config ./package.toml --publish -Warning: A new '--platform' flag is available to set the target platform for the buildpack package, using 'linux' as default +Warning: A new '--target' flag is available to set the platform for the buildpack package, using 'linux' as default Successfully published package and saved to registry # Or pack buildpack package --config ./package.toml --format file -Warning: A new '--platform' flag is available to set the target platform for the buildpack package, using 'linux' as default +Warning: A new '--target' flag is available to set the platform for the buildpack package, using 'linux' as default Successfully created package and saved to file ``` -We will keep `linux` as the default platform, and we don't expect end-users to change anything else on their side. +We will keep `linux` as the default target platform, and we don't expect end-users to change anything else on their side. In case, Buildpack authors tries to use the new folder structure like: @@ -308,8 +312,7 @@ The current `pack` version creates an OCI artifact copying all the files on it. ![](https://hackmd.io/_uploads/BkGkZ_mla.png) -But I think this buildpack can't be executed because `./bin/build` or `./bin/detect` can't be found. We could add some validations and throw an error, -but I do not see any value on doing that. +But I think this buildpack can't be executed because `./bin/build` or `./bin/detect` can't be found. We could add some validations and throw an error, but I do not see any value on doing that. > **Important** > New folder structure is not useful if Buildpack Authors don't remove stacks and migrate to use targets @@ -319,11 +322,11 @@ Trying to use the new flags ```bash pack buildpack package --config ./package.toml --publish --multi-arch -Error: 'targets' or 'platforms' must be defined when creating a multi-architecture buildpack +Error: 'targets' must be defined when creating a multi-architecture buildpack # Or pack buildpack package --config ./package.toml --format file --multi-arch -Error: 'targets' or 'platforms' must be defined when creating a multi-architecture buildpack +Error: 'targets' must be defined when creating a multi-architecture buildpack ``` In these cases, we don't have enough information to create a multi-arch buildpack, and pack must fail its execution. @@ -331,32 +334,32 @@ In these cases, we don't have enough information to create a multi-arch buildpac A valid way to re-write the command could be: ```bash -pack buildpack package --config ./package.toml --publish --platform linux/arm64 --platform linux/amd64 --multi-arch -A multi-arch buildpack package will be created for platforms: 'linux/amd64', 'linux/arm64' +pack buildpack package --config ./package.toml --publish --target linux/arm64 --target linux/amd64 --multi-arch +A multi-arch buildpack package will be created for target platforms: 'linux/amd64', 'linux/arm64' Successfully published package and saved to registry # Or -pack buildpack package --config ./package.toml --format file --platform linux/arm64 --platform linux/amd64 --multi-arch -A multi-arch buildpack package will be created for platforms: 'linux/amd64', 'linux/arm64' +pack buildpack package --config ./package.toml --format file --target linux/arm64 --target linux/amd64 --multi-arch +A multi-arch buildpack package will be created for target platforms: 'linux/amd64', 'linux/arm64' Successfully created package and saved to file ``` In these cases, two OCI images, with the same binaries, will be created and pushed into the registry, for each image the configuration file will be created with the correct `os` and `architecture` and an [image index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) will be created to combine them -what about creating a single image for a new platform? +what about creating a single image for a new target platform? ```bash -pack buildpack package --config ./package.toml --publish --platform linux/arm64 +pack buildpack package --config ./package.toml --publish --target linux/arm64 Successfully published package and saved to registry # Or -pack buildpack package --config ./package.toml --format file --platform linux/arm64 +pack buildpack package --config ./package.toml --format file --target linux/arm64 Successfully created package and saved to file ``` In these cases, pack will create buildpack package image with the provided binaries and a [configuration](https://github.com/opencontainers/image-spec/blob/main/config.md#properties) -with the following platform: +with the following target platform: ```json { @@ -371,12 +374,12 @@ These cases are similar to the previous one, but we can change the warning messa ```bash pack buildpack package --config ./package.toml --publish -Warning: 'platform.os' field in package.toml will be deprecated, use new '--platform' flag to set target platform +Warning: 'platform.os' field in package.toml will be deprecated, use new '--target' flag to set target platform Successfully published package and saved to registry # Or pack buildpack package --config ./package.toml --format file -Warning: 'platform.os' field in package.toml will be deprecated, use new '--platform' flag to set target platform +Warning: 'platform.os' field in package.toml will be deprecated, use new '--target' flag to set target platform Successfully created package and saved to file ``` @@ -413,19 +416,19 @@ this will avoid breaking the current behavior. ```bash pack buildpack package --config ./package.toml --publish Warning: Multiple targets 'linux/amd64', 'linux/arm64' are present at 'buildpack.toml' -Warning: A new '--platform' flag is available to set the target platform for the buildpack package, using 'linux' as default +Warning: A new '--target' flag is available to set the target platform for the buildpack package, using 'linux' as default Successfully published package and saved to registry # Or pack buildpack package --config ./package.toml --format file Warning: Multiple targets 'linux/amd64', 'linux/arm64' are present at 'buildpack.toml' -Warning: A new '--platform' flag is available to set the target platform for the buildpack package, using 'linux' as default +Warning: A new '--target' flag is available to set the target platform for the buildpack package, using 'linux' as default Successfully created package and saved to file # Or pack buildpack package --config ./package.toml Warning: Multiple targets 'linux/amd64', 'linux/arm64' are present at 'buildpack.toml' -Warning: A new '--platform' flag is available to set the target platform for the buildpack package, using 'linux' as default +Warning: A new '--target' flag is available to set the target platform for the buildpack package, using 'linux' as default Successfully created package and saved to docker daemon ``` @@ -438,10 +441,10 @@ The OCI Image [configuration](https://github.com/opencontainers/image-spec/blob/ } ``` -On the other hand, when end-users use the new `platform` flag +On the other hand, when end-users use the new `target` flag ```bash -pack buildpack package --config ./package.toml --publish --platform linux/arm64 +pack buildpack package --config ./package.toml --publish --target linux/arm64 Successfully published package and saved to registry ``` @@ -457,15 +460,15 @@ The OCI Image [configuration](https://github.com/opencontainers/image-spec/blob/ Trying to create a buildpack package for an unsupported platform should throw an error ```bash -pack buildpack package --config ./package.toml --publish --platform windows/arm64 -Error: platform 'windows/arm64' is not supported in buildpack targets, check your 'buildpack.toml' +pack buildpack package --config ./package.toml --publish --target windows/arm64 +Error: target platform 'windows/arm64' is not supported in buildpack targets, check your 'buildpack.toml' ``` Let's go one more step further, and check what will happen if Buildpack Authors use the new `--multi-arch` flag ```bash pack buildpack package --config ./package.toml --publish --multi-arch -Info: A multi-arch buildpack package will be created for platforms: 'linux/amd64', 'linux/arm64' +Info: A multi-arch buildpack package will be created for targets platforms: 'linux/amd64', 'linux/arm64' Successfully published package and saved to registry ``` @@ -560,10 +563,10 @@ name = "windows" versions = ["10.0.20348.1970"] ``` -If the Buildpack Author wants to create a single buildpack package they will use the `platform` flag +If the Buildpack Author wants to create a single buildpack package they will use the `target` flag ```bash -pack buildpack package --config ./package.toml --publish --platform linux/arm64 +pack buildpack package --config ./package.toml --publish --target linux/arm64 Successfully published package and saved to registry ``` @@ -591,12 +594,12 @@ A fully multi-arch buildpack could be created as follows: ```bash pack buildpack package --config ./package.toml --publish --multi-arch -Info: A multi-arch buildpack package will be created for platforms: 'linux/amd64', 'linux/arm64', 'windows/amd64' +Info: A multi-arch buildpack package will be created for target platforms: 'linux/amd64', 'linux/arm64', 'windows/amd64' Successfully published package and saved to registry ``` In this case, three OCI images will be created and pushed into the registry, for each image the configuration file will be -created with the correct platform: `os` and `architecture`, +created with the correct target: `os` and `architecture`, an [image index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) will be created to combine them, with a content similar to: @@ -639,7 +642,7 @@ with a content similar to: ``` > **Important** -> In this case, each image has different binaries according to the platform and the new folder structure +> In this case, each image has different binaries according to the target platform and the new folder structure ## Builder @@ -659,7 +662,7 @@ We propose: - When `stacks` is present in [builder.toml](https://buildpacks.io/docs/reference/config/builder-config/), throw a warning message indicating the field is deprecated and it will be removed - When `targets` is not present in [builder.toml](https://buildpacks.io/docs/reference/config/builder-config/), throw a warning messages indicating - a new `--platform` flag is available + a new `--target` flag is available - Keep doing our current process to create a builder ### To improve user experience @@ -685,7 +688,7 @@ image = "" # MUST point to an Image Index that matches targets image = "" -# Platforms to support with the Builder +# Target platforms to support with the Builder [[targets]] os = "" arch = "" @@ -694,7 +697,7 @@ variant = "" name = "" versions = [""] ``` -- Add a new `--platform` optional flag with format `[os][/arch][/variant]:[name@version]` to create a builder for a particular target, this will help end-users to specify the platform for which they want to create single OCI artifact. +- Add a new `--target` optional flag with format `[os][/arch][/variant]:[name@version]` to create a builder for a particular target, this will help end-users to specify the platform for which they want to create single OCI artifact. - Add a new boolean `--multi-arch` flag to indicate pack it must create multiples OCI artifacts and combine them with an [image index](https://github.com/opencontainers/image-spec/blob/master/image-index.md), this flag must be used in conjunction with `--publish` or `--format file` flags and error out when `daemon` is selected. @@ -739,7 +742,7 @@ In these cases, the expected output will be similar to: ```bash pack builder create --config ./builder.toml Warning: "stack" has been deprecated, prefer "targets" instead: https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md -Warning: A new '--platform' flag is available to set the target platform for the builder, using 'linux/amd64' as default +Warning: A new '--target' flag is available to set the target platform for the builder, using 'linux/amd64' as default Successfully created builder image Tip: Run pack build --builder to use this builder ``` @@ -749,14 +752,14 @@ printed to help end-users to check for new updates, maybe link to a migration gu Trying to use the new flags: ```bash -pack builder create --config ./builder.toml --platform linux/arm64 +pack builder create --config ./builder.toml --target linux/arm64 Warning: "stack" has been deprecated, prefer "targets" instead: https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md Warning: creating a builder for platform "linux/arm64" but "targets" is not defined, update your "builder.toml" to include "targets" Successfully created builder image Tip: Run pack build --builder to use this builder ``` -Pulling operations will be configured to use `linux/arm64` as platform, +Pulling operations will be configured to use `linux/arm64` as target platform, the OCI Image [configuration](https://github.com/opencontainers/image-spec/blob/main/config.md#properties) file will have: ```json @@ -770,25 +773,25 @@ What about multi-architecture builders? ```bash pack builder create --config ./builder.toml --multi-arch --publish -Error: 'targets' or 'platforms' must be defined when creating a multi-architecture builder +Error: 'targets' must be defined when creating a multi-architecture builder ``` In this case, pack doesn't have enough information to create a multi-arch builder and fail its execution. -Using `platform` flag: +Using `target` flag: ```bash pack builder create --config ./builder.toml \ --multi-arch \ - --plaform linux/amd64 \ - --platform linux/arm64 \ + --target linux/amd64 \ + --target linux/arm64 \ --publish Successfully created builder image Tip: Run pack build --builder to use this builder ``` In this case, two OCI images will be created and pushed into the registry, for each image the configuration file will be -created with the correct platform: `os` and `architecture`, +created with the correct target: `os` and `architecture`, an [image index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) will be created to combine them #### `Targets` are present in `builder.toml` @@ -825,24 +828,24 @@ Let's suppose we execute the command in a host "linux/amd64" machine ```bash pack builder create --config ./builder.toml -Info: creating a builder for platform "linux/amd64" +Info: creating a builder for target "linux/amd64" Successfully created builder image Tip: Run pack build --builder to use this builder ``` -`platform` flag is not defined, we keep our current behavior and detect the host `os` and `architecture`. Because a +`target` flag is not defined, we keep our current behavior and detect the host `os` and `architecture`. Because a `target` matches the host `platform` it is used to create the builder. Trying to use the new flags: ```bash -pack builder create --config ./builder.toml --platform linux/arm64 -Info: creating a builder for platform "linux/arm64" +pack builder create --config ./builder.toml --target linux/arm64 +Info: creating a builder for target "linux/arm64" Successfully created builder image Tip: Run pack build --builder to use this builder ``` -Pulling operations will be configured to use `linux/arm64` as platform, +Pulling operations will be configured to use `linux/arm64` as target platform, the OCI Image [configuration](https://github.com/opencontainers/image-spec/blob/main/config.md#properties) file will have: ```json @@ -856,27 +859,27 @@ What about multi-architecture builders? ```bash pack builder create --config ./builder.toml --multi-arch --publish -Info: A multi-arch builder will be created for platforms: 'linux/amd64', 'linux/arm64' +Info: A multi-arch builder will be created for targets platform: 'linux/amd64', 'linux/arm64' Successfully created builder image Tip: Run pack build --builder to use this builder ``` -Using `platform` flag: +Using `target` flag: ```bash pack builder create --config ./builder.toml \ --multi-arch \ - --plaform linux/amd64 \ - --platform linux/arm64 \ + --target linux/amd64 \ + --target linux/arm64 \ --publish -Info: A multi-arch builder will be created for platforms: 'linux/amd64', 'linux/arm64' +Info: A multi-arch builder will be created for targets platform: 'linux/amd64', 'linux/arm64' Successfully created builder image Tip: Run pack build --builder to use this builder ``` In this case, two OCI images will be created and pushed into the registry, for each image the configuration file will be -created with the correct platform: `os` and `architecture`, +created with the correct target platform: `os` and `architecture`, an [image index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) will be created to combine them @@ -913,7 +916,7 @@ create [image indexes](https://github.com/opencontainers/image-spec/blob/master/ [unresolved-questions]: #unresolved-questions - If I already have an [image indexe](https://github.com/opencontainers/image-spec/blob/master/image-index.md) created and I want to add support for new plaform, how do I do it? -- What are the intermediate images for each platform named/called? +- What are the intermediate images for each target named/called? - What happen if I want to exclude some buildpack for a particular target? - What happen if I want to include the same file or folder for every image, do I have to copy then inside the {os}-{arch} folder?