Cannot copy multiarch images to immutable container registries

[vincentburesi-es]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Using the command

docker buildx imagetools create -t <dest-registry>:<new-version> <src-registry>:<old-version>

to copy a multiarch image (manifest list tagged with + multiple arch-specific images also on ) doesn't work when dest is an immutable registry (tested on Amazon ECR set to immutable, different from the src registry)

Expected behaviour

I expect only the top level manifest list to be pushed with the tag, not the images themselves.

Actual behaviour

Returns 400 error on the second image pushed.

The registry contains a leftover first image, tagged with .

It looks like all arch-specific images are pushed with tag, and any push after the first is rejected due to registry immutability.

The result is fine on a mutable registry, but i believe it is because the manifest list is pushed last and therefore overwrites the tag last.

Buildx version

github.com/docker/buildx v0.19.3 48d6a39

Docker info

Client: Docker Engine - Community
 Version:    27.5.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.19.3
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.4
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 7
  Running: 2
  Paused: 0
  Stopped: 5
 Images: 15
 Server Version: 27.5.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-51-generic
 Operating System: Linux Mint 22
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.31GiB
 Name: vincent-PC-FIXE
 ID: X6N3:3LYD:YOX5:YY7Q:EIOU:C7BA:TG4E:MBJJ:QWFB:SEAD:MZE5:KKYR
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Builders list

NAME/NODE        DRIVER/ENDPOINT                   STATUS    BUILDKIT   PLATFORMS
mybuilder*       docker-container                                       
 \_ mybuilder0    \_ unix:///var/run/docker.sock   running   v0.16.0    linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (7 more)
default          docker                                                 
 \_ default       \_ default                       running   v0.18.2    linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (7 more)

Configuration

Images were built locally with

docker buildx build --platform $PLATFORM_LIST -t <src-registry>:<old-version> .

then sent to the registry through

docker buildx build --push --platform $PLATFORM_LIST -t <src-registry>:<old-version> .

(the exact same command with --push added) and were then tested to be working on both archs.

Build logs

Additional info

Pushing multiarch images to a repository seems to work fine in the docker buildx build --push --platform $PLATFORM_LIST command, I can only infer that the push process used in the docker buildx imagetools create command is different, maybe it is possible to compare the code of the two and find the issue this way ?