IID not written when using docker CLI

[paulomorgado]

Issue Description

docker build --iidfile <path to iidfile> ... produces an empty file while podman build --iidfile <path to iidfile> ... produces a file with the image ID.

Steps to reproduce the issue

Steps to reproduce the issue

1. run docker build --iidfile iid.txt -f Dockerfile .
   1. the iid.txt file is empty
2. run podman build --iidfile iid.txt -f Dockerfile .
   1. the iid.txt file has the image ID

Describe the results you received

No image ID written to the iidfile when using docker CLI.

Describe the results you expected

Image ID written to the iidfile when using docker CLI.

podman info output

PS> docker build -f .\Dockerfile --iidfile .\tmp\iid.txt .
[+] Building 2.5s (19/19) FINISHED                                                                                                              docker-container:default
 => [internal] load build definition from Dockerfile                                                                                                                0.1s
 => => transferring dockerfile: 2.23kB                                                                                                                              0.0s
 => resolve image config for docker-image://docker.io/docker/dockerfile:1.6                                                                                         1.3s
 => CACHED docker-image://docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021                                   0.0s
 => => resolve docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021                                              0.0s
 => [internal] load build definition from Dockerfile                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/debian:12-slim                                                                                                   0.9s
 => [internal] load .dockerignore                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                     0.0s
 => [internal] preparing inline document                                                                                                                            0.0s
 => [internal] preparing inline document                                                                                                                            0.0s
 => [1/8] FROM docker.io/library/debian:12-slim@sha256:ca3372ce30b03a591ec573ea975ad8b0ecaf0eb17a354416741f8001bbcae33d                                             0.0s
 => => resolve docker.io/library/debian:12-slim@sha256:ca3372ce30b03a591ec573ea975ad8b0ecaf0eb17a354416741f8001bbcae33d                                             0.0s
 => [internal] load build context                                                                                                                                   0.0s
 => => transferring context: 80B                                                                                                                                    0.0s
 => [internal] preparing inline document                                                                                                                            0.0s
 => [internal] preparing inline document                                                                                                                            0.0s
 => CACHED [2/8] RUN <<'RUN_EOF' (#!/bin/bash...)                                                                                                                   0.0s
 => CACHED [3/8] RUN update-ca-certificates                                                                                                                         0.0s
 => CACHED [4/8] RUN <<'RUN_EOF' (#!/bin/bash...)                                                                                                                   0.0s
 => CACHED [5/8] RUN <<'RUN_EOF' (#!/bin/bash...)                                                                                                                   0.0s
 => CACHED [6/8] RUN <<'RUN_EOF' (#!/bin/bash...)                                                                                                                   0.0s
 => CACHED [7/8] COPY --chmod=755 terraform-outputs-to-tfvars.py /usr/local/bin/                                                                                    0.0s
 => CACHED [8/8] COPY --chmod=755 build.sh /usr/local/bin/                                                                                                          0.0s
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
PS> cat .\tmp\iid.txt
PS> podman build -f .\Dockerfile --iidfile .\tmp\iid.txt .
STEP 1/9: FROM debian:12-slim
STEP 2/9: RUN <<'RUN_EOF' (#!/bin/bash...)
--> Using cache be31f374d319e97eef9857dcbdd50f983c64ab7919a07f2504edbcac2b0780a6
--> be31f374d319
STEP 3/9: RUN update-ca-certificates
--> Using cache 5db90e28535565a3d10db7d79a0eebad716a478344a4a1abc4887b9325f46394
--> 5db90e285355
STEP 4/9: RUN <<'RUN_EOF' (#!/bin/bash...)
--> Using cache f5d8a88f2d1d5359d6f386f4e0029740a8bd9f5a0610df7f6ab4f38be70b5051
--> f5d8a88f2d1d
STEP 5/9: RUN <<'RUN_EOF' (#!/bin/bash...)
--> Using cache 1de49e144d06947e829031b7f507f319b5d5d86e4702fcdb6daaf40038264004
--> 1de49e144d06
STEP 6/9: RUN <<'RUN_EOF' (#!/bin/bash...)
--> Using cache 4c483af2f0a60a7c5ce204ad4e0943ef20da59b2a2b30c0126afbaae8bf60e0e
--> 4c483af2f0a6
STEP 7/9: COPY --chmod=755 terraform-outputs-to-tfvars.py /usr/local/bin/
--> Using cache 5ef146e6c468f81eba7a3197f1abcd9b843d3a2cf398c58a4ba953dda45d4ac7
--> 5ef146e6c468
STEP 8/9: COPY --chmod=755 build.sh /usr/local/bin/
--> Using cache 05daa8030e7caf42a6e7432dd3d3818e38547d2eb98fe792e6af167989513f2e
--> 05daa8030e7c
STEP 9/9: ENTRYPOINT ["build.sh"]
--> Using cache bcea3cd5427687ff2c6d90613ee7cd1a9cec645c7e8a0d77294e6c7ab35a56a0
--> bcea3cd54276bcea3cd5427687ff2c6d90613ee7cd1a9cec645c7e8a0d77294e6c7ab35a56a0
PS> cat .\tmp\iid.txt
sha256:bcea3cd5427687ff2c6d90613ee7cd1a9cec645c7e8a0d77294e6c7ab35a56a0

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

Windows 11

Client:       Podman Engine
Version:      5.3.0
API Version:  5.3.0
Go Version:   go1.23.3
Git Commit:   874bf2c301ecf0ba645f1bb45f81966cc755b7da
Built:        Wed Nov 13 13:19:59 2024
OS/Arch:      windows/amd64
 
Server:       Podman Engine
Version:      5.2.3
API Version:  5.2.3
Go Version:   go1.22.7
Built:        Tue Sep 24 01:00:00 2024
OS/Arch:      linux/amd64

Additional information

No response

[rhatdan]

What do you think should be default behaviour?

$ podman build --help | grep iid
      --iidfile file                                 file to write the image ID to
$ docker build --help | grep iid
      --iidfile string                Write the image ID to a file

If this is real docker not writing to the file, then it seems like a Docker bug?

[paulomorgado]

docker is writing nothing to the file when using podman engine, but writes the image ID when using docker engine.

Looks like something in the API that podman exposes to docker.

[rhatdan]

Ok do docker->Podman.sock, ends up with nothing in the file, which means we are somehow not responding the way docker expects.

[paulomorgado]

That's probably it.

I need to debug a bit more to figure out what exactly is missing or out of place.

[paulomorgado]

I built this simple C# program:

using Docker.DotNet;
using Docker.DotNet.Models;
using System.Formats.Tar;

using var contents = new MemoryStream();
using (var tarWriter = new TarWriter(contents, TarEntryFormat.Pax, true))
{
    using var dockerfile = new MemoryStream();
    dockerfile.Write(
        """
        FROM alpine:latest

        RUN echo 'x' > x.txt
        """u8);
    dockerfile.Seek(0, SeekOrigin.Begin);

    var tarEntry = new PaxTarEntry(TarEntryType.RegularFile, "Dockerfile");
    tarEntry.DataStream = dockerfile;

    await tarWriter.WriteEntryAsync(tarEntry);
}
contents.Seek(0, SeekOrigin.Begin);

using var configuration = new DockerClientConfiguration();

var client = configuration.CreateClient();

var imageBuildParameters = new ImageBuildParameters
{
    Dockerfile = "Dockerfile",
    Tags = ["test"],
};

await client.Images.BuildImageFromDockerfileAsync(
    imageBuildParameters,
    contents,
    [],
    new Dictionary<string, string>(),
    new Progress<JSONMessage>(value =>
    {
        if (!string.IsNullOrEmpty(value.Stream))
        {
            Console.WriteLine($"Stream: {value.Stream}");
        }

        if (!string.IsNullOrEmpty(value.Status))
        {
            Console.WriteLine($"Status: {value.Status}");
        }

        if (value.Error is { } error)
        {
            Console.WriteLine($"Error {error.Code}: {error.Message}");
            throw new Exception(error.Message);
        }

        if (!string.IsNullOrEmpty(value.ProgressMessage))
        {
            Console.WriteLine($"Progress: {value.ProgressMessage}");
        }

        if (!string.IsNullOrEmpty(value.ID))
        {
            Console.WriteLine($"ID: {value.ID}");
        }
    }));

And got this output from Rancher Desktop engine:

Stream: Step 1/2 : FROM alpine:latest
Stream:

Stream:  ---> 63b790fccc90

Stream: Step 2/2 : RUN echo 'x' > x.txt
Stream:

Stream:  ---> Using cache

Stream:  ---> 6b7927809dfe

Stream: Successfully built 6b7927809dfe

Stream: Successfully tagged test:latest

And this from Podman engine:

Stream: STEP 1/2: FROM alpine:latest
 
Stream: STEP 2/2: RUN echo 'x' > x.txt
 
Stream: --> Using cache 0e707b89078f1e6c085941ad61de66d5439299bae750e028fe3ee48bf22eaa1b
 
Stream: COMMIT docker.io/library/test:latest
 
Stream: --> 0e707b89078f
 
Stream: Successfully tagged docker.io/library/test:latest
 
Stream: 0e707b89078f1e6c085941ad61de66d5439299bae750e028fe3ee48bf22eaa1b
 
Stream: Successfully built 0e707b89078f
 
Stream: Successfully tagged test

So, not much difference here.

But using this docker CLI:

Client:
Version:           27.3.1
API version:       1.41 (downgraded from 1.47)
Go version:        go1.22.7
Git commit:        ce12230
Built:             Fri Sep 20 11:42:27 2024
OS/Arch:           windows/amd64
Context:           default
 
Server: linux/amd64/fedora-40
Podman Engine:
  Version:          5.2.3
  APIVersion:       5.2.3
  Arch:             amd64
  BuildTime:        2024-09-24T01:00:00+01:00
  Experimental:     false
  GitCommit:
  GoVersion:        go1.22.7
  KernelVersion:    5.15.167.4-microsoft-standard-WSL2
  MinAPIVersion:    4.0.0
  Os:               linux
Conmon:
  Version:          conmon version 2.1.12, commit:
  Package:          conmon-2.1.12-2.fc40.x86_64
OCI Runtime (crun):
  Version:          crun version 1.17
commit: 000fa0d4eeed8938301f3bcf8206405315bc1017
rundir: /run/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  Package:          crun-1.17-1.fc40.x86_64
Engine:
  Version:          5.2.3
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:
  Built:            Tue Sep 24 01:00:00 2024
  OS/Arch:          linux/amd64
  Experimental:     false

the iid file ends up empty. The file is created with 0 bytes.

[rhatdan]

I think we need to examine the difference between the API when talking to the docker daemon versus talking to the podman service.

[paulomorgado]

I've looked a bit more into it, and it looks like an issue with https://github.com/docker/buildx.

I've opened docker/buildx#2820