Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build-image.py fails on the main branch when using Fedora #1075

Open
deeplow opened this issue Feb 10, 2025 · 5 comments
Open

build-image.py fails on the main branch when using Fedora #1075

deeplow opened this issue Feb 10, 2025 · 5 comments
Labels
bug Something isn't working P:linux

Comments

@deeplow
Copy link
Contributor

deeplow commented Feb 10, 2025

What happened?

I followed the steps in BUILD.md for a Fedora system and it failed to build the image.

[user@computer dangerzone]$ python3 ./install/common/build-image.py 
Building for architecture 'x86_64'
Will tag the container image as 'dangerzone.rocks/dangerzone:0.8.0-123-g88a6b37'
Building container image
[1/2] STEP 1/22: FROM debian:bookworm-20250113-slim AS dangerzone-image
[1/2] STEP 2/22: ARG GVISOR_ARCHIVE_DATE=20250120
--> Using cache 7ebe23caa3e7595d6e0e36be4665b4e50f1e1e19c13aab99cffcb4fa9e8d5c24
--> 7ebe23caa3e7
[1/2] STEP 3/22: ARG DEBIAN_ARCHIVE_DATE=20250127
--> Using cache 9d6a9a7c94a1ac141cf2f19e3d8506ff5d898eda343f85fe47db7d13725a529d
--> 9d6a9a7c94a1
[1/2] STEP 4/22: ARG H2ORESTART_CHECKSUM=7760dc2963332c50d15eee285933ec4b48d6a1de9e0c0f6082946f93090bd132
--> Using cache 81bf6b95892efa57f8ee79c67630f201cb9bac293b795e0057ffa2eceeaf2f82
--> 81bf6b95892e
[1/2] STEP 5/22: ARG H2ORESTART_VERSION=v0.7.0
--> Using cache 74de2ff3564e18fc3674f4976e7faa486d3bfd7e88a19e40683c5728ec35649d
--> 74de2ff3564e
[1/2] STEP 6/22: ENV DEBIAN_FRONTEND=noninteractive
--> Using cache eb11fc8c4c7be51bd6229b6b6f7009c75b2d379c0f31185ba22cae10095f7154
--> eb11fc8c4c7b
[1/2] STEP 7/22: RUN   --mount=type=cache,target=/var/cache/apt,sharing=locked   --mount=type=cache,target=/var/lib/apt,sharing=locked   --mount=type=bind,source=./container_helpers/repro-sources-list.sh,target=/usr/local/bin/repro-sources-list.sh   --mount=type=bind,source=./container_helpers/gvisor.key,target=/tmp/gvisor.key   : "Hacky way to set a date for the Debian snapshot repos" &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list.d/debian.sources &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list &&   repro-sources-list.sh &&   : "Setup APT to install gVisor from its separate APT repo" &&   apt-get update &&   apt-get upgrade -y &&   apt-get install -y --no-install-recommends apt-transport-https ca-certificates gnupg &&   gpg -o /usr/share/keyrings/gvisor-archive-keyring.gpg --dearmor /tmp/gvisor.key &&   echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases ${GVISOR_ARCHIVE_DATE} main" > /etc/apt/sources.list.d/gvisor.list &&   : "Install the necessary gVisor and Dangerzone dependencies" &&   apt-get update &&   apt-get install -y --no-install-recommends       python3 python3-fitz libreoffice-nogui libreoffice-java-common       python3 python3-magic default-jre-headless fonts-noto-cjk fonts-dejavu       runsc unzip wget &&   : "Clean up for improving reproducibility (optional)" &&   rm -rf /var/cache/fontconfig/ &&   rm -rf /etc/ssl/certs/java/cacerts &&   rm -rf /var/log/* /var/cache/ldconfig/aux-cache
/bin/sh: 1: repro-sources-list.sh: Permission denied
Error: building at STEP "RUN --mount=type=cache,target=/var/cache/apt,sharing=locked --mount=type=cache,target=/var/lib/apt,sharing=locked --mount=type=bind,source=./container_helpers/repro-sources-list.sh,target=/usr/local/bin/repro-sources-list.sh --mount=type=bind,source=./container_helpers/gvisor.key,target=/tmp/gvisor.key : "Hacky way to set a date for the Debian snapshot repos" &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list.d/debian.sources &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list &&   repro-sources-list.sh &&   : "Setup APT to install gVisor from its separate APT repo" &&   apt-get update &&   apt-get upgrade -y &&   apt-get install -y --no-install-recommends apt-transport-https ca-certificates gnupg &&   gpg -o /usr/share/keyrings/gvisor-archive-keyring.gpg --dearmor /tmp/gvisor.key &&   echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases ${GVISOR_ARCHIVE_DATE} main" > /etc/apt/sources.list.d/gvisor.list &&   : "Install the necessary gVisor and Dangerzone dependencies" &&   apt-get update &&   apt-get install -y --no-install-recommends       python3 python3-fitz libreoffice-nogui libreoffice-java-common       python3 python3-magic default-jre-headless fonts-noto-cjk fonts-dejavu       runsc unzip wget &&   : "Clean up for improving reproducibility (optional)" &&   rm -rf /var/cache/fontconfig/ &&   rm -rf /etc/ssl/certs/java/cacerts &&   rm -rf /var/log/* /var/cache/ldconfig/aux-cache": while running runtime: exit status 127
Traceback (most recent call last):
  File "/home/user/dangerzone/./install/common/build-image.py", line 145, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/dangerzone/./install/common/build-image.py", line 103, in main
    subprocess.run(
  File "/usr/lib64/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['podman', 'build', 'dangerzone/', '-f', 'Dockerfile', '--tag', 'dangerzone.rocks/dangerzone:0.8.0-123-g88a6b37']' returned non-zero exit status 127.

Linux distribution

Fedora 40

Dangerzone version

main@88a6b377

Podman info

[user@dz-computer dangerzone]$ podman version
podman info -f 'json'
podman images
podman run hello-world
Client:       Podman Engine
Version:      5.3.1
API Version:  5.3.1
Go Version:   go1.22.7
Built:        Thu Nov 21 00:00:00 2024
OS/Arch:      linux/amd64
{
  "host": {
    "arch": "amd64",
    "buildahVersion": "1.38.0",
    "cgroupManager": "systemd",
    "cgroupVersion": "v2",
    "cgroupControllers": [
      "memory",
      "pids"
    ],
    "conmon": {
      "package": "conmon-2.1.12-2.fc40.x86_64",
      "path": "/usr/bin/conmon",
      "version": "conmon version 2.1.12, commit: "
    },
    "cpus": 2,
    "cpuUtilization": {
      "userPercent": 3.22,
      "systemPercent": 1.28,
      "idlePercent": 95.5
    },
    "databaseBackend": "sqlite",
    "distribution": {
      "distribution": "fedora",
      "version": "40"
    },
    "eventLogger": "journald",
    "freeLocks": 2048,
    "hostname": "computer",
    "idMappings": {
      "gidmap": [
        {
          "container_id": 0,
          "host_id": 1000,
          "size": 1
        },
        {
          "container_id": 1,
          "host_id": 524288,
          "size": 65536
        }
      ],
      "uidmap": [
        {
          "container_id": 0,
          "host_id": 1000,
          "size": 1
        },
        {
          "container_id": 1,
          "host_id": 524288,
          "size": 65536
        }
      ]
    },
    "kernel": "6.6.68-1.qubes.fc37.x86_64",
    "logDriver": "journald",
    "memFree": 21479424,
    "memTotal": 1246568448,
    "networkBackend": "netavark",
    "networkBackendInfo": {
      "backend": "netavark",
      "version": "netavark 1.13.1",
      "package": "netavark-1.13.1-1.fc40.x86_64",
      "path": "/usr/libexec/podman/netavark",
      "dns": {
        "version": "aardvark-dns 1.13.1",
        "package": "aardvark-dns-1.13.1-1.fc40.x86_64",
        "path": "/usr/libexec/podman/aardvark-dns"
      }
    },
    "ociRuntime": {
      "name": "crun",
      "package": "crun-1.19.1-1.fc40.x86_64",
      "path": "/usr/bin/crun",
      "version": "crun version 1.19.1\ncommit: 3e32a70c93f5aa5fea69b50256cca7fd4aa23c80\nrundir: /run/user/1000/crun\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL"
    },
    "os": "linux",
    "remoteSocket": {
      "path": "/run/user/1000/podman/podman.sock",
      "exists": true
    },
    "rootlessNetworkCmd": "pasta",
    "serviceIsRemote": false,
    "security": {
      "apparmorEnabled": false,
      "capabilities": "CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT",
      "rootless": true,
      "seccompEnabled": true,
      "seccompProfilePath": "/usr/share/containers/seccomp.json",
      "selinuxEnabled": true
    },
    "slirp4netns": {
      "executable": "",
      "package": "",
      "version": ""
    },
    "pasta": {
      "executable": "/usr/bin/pasta",
      "package": "passt-0^20241211.g09478d5-1.fc40.x86_64",
      "version": "pasta 0^20241211.g09478d5-1.fc40.x86_64\nCopyright Red Hat\nGNU General Public License, version 2 or later\n  \u003chttps://www.gnu.org/licenses/old-licenses/gpl-2.0.html\u003e\nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to the extent permitted by law.\n"
    },
    "swapFree": 851701760,
    "swapTotal": 1073737728,
    "uptime": "0h 21m 25.00s",
    "variant": "",
    "linkmode": "dynamic"
  },
  "store": {
    "configFile": "/home/user/.config/containers/storage.conf",
    "containerStore": {
      "number": 0,
      "paused": 0,
      "running": 0,
      "stopped": 0
    },
    "graphDriverName": "overlay",
    "graphOptions": {
      
    },
    "graphRoot": "/home/user/.local/share/containers/storage",
    "graphRootAllocated": 2040373248,
    "graphRootUsed": 1673891840,
    "graphStatus": {
      "Backing Filesystem": "extfs",
      "Native Overlay Diff": "true",
      "Supports d_type": "true",
      "Supports shifting": "false",
      "Supports volatile": "true",
      "Using metacopy": "false"
    },
    "imageCopyTmpDir": "/var/tmp",
    "imageStore": {
      "number": 6
    },
    "runRoot": "/run/user/1000/containers",
    "volumePath": "/home/user/.local/share/containers/storage/volumes",
    "transientStore": false
  },
  "registries": {
    "search": [
  "registry.fedoraproject.org",
  "registry.access.redhat.com",
  "docker.io"
]
  },
  "plugins": {
    "volume": [
      "local"
    ],
    "network": [
      "bridge",
      "macvlan",
      "ipvlan"
    ],
    "log": [
      "k8s-file",
      "none",
      "passthrough",
      "journald"
    ],
    "authorization": null
  },
  "version": {
    "APIVersion": "5.3.1",
    "Version": "5.3.1",
    "GoVersion": "go1.22.7",
    "GitCommit": "",
    "BuiltTime": "Thu Nov 21 00:00:00 2024",
    "Built": 1732147200,
    "OsArch": "linux/amd64",
    "Os": "linux"
  }
}
REPOSITORY                TAG                     IMAGE ID      CREATED         SIZE
<none>                    <none>                  eb11fc8c4c7b  17 minutes ago  77.8 MB
docker.io/library/debian  bookworm-20250113-slim  fa7572809b79  4 weeks ago     77.8 MB
Resolved "hello-world" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull quay.io/podman/hello:latest...
Getting image source signatures
Copying blob 81df7ff16254 done   | 
Copying config 5dd467fce5 done   | 
Writing manifest to image destination
!... Hello Podman World ...!

         .--"--.           
       / -     - \         
      / (O)   (O) \        
   ~~~| -=(,Y,)=- |         
    .---. /`  \   |~~      
 ~/  o  o \~~~~.----. ~~   
  | =(X)= |~  / (O (O) \   
   ~~~~~~~  ~| =(Y_)=-  |   
  ~~~~    ~~~|   U      |~~ 

Project:   https://github.com/containers/podman
Website:   https://podman.io
Desktop:   https://podman-desktop.io
Documents: https://docs.podman.io
YouTube:   https://youtube.com/@Podman
X/Twitter: @Podman_io
Mastodon:  @[email protected]

Document conversion logs

(not relevant)

Additional info

No response

@deeplow deeplow added the bug Something isn't working label Feb 10, 2025
@apyrgio
Copy link
Contributor

apyrgio commented Feb 10, 2025

Hey, thanks for filling this report!

Interesting, we haven't encountered this issue before, I believe. Looks like SELinux could be involved, if you are in a pure Fedora 40 distro. Is this the case, or are you building this in a qube?

In any case, can you list the permissions of the file in your system, and its security context?

ls -lZ dangerzone/container_helpers/repro-sources-list.sh

@almet almet changed the title build-image.py fails on main due to build-image.py fails on the main branch when using Fedora Feb 10, 2025
@almet almet added the P:linux label Feb 10, 2025
@deeplow
Copy link
Contributor Author

deeplow commented Feb 10, 2025

This is Fedora 40 on Qubes.

In any case, can you list the permissions of the file in your system, and its security context?

ls -lZ dangerzone/container_helpers/repro-sources-list.sh

Here you go:

$ ls -lZ dangerzone/container_helpers/repro-sources-list.sh
-rwxr-xr-x. 1 user user unconfined_u:object_r:user_home_t:s0 5442 Feb 10 11:20 dangerzone/container_helpers/repro-sources-list.sh

@almet
Copy link
Member

almet commented Feb 10, 2025

Hey @deeplow and thanks for opening this. I was surprised by the fact that we don't have CI to test the build instructions on different platforms (I though we had but nope), and so I opened an issue for it.

Also, reporting the errors I see on my machine when trying to build an image on fedora with our dev_scripts/env.py helper (errors are different):

[user@dangerzone-dev dangerzone]$ python3 ./install/common/build-image.py
Building for architecture 'x86_64'
Will tag the container image as 'dangerzone.rocks/dangerzone:0.8.0-159-gceab2c7-3a91'
Building container image
[1/2] STEP 1/22: FROM debian:bookworm-20250113-slim AS dangerzone-image
[1/2] STEP 2/22: ARG GVISOR_ARCHIVE_DATE=20250120
--> Using cache 96813686231748f1482f617d4be68b63e7c43f638e07b530b2fa63569890e0e2
--> 968136862317
[1/2] STEP 3/22: ARG DEBIAN_ARCHIVE_DATE=20250127
--> Using cache 100108a439b8812ae38bda35e9cb19994ad4ae8a597b6cb984263f3c6fdd7752
--> 100108a439b8
[1/2] STEP 4/22: ARG H2ORESTART_CHECKSUM=7760dc2963332c50d15eee285933ec4b48d6a1de9e0c0f6082946f93090bd132
--> Using cache 88b6a0c627d82a47b6633147866e61d9336eb55c5b52fcf9bd6470cb72f3c84e
--> 88b6a0c627d8
[1/2] STEP 5/22: ARG H2ORESTART_VERSION=v0.7.0
--> Using cache 57e2ccb69910292c3bbe3473c3b77859538fb4b55a4c8e2daad5c6c882c7c31c
--> 57e2ccb69910
[1/2] STEP 6/22: ENV DEBIAN_FRONTEND=noninteractive
--> Using cache 04f3bdbf56c7d586cb3ed82bca30377703bf01a772cb5ed88f289ba1511f0283
--> 04f3bdbf56c7
[1/2] STEP 7/22: RUN   --mount=type=cache,target=/var/cache/apt,sharing=locked   --mount=type=cache,target=/var/lib/apt,sharing=locked   --mount=type=bind,source=./container_helpers/repro-sources-list.sh,target=/usr/local/bin/repro-sources-list.sh   --mount=type=bind,source=./container_helpers/gvisor.key,target=/tmp/gvisor.key   : "Hacky way to set a date for the Debian snapshot repos" &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list.d/debian.sources &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list &&   repro-sources-list.sh &&   : "Setup APT to install gVisor from its separate APT repo" &&   apt-get update &&   apt-get upgrade -y &&   apt-get install -y --no-install-recommends apt-transport-https ca-certificates gnupg &&   gpg -o /usr/share/keyrings/gvisor-archive-keyring.gpg --dearmor /tmp/gvisor.key &&   echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases ${GVISOR_ARCHIVE_DATE} main" > /etc/apt/sources.list.d/gvisor.list &&   : "Install the necessary gVisor and Dangerzone dependencies" &&   apt-get update &&   apt-get install -y --no-install-recommends       python3 python3-fitz libreoffice-nogui libreoffice-java-common       python3 python3-magic default-jre-headless fonts-noto-cjk fonts-dejavu       runsc unzip wget &&   : "Clean up for improving reproducibility (optional)" &&   rm -rf /var/cache/fontconfig/ &&   rm -rf /etc/ssl/certs/java/cacerts &&   rm -rf /var/log/* /var/cache/ldconfig/aux-cache
+ . /etc/os-release
++ PRETTY_NAME='Debian GNU/Linux 12 (bookworm)'
++ NAME='Debian GNU/Linux'
++ VERSION_ID=12
++ VERSION='12 (bookworm)'
++ VERSION_CODENAME=bookworm
++ ID=debian
++ HOME_URL=https://www.debian.org/
++ SUPPORT_URL=https://www.debian.org/support
++ BUG_REPORT_URL=https://bugs.debian.org/
+ : 1
+ case "${ID}" in
+ : http://snapshot.debian.org/archive/
+ : ''
+ '[' -e /etc/apt/sources.list.d/debian.sources ']'
++ stat --format=%Y /etc/apt/sources.list.d/debian.sources
+ : 1737936000
+ rm -f /etc/apt/sources.list.d/debian.sources
++ printf '%(%Y%m%dT%H%M%SZ)T\n' 1737936000
+ snapshot=20250127T000000Z
+ echo 'deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/20250127T000000Z bookworm main'
+ echo 'deb [check-valid-until=no] http://snapshot.debian.org/archive/debian-security/20250127T000000Z bookworm-security main'
+ echo 'deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/20250127T000000Z bookworm-updates main'
+ '[' '' = 1 ']'
+ '[' 1 = 1 ']'
+ keep_apt_cache
+ rm -f /etc/apt/apt.conf.d/docker-clean
+ echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";'
+ : /dev/null
+ echo 1737936000
+ echo SOURCE_DATE_EPOCH=1737936000
SOURCE_DATE_EPOCH=1737936000
E: setgroups 65534 failed - setgroups (22: Invalid argument)
E: setegid 65534 failed - setegid (22: Invalid argument)
Reading package lists...
E: setgroups 65534 failed - setgroups (22: Invalid argument)
E: setegid 65534 failed - setegid (22: Invalid argument)
E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (22: Invalid argument)
E: Method http has died unexpectedly!
E: Sub-process http returned an error code (112)
Error: building at STEP "RUN --mount=type=cache,target=/var/cache/apt,sharing=locked --mount=type=cache,target=/var/lib/apt,sharing=locked --mount=type=bind,source=./container_helpers/repro-sources-list.sh,target=/usr/local/bin/repro-sources-list.sh --mount=type=bind,source=./container_helpers/gvisor.key,target=/tmp/gvisor.key : "Hacky way to set a date for the Debian snapshot repos" &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list.d/debian.sources &&   touch -d ${DEBIAN_ARCHIVE_DATE} /etc/apt/sources.list &&   repro-sources-list.sh &&   : "Setup APT to install gVisor from its separate APT repo" &&   apt-get update &&   apt-get upgrade -y &&   apt-get install -y --no-install-recommends apt-transport-https ca-certificates gnupg &&   gpg -o /usr/share/keyrings/gvisor-archive-keyring.gpg --dearmor /tmp/gvisor.key &&   echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases ${GVISOR_ARCHIVE_DATE} main" > /etc/apt/sources.list.d/gvisor.list &&   : "Install the necessary gVisor and Dangerzone dependencies"&&   apt-get update &&   apt-get install -y --no-install-recommends       python3 python3-fitz libreoffice-nogui libreoffice-java-common   python3 python3-magic default-jre-headless fonts-noto-cjk fonts-dejavu       runsc unzip wget &&   : "Clean up for improving reproducibility (optional)" &&   rm -rf /var/cache/fontconfig/ &&   rm -rf /etc/ssl/certs/java/cacerts &&   rm -rf /var/log/* /var/cache/ldconfig/aux-cache": while running runtime: exit status 100
Traceback (most recent call last):
  File "/home/user/dangerzone/./install/common/build-image.py", line 145, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/dangerzone/./install/common/build-image.py", line 103, in main
    subprocess.run(
  File "/usr/lib64/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['podman', 'build', 'dangerzone/', '-f', 'Dockerfile', '--tag', 'dangerzone.rocks/dangerzone:0.8.0

@apyrgio
Copy link
Contributor

apyrgio commented Feb 10, 2025

What tha! Thanks for the extra logs Alexis, I hadn't realized our CI was no longer building the container image for our supported distros.

This is very interesting then, because it ties in nicely with #1074. If we want to make sure that our image is reproducible, we must ensure that we can reproduce it across different runtimes, and across different OSes as well. Else, we may have to restrict building this image to a specific set of OSes / runtimes (which I'd prefer not doing yet). And not only that, we must ensure that our CI can do that, so that there are no regressions.

For this particular case, I think copying the script in the container image, instead of mounting it would work. I'll try to work on it, but in the broader context of #1074, so it may take a while. @deeplow, if you're in a hurry, I can prioritize the Fedora part more, let me know :-)

@deeplow
Copy link
Contributor Author

deeplow commented Feb 10, 2025

if you're in a hurry, I can prioritize the Fedora part more, let me know :-)

Thanks! Not in a hurry, I just wanted to give DZ a spin. And I think I can do so from a release tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P:linux
Projects
Status: Todo
Development

No branches or pull requests

3 participants