docker build using colima with lima v0.23.1 hangs

[ns-dfarino]

Description

This command hangs indefinitely using lima v0.23.1:

docker buildx build -f - . <<'EOF'
FROM ubuntu:latest
RUN apt-get update
RUN apt-get install -y gnupg
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys ABF5BD827BD9BF62
EOF

This is not a problem when using the previous version of lima (v0.22.0).

Colima was started with:

colima start --vm-type=vz --cpu 12 --memory 16 --runtime=docker --arch aamd64 --vz-rosetta

System information:

$ colima --version
colima version 0.7.3

$ lima --version
limactl version 0.23.1

$ system_profiler SPSoftwareDataType SPHardwareDataType
Software:

    System Software Overview:

      System Version: macOS 14.6.1 (23G93)
      Kernel Version: Darwin 23.6.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: REDACTED
      User Name: REDACTED
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 10 days, 15 hours, 18 minutes

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: Mac15,7
      Model Number: Z1AJ0018XLL/A
      Chip: Apple M3 Pro
      Total Number of Cores: 12 (6 performance and 6 efficiency)
      Memory: 36 GB
      System Firmware Version: 10151.140.19
      OS Loader Version: 10151.140.19
      Serial Number (system): REDACTED
      Hardware UUID: REDACTED
      Provisioning UDID: REDACTED
      Activation Lock Status: Disabled

[AkihiroSuda]

Which command hangs?

[afbjorklund]

Sometimes hkp: is blocked by a firewall, then one can maybe download pgp keys with https: instead.

[ns-dfarino]

Thanks for taking a look. The command that hangs is that docker buildx build command. This is a minimal repro example from a larger project.

That same command works fine when I downgrade lima back to v0.22.0, so its not firewall related. It's something that changed when upgrading to lima v0.23.1.

[jlagerweij]

Came here to search for just an issue like that.

I have the same problem in a larger project using the container ghcr.io/navikt/mock-oauth2-server. For me a downgrade to lima v0.22.0 also was the solution.

I have tried the docker command from this issue on my MacBook with an Intel processor and saw it hang too. Using the https: command variant doesn't change. It still hangs.

[mscrivo]

Might be related or not, really not sure, but sounds related, and certainly did not happen with 0.22.0. We've been seeing random hangs from running containers with 0.23.1. What happens is requests to http services running in containers just hang indefinitely. If I wait a min or so and try again, often it will start working properly. Downgrading lima to 0.22.0 makes the problem go away entirely.

Started the colima VM with: colima start --vm-type=vz --cpu 4 --memory 16 --disk 128 -V /private/tmp:w -V "$HOME":w -V /var/folders:w

This is on an M1 Pro macbook. Happy to provide any other debugging info that might be helpful.

For reference, used this to downgrade since brew doesn't make it easy:

colima stop
wget https://github.com/lima-vm/lima/releases/download/v0.22.0/lima-0.22.0-Darwin-arm64.tar.gz -P ~/Downloads
tar xfvz ~/Downloads/lima-0.22.0-Darwin-arm64.tar.gz -C /opt/homebrew/Cellar/lima/0.23.1
colima start

[ns-dfarino]

On my system, a git bisect points to 43a5f4a

[balajiv113]

@ns-dfarino Can you try with latest lima head it has gvisor-tap-vsock 0.7.5 ??

git bisect points to 43a5f4a

Gvisor-tap-vsock has a regression with 0.7.4. This is fixed in 0.7.5. containers/gvisor-tap-vsock#386

[ns-dfarino]

@balajiv113 very good! 4592e34 works correctly.

[jlagerweij]

I also checked to see if this fixes my problem and it does. Great stuff.

[AkihiroSuda]

Released https://github.com/lima-vm/lima/releases/tag/v0.23.2