lima setup: impish image url not found

[pfiaux]

Describe the bug
Base image for lima vm not found.

To Reproduce
Following the setup guide on macos without docker trying to install lima. When I tied to start the vm with
limactl start ./lima/bass.yaml
I got the following error:

INFO[0012] Attempting to download the image from "https://cloud-images.ubuntu.com/impish/current/impish-server-cloudimg-amd64.img"  digest=
FATA[0012] failed to download the image, attempted 2 candidates, errors=[failed to download "https://cloud-images.ubuntu.com/impish/current/impish-server-cloudimg-amd64.img": expected HTTP status 200, got 404 Not Found unsupported arch: "aarch64"]

Looking at https://cloud-images.ubuntu.com/ impish is nolonger listed on the top level (it is still under releases tho)

Expected behavior
VM starts without error.

Additional context
I updated lima/bass.yaml as follows:

images:
- location: "https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img"
  arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-arm64.img"
  arch: "aarch64"

After that the image starts:

forwarded buildkitd socket to $HOME/Library/Application Support/bass/buildkitd.sock

`bass` will automatically discover it there - start playing!

Took me a while to figure out why I couldn't get bass running after that, until I remembered I had another bass installed (https://github.com/edc/bass), then it was fine 👍

[vito]

Thanks for the report and info!

This is a good reminder for me; some recent changes (#229, #231) now require custom Buildkit config so I need to figure out what to do with the Lima stack. Feel free to PR the change you made here if you find the time though.

Right now I'm just depending on Lima's out-of-the-box Buildkit installation + config. I'll need to point it to my fork of Buildkit, install some CNI plugin binaries, and add Bass's custom config.

In the long run the Lima stack might be kind of a pain to maintain in parallel to the Docker setup. Would you be terribly offended if I removed it and just focused on having Docker as a dependency?

[pfiaux]

Let's just say I wouldn't be thrilled but let's face it there's not that many ways around Docker currently, especially on macOS, so I'd probably get over it until there's a way to get buildkit running on its own.

I'd actually have a separate issue regarding the docker setup, I tried it at work which was behind a proxy, I'll need to run it again so I can share the exact error message.

[vito]

Fair. :)

The trouble I'm running into now is Buildkit isn't a one-size-fits-all dependency. Configuring Buildkit for Bass's upcoming networking capabilities would affect how Buildkit works for anything else that uses it in a way that might be incompatible. Bass also wants Buildkit to allow certain security entitlements that might introduce risk for other things that use it. So having Bass run its own Buildkit is the safest approach, and Docker is the easiest way to do that across all platforms.

But I'm supportive of keeping Docker optional. Bass will always look for a pre-configured Buildkit via one of the following:

• BUILDKIT_HOST=unix:///path or BUILDKIT_HOST=tcp://ip:port
• $XDG_CONFIG_DIR/bass/buildkitd.sock
• $XDG_RUNTIME_DIR/buildkit/buildkitd.sock

But it'll have to trust that the Buildkit is configured appropriately. Maybe I could have a bass --check or something to validate it and show exactly what config needs to be changed. 🤔 (#235)

The other half of it is maintaining toolchains like Lima. I'm supportive of keeping an official config available, but right now I can only promise a best-effort basis (...which is really what it is already). I'll look into what it would take to update it soon, unless someone gets to that before me!

[pfiaux]

The lima setup made sense, run a vm with buildkit and expose it, I assumed the Docker setup was using the builtin buildkit in a similar way.

How does bass run it's own buildkit via docker? Is it another image? If you point me more or less where in the code it happens I'm happy to read the code directly tho I'm not sure where to start looking.
But if it's running buildkit in a container, then the dependency isn't necessarily docker but a container runtime right? so another runtime minikube, nerdctl (which builds on top of lima) or something else would do (technically it could even be remote then) or does it rely on some docker specific features?

Also I'll close this, the impish problem is solved for me 👍

[vito]

Bass runs its own Buildkit daemon using a now-custom image, basslang/buildkit. So yeah, good point, this could be done using any other container runtime; there's no strong dependency on Docker itself. And yeah, a remote Buildkit should also work just fine. It's always been a design goal but I don't test it day-to-day.

The Docker setup happens in pkg/runtimes/util/buildkitd/:

bass/pkg/runtimes/util/buildkitd/buildkitd.go

Lines 190 to 203 in 6aea912

	cmd = exec.CommandContext(ctx,
	"docker",
	"run",
	"--net=host",
	"-d",
	"--restart", "always",
	"-v", volumeName+":/var/lib/buildkit",
	"--name", containerName,
	"--privileged",
	image+":"+Version,
	"dumb-init",
	"buildkitd",
	"--debug",
	)

The basslang/buildkit image is built by bass/buildkit.bass where you can find the custom buildkitd.toml and CNI config:

bass/bass/buildkit.bass

Lines 13 to 21 in 6aea912

	(defn image [os arch config-dir]
	(from (resolve {:platform {:os os} ; TODO: :arch arch}
	:repository *buildkit-repository*
	:tag (str *buildkit-version* *buildkit-variant*)})
	($ apk add --no-cache dumb-init iptables ip6tables dnsmasq) ; TODO: nix?
	($ mkdir -p /opt/cni/bin/ /etc/buildkit/)
	($ cp $dnsname /opt/cni/bin/dnsname)
	($ tar -zxf (cni os arch) -C /opt/cni/bin/)
	($ sh -c "cp $0/* /etc/buildkit/" $config-dir)))