Freebsd 14.1 out with suspend resume

[syleishere]

Support needs to be added for bhyvectl --suspend and bhyve -r

[yaroslav-gwit]

Can't see anything in the release notes. Could you please link to the info?

EDIT:
Plus it looks like you have to use the checkpoint instead of suspend:

       --checkpoint=<filename>
		    Save a snapshot of a virtual machine.   The	 guest	memory
		    contents  are  saved in the	file given in <filename>.  The
		    guest  device  and	vCPU  state  are  saved	 in  the  file
		    <filename>.kern.

       --suspend=<filename>
		    Save   a   snapshot	  of  a	 virtual  machine  similar  to
		    --checkpoint.  The virtual machine	will  terminate	 after
		    the	snapshot has been saved.

[syleishere]

I saw the options in the man page in FreeBSD 14.1, I assumed they were working without patching kernel with BHYVE_SNAPSHOT. Trying from command line, options are not recognized, so I guess it has not officially been added yet.

For anyone that wants it working procedure is as follows:

cd /usr/src/sys/amd64/conf
cp GENERIC MYKERNEL
(edit MYKERNEL)  (add: options         BHYVE_SNAPSHOT)
cd /usr/src
make -j$(sysctl -n hw.ncpu) buildworld -DWITH_BHYVE_SNAPSHOT
make -j$(sysctl -n hw.ncpu) buildkernel KERNCONF=MYKERNEL
make installkernel KERNCONF=MYKERNEL
shutdown -r now
etcupdate -p
cd /usr/src; make installworld
shutdown -r now
etcupdate -B
shutdown -r now

[yaroslav-gwit]

Just checked the FreeBSD VMs, and the feature is still hidden behind a build flag on the 14-STABLE and 15-CURRENT. Looks like we're not gonna get it any time soon (at least by default that is).

[syleishere]

yaroslav, checked your github repo, haven't tried it but I noticed it says on your hoster page OpenBSD etc not supported. I have had no issues at all running OpenBSD, NetBSD, home assistant etc. Here is link to blog to help you make your project better :)
https://blog.sunsaturn.com/freebsd/create-virtual-machines-with-freebsd-vm-bhyve/

Good job on doing all that in GO :) Personally I would have done it on flet framework with a fastapi server, that way it would support all OS's including google tv :) Be nice to see interface on the TV, its 2024 :) Plus if you like working with python, when mojo hits full release status be easy transition to a language faster than C and rust.

[yaroslav-gwit]

I know OpenBSD works under Bhyve, but it threw some bootloader errors when I first tried it in the UEFI mode. Unfortunately, fixing it is not a priority right now (and I personally haven't had a use-case for it, yet).

vm-bhyve has been a huge inspiration for me, that's why I stick around here and help other users, whenever I can 😄

1st iteration of Hoster was written in Python, but it wasn't cutting it:

• Slow runtime speed.
• Where do you put your venv dependencies? On the system? What if the end user updates them? In a separate folder? What if the user decided to switch to a newer Python version by default? I had this happen to my "test subjects" all the time.
• Async/background job execution is very painful.
• Python project is getting harder to develop as the codebase grows (any dynamic/typeless lang does this).

So I've learned Go (and JS, eventually, for the front) and never looked back. The project is far from being ready for the prime time, but it's getting there (at least I've got few paying customers, who additionally sponsor my small dev lab). So in the end, I hope to create a virtualisation WebUI (+ backend + CLI, but hey, who is counting?) for the FreeBSD community. The one they want and like (hopefully? lol) 😅

(written using Go+VueJS):

[antranigv]

@yaroslav-gwit that's pretty impressive! I mean I won't use it personally, but I can see this deployed for customers who want a GUI for management.

Just to be clear, are you automating bhyve, or vm-bhyve under the hood?

[yaroslav-gwit]

I used to automate the vm-bhyve in the beginning, but it became a complete rewrite in the end (because I had to write my own VM process daemon, etc etc). It also has a REST API interface and a "modern feel" CLI, with the autocomplete, pretty tables and whatnot (checkout the repo, it's all there).

[syleishere]

Can you still attach to vm's with tmux when they are started? Think that's one of biggest benefits of vm-bhyve along with command line interface.

GUI is very impressive, not sure why you abandoned python, you could have always started a port and put all your dependencies there.

Personally I would have done it in flet, doing flutter in python has many benefits, although no freebsd support in flutter means it would have had to been a web interface with something like fastapi on FreeBSD. I figure if done this way, when mojo becomes main stream in a few years, be easy switch to get same performance as rust. Benefits are would be easy to incorporate as an app afterwards as well as controlling your VMs with your remote on Google tv etc.

Beautiful GUI though, very impressive, good job.

[yaroslav-gwit]

This command will give you the serial console:

hoster vm serial-console yourVmName

It uses tmux in the CLI mode (creates a new session if there isn't one already for this particular VM, or re-uses the old one if it can find it), but I haven't added the serial console capabilities to the GUI yet (again, the GUI is still a PoC at this point, so only a handful of functions work there).

Tried Flutter (I was tempted by their promise of "one codebase for all your UIs/platforms"), but it had many shortcomings that I won't go into here. Long story short - it didn't fit the usecase.

With Go, I now have multiple background daemon helpers (snapshot/replication scheduler, built-in DNS server so you don't have to deal with VM IPs - just use the VM name directly, VM process supervisor, HA/VM failover daemon, etc) that are simply much harder to code in Python (you'd have to use threads, which is a nightmare on it's own). Plus we get the performance benefits now, not if/when. And you can still compile Go/JS on most platforms, if one day someone would want to use their RPi4 with L**ux ( 🤢 ) on it to host the UI.

The desktop GUI would be relatively easy to port as a simplified app (no auth, no host groups/clusters, etc) within the Electron wrapper, once the WebUI is ready ofc.

[syleishere]

If you could recode the GUI in flet, to call your go backend API, I could help you port it to android, windows, web, Linux, FreeBSD, macOS and google TV, that would be nice and scalable with many clients.

I do this with flet currently just using a fastapi backend to call. If each freebsd host had your GO daemon running as an API, we could interact with it with any client from any OS/device this way.

Just a thought anyways, if you coded GO daemon to listen on a port, accept commands and return JSON responses, could separate the server and client logic, be easier to maintain.

All OS's would have the client native, since FreeBSD doesn't support flutter natively, I have gotten around this by running a fastapi web interface without issues.

Flet is really easy, doing flutter in Python, don't even have to worry about things like state management in flutter, could just pass your class config object to every page, worth looking into.

For most part it is easy, when you'd start getting to point of making it a 2024 app adding TV support, can show you tricks to add focus support so remote control can move between elements, I think you have great design skills, would love to see everyone use it on every device they use :)

And yes I understand your issues with python threads, problem is the GIL that won't get unlocked till next version and only in development. Mostly flet is using asyncio, which is perfect for GUI. When I've needed to run a long process in another thread I've had to start a subprocess and link that into asyncio, so your go version be better for that.