Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

propolis-server should specify explicit CPUID values for all VMs, even if the spec contained none #835

Open
gjcolombo opened this issue Jan 16, 2025 · 0 comments
Assignees
Labels
server Related specifically to the Propolis server API and its VM management functions.

Comments

@gjcolombo
Copy link
Contributor

Today the cpuid field of an instance spec's Board is an Option:

/// The CPUID values to expose to the guest. If `None`, bhyve will derive
/// default values from the host's CPUID values.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub cpuid: Option<Cpuid>,
// TODO: Processor and NUMA topology.

Specifying None gives the VM some sensible defaults for whatever host it happens to be running on. This is useful for running Propolis VMs on non-Oxide hardware (for ad hoc testing, in CI, as part of an Omicron dev cluster on a commodity machine, etc.), where the user may not have a CPUID profile ready to use in advance.

propolis-server handles a None CPUID value by doing nothing and letting bhyve's CPUID emulation decide what values to supply. However, we're close to needing something a bit more sophisticated:

  • To fix want paravirtualized clock enlightenment #328 (and also to enable x2APIC support), Propolis needs to set appropriate values for a VM's hypervisor leaves (0x4000_0000-FFFF)
  • To set a VM's hypervisor CPUID leaves, Propolis also needs to set its standard/explicit leaves (there's no way to use bhyve-supplied defaults for some leaves but not others)
  • To set those leaves, Propolis needs to be told what values to use, i.e. the cpuid field in the instance spec must be Some

The problem is that it's not obvious what values to use in software that might run on arbitrary hardware, like PHD or an Omicron dev cluster. For ad hoc servers, it's possible to work around this by using the cpuid-gen utility in the repo to extract host CPUID information to put into a hand-crafted instance spec, but this is at least mildly annoying to do.

I propose changing the semantics of cpuid: None in propolis-server from "use bhyve's legacy emulation" to "extract the values bhyve's legacy emulation would provide and feed them back in as explicit CPUID values"--i.e., do the cpuid-gen trick, but do it automatically. This would allow PHD to test enlightenments without committing to any particular set of CPUID values in the framework (and without requiring some other way to specify default CPUID values to PHD out-of-band, which just moves the problem back up to whoever is running PHD). It would also make our CI VMs behave more like Omicron production VMs (in the sense that both environments will use the "explicit CPUID" code in bhyve, even though they'll get their CPUID values in different ways).

This would be very nice to have before doing #328.

This can be implemented today, but before deploying it widely we should try to land illumos CL 3953 to fix some small gaps between bhyve's legacy and explicit CPUID emulation paths.

@gjcolombo gjcolombo added the server Related specifically to the Propolis server API and its VM management functions. label Jan 16, 2025
@gjcolombo gjcolombo self-assigned this Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
server Related specifically to the Propolis server API and its VM management functions.
Projects
None yet
Development

No branches or pull requests

1 participant