Include humility in the switch zone?

[jgallagher]

For similar reasoning to #6153, it would smooth over some support operator wrinkles to ship humility in the switch zone. This is a little trickier than faux-mgs from a versioning point of view; we might very well want a humility that's newer than whatever the latest was when a release was cut, but I think most of the time "humility as of the release" would probably be fine?

This probably depends on oxidecomputer/humility#421.

[jclulow]

For both this and for the related #6153 it seems like we could solve them by producing an artefact that can be easily loaded in via SSH rather than by shipping them? Then we could also easily load an updated version from outside as part of a support activity.

[jgallagher]

I think that's the state we're in today, right? faux-mgs and humility binaries are readily available, but schlepping them around via SSH is annoying.

[jgallagher]

FWIW I'm a little less solid on including humility; shipping the version of faux-mgs that matches the version of MGS in the switch zone seems perfectly reasonable. There's an outside shot we could fix a bug in faux-mgs specifically that warranted copying in a newer one as a support operation, but "faux-mgs packaged with the OS matches MGS packaged with the OS" should be fine most of the time, I think?

[leftwo]

Given that someday we will have sleds with different software versions on them, there is always a chance for a mismatch when trying to use a tool on one sled to talk to another. However, having the version that matches the OS is probably as best we can do.

There could always be bugs that would require an updated/different version of a binary.

Packaging and SSH'ing a binary in is an option, while we can still SSH. This also requires you to find the matching version
of a binary for a specific rack (if it does need a specific version)

I do expect that there could be a problem we have to debug after we have turned off SSH that requires a new
binary, so we will need some solution to "getting a binary into a rack", but I don't know what that is yet :)