Prefix-aware Units

[AvlWx2014]

Hi all, I'm new here so forgive me if this has been discussed. I've just started evaluating Pint to replace the half-hearted "units" API in one of my applications. I say "units" in quotes because each unit is really just a constant scale factor so that expressions like:

memory = 1 * GiB

look like quantities with units, but evaluate to a plain integer.

This is fine everywhere except formatting and parsing. Formatting is naive and amounts to a lot of match/case branches to format things correctly, while parsing is broken if the units in your quantity string are not one of the supported constants or if the case doesn't match like gb instead of GB.

The only upside of doing something bespoke is that I have fine-grained control over formatting, which is nice since this application interacts with the Kubernetes API where quantity strings are... strange. Kubernetes only really accepts unitless quantity strings like 1Gi (1 Gibibyte), 1073741824 (also 1 Gibibyte), 192m (192 milliCPU), or 1 (1 CPU or 1000milliCPU). Notice also the lack of any spaces between the magnitude and units portions of the strings - this is also strictly enforced by the Kubernetes API's quantity parser which uses the regular expression:

'^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$'

So far I've attempted my own extensions to pint to cover Kubernetes-specific uses looking something like this:

from typing import Final, Unpack

from pint import UnitRegistry
from pint.delegates.formatter._compound_unit_helpers import SortFunc, BabelKwds
from pint.delegates.formatter.plain import CompactFormatter
from pint.delegates.formatter.full import FullFormatter, REGISTERED_FORMATTERS
from pint.facets import MagnitudeT
from pint.facets.plain import PlainQuantity

u: Final[UnitRegistry] = UnitRegistry()

u.formatter = FullFormatter()
# define new base unit CPU
u.define("CPU = [] = _")
# define alias Byte of byte so Bytes can be used to avoid confusion with the built-in bytes type
u.define("@alias byte = Byte")

class KubernetesFormatter(CompactFormatter):
    def format_quantity(
        self,
        quantity: PlainQuantity[MagnitudeT],
        qspec: str = "",
        sort_func: SortFunc | None = None,
        **babel_kwds: Unpack[BabelKwds],
    ) -> str:
        reduced = quantity.to_root_units()
        reduced_units = reduced.units

        magnitude = self.format_magnitude(quantity.magnitude, mspec="", **babel_kwds)
        units = self.format_unit(quantity.units, uspec="~", sort_func=sort_func, **babel_kwds)

        unitless = units.removesuffix(str(reduced_units))
        return f"{magnitude}{unitless}"

u.formatter._formatters["K8s"] = KubernetesFormatter(registry=u)
REGISTERED_FORMATTERS["K8s"] = KubernetesFormatter()

which as you can see contains a few hacks that rely on internal API, and sadly doesn't quite work. While it does work for quantities like:

print(f"{192 * u.mCPU: K8s}")
# 192m
print(f"{1000 * u.mCPU: K8s}")
# 1000m

since CPU is defined to be a root unit, it fails for memory quantities:

print(f"{256 * u.GiB:K8s}")
# 256GiB

since the root unit for GiB is actually bits not bytes making the removesuffix call a no-op. I tried also using to_reduced_units instead, but this has the same issue with memory quantities because e.g., 256 * u.GiB reduces to a dimensionless quantity which again makes the removesuffix call a no-op.

The idea I wanted to discuss here is making the Unit type "prefix-aware" such that given an object of type Unit I can get Unit.prefix(es) and Unit.base separately. Any prefixes would represent the scale factors for quantities and measurements, while the base units would represent, well... the base units. In this way prefixes could be thought of as modifying the base units they're attached to instead of the combination of prefix and base units being thought of as a single distinct unit which I think is the paradigm currently.

For something like mCPU this would resolve to m (milli) for the prefix and CPU for the base unit; for memory something like GiB would resolve to Gi and B for the prefix and base units respectively; and so on.

If necessary to disambiguate prefixes that might collide with a base unit like m for milli and m for meters or c for centi and c for the speed of light a prefix string could be represented with the trailing - like in default_en.txt e.g., m- for milli. A convenience method could easily be provided to strip the trailing - for clients during formatting if needed.

With prefix-aware units one could easily separate base units and their prefixes for formatting purposes like:

magnitude = self.format_magnitude(quantity.magnitude, mspec="", **babel_kwds)
units = self.format_unit(quantity.units, uspec="~", sort_func=sort_func, **babel_kwds)
unitless = units.removesuffix(str(quantity.units.base))

Looking forward to everyone's thoughts!

[andrewgsavage]

you can make a custom unit file that has memory as a base unit. I think redefining base units doesn't work and needs fixing. .to_preferred(['bytes', 'CPU']) should also work.

yes, accessing the prefix for a unit feels like something that is missing from pint and would be a welcome addition . I'm not sure what such a function would return,. A PR is welcomed!