refactor: autogenerate metrics docs

[Joibel]

Motivation

As mentioned at some point during the opentelemetry metrics changes, it would be nice if documentation could be autogenerated.

This PR does this so that there is a single source of truth for metrics and attributes.

This makes attribute documentation consistent, and the docs match the implementation in more places.

It will help when we do something about server metrics, and also with tracing (as tracing will want managed attributes and auto-generated docs).

Modifications

Create a new yaml file util/telemetry/builder/values.yaml which contains the attributes and metrics.

util/telemetry/builder/builder.go converts this into go structs and documentation which is inserted into the metrics.md docs. The go structs implement a new interface for builtin metrics.

This code change has the downside of making the shared utils directory aware of every metric again.

This fixes a few things:

• The HELP for prometheus metrics will now exactly match the first line in the documentation.
• The documentation for metrics are guaranteed to be in alphabetical order.
• workers_busy_count has the correct attribute name (this is a legacy metric - the implementation matched 3.5 which is correct, but the docs did not match the implementation of having a different attribute name from the other queue metrics).

This can be further improved by enforcing the attributes which may mismatch at the moment, and removing all of the calls to Name() by instead allowing the operators to operate on the builtin metrics. This is left for a separate PR. This will also optimise the calls to the operators by pre-creating the attributes.

Verification

Unit and integration tests.

Some inspection of prometheus metrics from before and after.

[tico24]

nits commented.

I'm slightly surprised that the new markdown tables are passing CI. I thought there was a table linter? They aren't nice and pretty like they were before.

[tico24]

Suggested change

	# Built telemetry files
	# Build telemetry files

[Joibel]

Fixed

[tico24]

... was run?

[Joibel]

Chosen a past and present variant

[tico24]

was run?

[Joibel]

Chosen a past and present variant

[tico24]

we doing uppercase P for pods or not?

[Joibel]

No, no upper case Ps for you

[Joibel]

I'm slightly surprised that the new markdown tables are passing CI. I thought there was a table linter? They aren't nice and pretty like they were before.

They aren't. The linter doesn't mind it, and some have been modified to look like this elsewhere by hand.

To make them pretty would need a prettifier, and this way is also easier for hand editing a bit like one sentence per line is.

Not remedying.

[isubasinghe]

This is fine, I think I'd personally prefer if we used a markdown AST like https://github.com/yuin/goldmark.

This makes it possible to render the markdown nicely, it also makes parsing and doing inline modifications much easier.

No strong preferences though, this is not critical code I guess, as long as its generating valid markdown, its fine.

[Joibel]

This is fine, I think I'd personally prefer if we used a markdown AST like https://github.com/yuin/goldmark.

This makes it possible to render the markdown nicely, it also makes parsing and doing inline modifications much easier.

Goldmark doesn't do nice rendering, it has an emphasis on reading markdown. I've implemented nao1215/markdown which brings in more dependencies, but does also make the tables prettier so that @tico24 can pretend he's happy too.

[terrytangyuan]

This is great. Thanks!

[isubasinghe]

Thanks for the changes.