Expose metrics on the resources that are being managed by the operator

[robert-mcnamara]

Is your feature request related to a problem? Please describe.
We are considering using the Artemis operator to manage multiple Activemq clusters with in a Kubernetes cluster.
We have been evaluation how observability would work for a deployment like this.

We feel it would be helpful to have custom metrics available to show the number of resources (brokers, addresses etc.) that is managed by the operator, to allow this to be visualised at a glance via Prometheus/Grafana

Describe the solution you'd like
Custom metrics partitioned be kind for resources that are being managed by the operator.

Additional context
Strimzi which is an operator for Kafka provides similar functionality
See screenshot of the Strimzi dashboard for reference

[brusdev]

@robert-mcnamara does this enhancement depend on #694?

[robert-mcnamara]

@brusdev - no not dependant in my opinion, the operator already has a running metrics http-server so registering custom metrics should be possible.

#694 would make testing and being able to visualise the output easier, but would not block this enhancement.

I am happy to take a look at this after submitting a PR for #694, it will take a little time to get familiar with structure and control flow of the operator

[gtully]

The number or ArtemisCrs that an operator is watching would be a good first metric.
The operator is largely stateless and event driven, but it could track sufficient information to report a count or even periodically generate the info with a query.
There is currently some caching of the pod status for the CR status, and multiple calls to the operand status that should be cached. Some combination of caching and updating should be able to generate a useful metric. We would need to think a bit to get the first one right!