Automatically overwrite dashboard UIDs with a prescribed naming pattern, on the provider side

[sed-i]

Issue

• https://github.com/canonical/observability/blob/main/decision-records/2024-08-29--dashboard-uid-collision.md

Solution

Use shake_256 to generate a prescribed uid, overwriting any pre-existing uid.

In tandem with:

• Add dashboard uid generation helper cos-lib#114
• Automatically overwrite dashboard UIDs with a prescribed naming pattern grafana-agent-operator#224
• Replace GrafanaDashboard with LZMABase64 grafana-agent-k8s-operator#337
• chore: update charm libraries cos-proxy-operator#175

Context

• Afaict, the only user of the .add_dashboard method is https://github.com/canonical/cos-registration-server-k8s-operator/blob/6e156f0b1367f7cc6ce12d89f6a367bf491fea7f/src/charm.py#L240.
• Dahsboard templates (example) are rendered on the consumer side (grafana itself). The templates themselves are still valid json, so it's ok to json.loads them in the provider side.

Testing Instructions

First, prepare the charms locally:

1. Copy the grafana_dashboard lib from this PR into grafana-agent-operator from Automatically overwrite dashboard UIDs with a prescribed naming pattern grafana-agent-operator#224.
2. Copy cos-agent lib from Automatically overwrite dashboard UIDs with a prescribed naming pattern grafana-agent-operator#224 into zookeeper.
3. In zookeeper, update deps using the following commands. Then remove all the --hash from requirements.txt:

poetry update
poetry lock
poetry install
poetry export -f requirements.txt --output requirements.tx

4. Pack and deploy all charms as follows:

graph LR
subgraph lxd
zookeeper --- grafana-agent
zookeeper2 --- grafana-agent
end
subgraph microk8s
grafana-agent --- grafana
end

Loading

bundle: kubernetes
applications:
  graf:
    charm: ./grafana-k8s_ubuntu-20.04-amd64.charm
    scale: 1
    trust: true
--- # overlay.yaml
applications:
  graf:
    offers:
      graf:
        endpoints:
        - grafana-dashboard
        acl:
          admin: admin

default-base: [email protected]/stable
saas:
  graf:
    url: microk8s:admin/graf.graf
applications:
  ga2:
    charm: ./grafana-agent_ubuntu-22.04-amd64.charm
  zk:
    charm: ./zookeeper_ubuntu-22.04-amd64.charm
    num_units: 2
  zk2:
    charm: ./zookeeper_ubuntu-22.04-amd64.charm
    num_units: 2
relations:
- - ga2:cos-agent
  - zk:cos-agent
- - ga2:cos-agent
  - zk2:cos-agent
- - ga2:grafana-dashboards-provider
  - graf:grafana-dashboard

Upgrade Notes

• This change will break existing bookmarks to existing dashboards, because charms that fetch-lib this change would have the uid of all dashboards changed.
• If you have more than one app of the same charm deployed, they would all need to be refreshed together to avoid dashboard title collision. If you do not refresh all of them, grafana will output the following logs:
  • dashboard title is not unique in folder
  • dashboards provisioning provider has no database write permissions because of duplicates
  • Not saving new dashboard due to restricted database access

To enable automatic uid collision prevention for the dashboards in your charm:

1. If cos-lib (cosl) is pinned in your charm, bump it to 0.0.50 or higher.
2. Fetch-lib cos-agent. You should use 0.13 or higher.
3. Refresh your charm.
4. Refresh grafana-agent to revision 369+.

[sed-i]

I'd appreciate your review, @Guillaumebeuzeboc, @taurus-forever, @jdkandersson, @cbartz, @camille-rodriguez, @slapcat, @facundofc.

[jdkandersson]

I'll delegate to @cbartz who has the best understanding of the issue in our team, thank you for preparing the PR!

[sed-i]

Tested with:

graph LR
graf --- am
graf --- am2

Loading

Where alertmanager was first deployed from edge (with the "old" dashboard lib), then refreshed from path with the new lib from this PR.

bundle: kubernetes
applications:
  am:
    charm: local:alertmanager-k8s-1
    resources:
      alertmanager-image: 99
    scale: 2
    trust: true
  am2:
    charm: local:alertmanager-k8s-2
    resources:
      alertmanager-image: 99
    scale: 1
    trust: true
  graf:
    charm: grafana-k8s
    channel: latest/stable
    revision: 122
    resources:
      grafana-image: 70
      litestream-image: 45
    scale: 1
    trust: true
relations:
- - graf:grafana-dashboard
  - am:grafana-dashboard
- - graf:grafana-dashboard
  - am2:grafana-dashboard

[lucabello]

Approving, but please fix integration tests before merging :)