Service Catalog Helm Chart Repo Migration

[mattfarina]

As part of the Helm v2 deprecation announcement we also announced that the stable and incubator repos were going away on November 13th. What was, unfortunately missed, is that Helm is hosting the service catalog helm repo and that will likely be going away as well. It is currently hosted by Google on a corp managed bucket.

Today I was reviewing what is going away and noticed the service catalog was wrapped up in that. I'm sorry we missed this. The service catalog repo was setup when Helm was part of Kubernetes. The stable and incubator repos went to the new CNCF project but their hosting stayed in the same place. The service catalog repo is hosted there as well.

As part of helm/community#114 we noted that the Helm community was ok with our repos going away on Nov 13th.

I think service catalog was forgotten in this.

There are two possible paths...

1. Work out new details on hosting this same bucket. I would reach out to @bgrant0607 as a start.
2. More the repo to a new location. One option is using GitHub pages. We have some tools that might help like chart releaser (even available in a GitHub action).

[mszostok]

Hi @mattfarina

Thanks for finding that 👍 so for now we do not know if the current URL (https://svc-catalog-charts.storage.googleapis.com) will be deprecated as there is still an option 1, right?

Please let us know if you will know sth more, so we will have enough time to switch our pipelines as described in second option and announce that change.

[mattfarina]

@mszostok If you want to pursue option 1 you will need to figure this out. @bgrant0607, I think, can help from the Google side. The hosting would need to transfer to a CNCF/k8s managed location. If you do not pursue this option I would expect it to be garbage collected on Nov 13th. If it does not happen it would be a pleasant surprise.

[bgrant0607]

@mszostok How feasible is it to change the path? @mattfarina is correct that the current plan is to delete the storage buckets on November 13.

[mszostok]

@bgrant0607 we need to back up all charts and push to new location as described in step 2, update release scripts and update documentation and announce that change as probably a lot of people are using this on their pipelines.

so this is doable, but we need to have manpower to set this up.

/cc @jberkhahn

[bgrant0607]

The alternative is to set up a new CNCF GCP org and billing account, setup necessary identities / accounts, groups, folders, authorization policies, etc., and transfer the projects.

[jberkhahn]

@bgrant0607 where are the k8s charts hosted? is there somewhere we could host our stuff and keep the same URL or has have they started using github pages

[bgrant0607]

@jberkhahn

The locations of stable and incubator repos are documented here:
https://github.com/helm/charts#how-do-i-install-these-charts

It looks like service catalog charts are hosted here, in another GCS bucket in the same GCP project:
https://svc-catalog-charts.storage.googleapis.com/

There's no way to move the storage and keep the URL the same.

[jberkhahn]

I don't understand. If the core k8s charts are hosted in a gcs bucket on a different account, why can't we get a bucket on whatever account is hosting the core k8s charts, remove the old bucket, and have the url route to the new bucket?

[bgrant0607]

@jberkhahn The current account used for all of the charts is going way. AIUI, you could create a new account and project, delete the current bucket, and try to create a new bucket with the same name in the new project.
https://cloud.google.com/storage/docs/moving-buckets

https://chartcenter.io/ was mentioned in helm/community#114. Is that an option for service catalog charts?

[bgrant0607]

Actually, are these charts still needed? The only updates in the last year look like:

      1896  2020-05-25T13:48:18Z  gs://svc-catalog-charts/ups-broker-0.3.0.tgz
      2048  2020-05-25T13:48:18Z  gs://svc-catalog-charts/test-broker-0.3.0.tgz
     22860  2020-05-25T13:48:18Z  gs://svc-catalog-charts/index.yaml
      3645  2020-05-25T13:48:18Z  gs://svc-catalog-charts/healthcheck-0.3.0.tgz
      9026  2020-05-25T13:48:18Z  gs://svc-catalog-charts/catalog-0.3.0.tgz
      9321  2020-05-20T08:35:29Z  gs://svc-catalog-charts/catalog-v0.2-0.2.3.tgz
      1899  2019-11-05T19:13:50Z  gs://svc-catalog-charts/ups-broker-0.3.0-beta.2.tgz
      2052  2019-11-05T19:13:50Z  gs://svc-catalog-charts/test-broker-0.3.0-beta.2.tgz
      3646  2019-11-05T19:13:50Z  gs://svc-catalog-charts/healthcheck-0.3.0-beta.2.tgz
      9012  2019-11-05T19:13:50Z  gs://svc-catalog-charts/catalog-0.3.0-beta.2.tgz
      1904  2019-11-05T16:51:59Z  gs://svc-catalog-charts/ups-broker-0.3.0-beta.1.tgz
      2057  2019-11-05T16:51:59Z  gs://svc-catalog-charts/test-broker-0.3.0-beta.1.tgz
      3652  2019-11-05T16:51:59Z  gs://svc-catalog-charts/healthcheck-0.3.0-beta.1.tgz
      9016  2019-11-05T16:51:59Z  gs://svc-catalog-charts/catalog-0.3.0-beta.1.tgz

[jhvhs]

@bgrant0607 We have decided to move our repos to GitHub pages. We have already moved the archive, and are almost done adapting the CI/CD pipeline to push the charts accordingly.

[scottrigby]

@jhvhs in case it's helpful, here a quick tool I sketched up to help with moving chart repo package history to github pages https://github.com/scottrigby/helm-adopt-package-history. It's already been used for several chart repos.

Ideally I'd love to use a tool like helm/chart-releaser to attach chart history version packages as GitHub Release artifacts, but that would have to be written to handle all package history for a chart, and I understand time is short here. As @mattfarina said, for new versions it works very well - a working example is here https://github.com/helm/charts-repo-actions-demo and many chart repos use this now.

[jhvhs]

Thank you, @scottrigby !

Ideally I'd love to use a tool like helm/chart-releaser

That's a brilliant tool, but sadly, as we have a set of multiple different charts in this repo, we have decided not to use the releases section.

[bgrant0607]

Thanks!