Merge with grafana.grafana collection

[rrey]

Grafana labs recently created a collection that leaves inside the Grafana GitHub Organization.

Having 2 collections for Grafana (grafana.grafana and community.grafana) could be confusing for users and we started a discussion with the grafana.grafana maintainer (@ishanjainn) and started discussing a possible merge of the collections under the grafana roof (The Grafana GitHub org).

Mighty @gundalow suggested that we made the discussion public for transparency and to gather feedback from the community.
So here comes this GitHub Discussion, hoping to benefit from the threaded answer in case a subject start going wild.

If you are a user of Grafana collection (the one or the other) and have any comment regarding the merge don't hesitate to provide feedback through this discussion.

The benefit I see from moving to Grafana Org are:

• Increased visibility
• Having people at Grafana Labs more involved in the collection maintenance, allowing the collection to stay up to date and working with new Grafana versions.

Let's aim for a super smooth migration where community collection users could simply change the collection namespace from community to grafana and have everything working.

[rrey]

TODO list to prepare the merge

First observations from @ishanjainn:

1. Grafana dashboards, data sources and folders modules are present in both the collections. These three modules in grafana.grafana collection don't work for OSS grafana and I believe these modules in community.grafana do work for Grafana Cloud (Pls correct me If I am wrong) so we should use ones in community.grafana.
2. Similar for Tests and CI workflows, We will have to merge them such that they work for both collections.

So I guess one of the first things to do would be to:

• Identify modules that are supposed to work only with grafana cloud (because feature is only available there) and the one that should work everywhere.
• choose a naming convention for modules that works only with grafana cloud to make it super easy to identify them. grafana_cloud_xxxx seems good enough imo
• For modules that should work everywhere, choose between community and grafana one and add integration tests to cover cloud and oss deployments.

Note for maintainers : reply to this thread with things you have in mind so that we know where to find things that should result in a GitHub issue.

[ishanjainn]

Regarding naming conventions, It will be nice to keep the modules that work across both Cloud and OSS, For example Dashboards as grafana.dashboard instead of grafana.grafana_dashboard to avoid repetition and ultimately have shorter names. For cloud, Yes having a cloud_ prefix as it is in grafana.grafana should be good to differentiate.

[rrey]

Migration path for collection users

We will have to announce depreciation of the abandoned collection the best ways possible, I see at least:

• Bullhorn newsletter
• Depreciation message in documentation
• Depreciation message at runtime (Check with @gundalow if there is recommendation around this)

The depreciation message should be displayed for a long period to allow people to move to the new collection without being forced to.
The message will have to explicitly reference the new collection name/location.

[ishanjainn]

Thoughts on how we can proceed

src - community.grafana repo
dest - grafana.grafana repo

1. We create a branch (For example merge) and do all merge over the branch (Create sub-branches where necessary)

2. We start with plugin folder
   i. We can straight away copy all folders from src to dest except the modules folder as dest only has modules folder
   ii. It will be great if we can make sure the dashboard, datasource and folder modules in community.grafana work for Grafana Cloud aswell before we the next step
   iii. Preferably copy over the contents from grafana_dashboard.py, grafana_datasource.py, grafana_folder.py to dashboard.py,
   datasource.py, folder.py respectively. It would be better to keep module names as grafana.dashboard as its the same for both OSS and Cloud. And for Cloud modules, We can keep grafana.cloud_plugin (As its defined right now)
   iv. We should copy the rest of the files from src to dest as they aren't present in dest

3. For the tests folder, We should be able to copy over all folders from src except integrations to dest.
   i. With integrations/targets/(*modules)/tasks, It will be nice to have a single main.yml (as it is in community.grafana) and include other playbooks. This way it will be easier to seperate tests for Grafana Cloud and Grafana OSS.

4. For GitHub Actions, Once we have tests folder and plugins folder ready for merge, We could decide on how the actions should run as right now I am not sure how the structure would look at the end. Let me know if you have any ideas

5. Rest of the folders and README can be updated once we are done with moving over the plugins, tests and .github folders

We could create issues on the grafana.grafana repo for the merging over and pick up issues individually as we go along

[ishanjainn]

Ah, Yes that makes sense and preferable

[ishanjainn]

For moving with git history, any particular way you want to go ahead with?

[ishanjainn]

Hey @rrey, Once we have this figured, can we create issues in the grafana.grafana repo and pick them up parallelly?

[rrey]

@ishanjainn I'll create a PR to migrate datasource module on your side.
Let's work together on this, I have not much time to do it alone unfortunately.

[rrey]

I have second thoughts about the merging strategy. I think you were right, we should merge community in your repo and then make required changes modules per modules.
@ishanjainn could you create a branch that we will use to merge everything to avoid impacting your main ?

[rrey]

@webknjaz
@Lusitaniae
@kongslund
@ggiinnoo
@emcence
@gotmax23
@CWollinger

The community.grafana collection is merging with grafana.grafana collection. I initiated the work here but Grafana Labs requires that contributors signs the CLA.
Would you accept to sign the CLA ?

[ishanjainn]

Largely with this PR grafana/grafana-ansible-collection#28

[webknjaz]

FWIW, I don't like signing the CLA that would have an effect on future contributions. I don't think I'll be contributing into that repository. Is there a way for the CLA to only allow the contributions that are already made? I think I only contributed the GHA workflow updates, nothing functional in the collection itself. Also, that PR does not include my patches anyway. I see that https://github.com/grafana/grafana-ansible-collection/blob/main/.github/workflows/ci-test.yml is somewhat outdated and could benefit from implementing what's in https://github.com/ansible-collections/community.grafana/blob/main/.github/workflows/ansible-test.yml, though.

[gotmax23]

I remain against the CLA. I'm not sure if the Community Steering Committee has pull here, but if we do, I'm -1 to the merger unless the CLA requirement is dropped.

[webknjaz]

I wonder if Graphana would consider an opposite move — to the @ansible-collections org w/o the CLA requirement 🤔
Their licensing is compatible.

[gotmax23]

I've opened ansible-community/community-topics#212

[Andersson007]

@rrey hi, a couple of merge related questions:

• Is the merge totally done?
• If yes, should the readme of community.grafana be updated to redirect people to grafana.grafana?
• Should the collection get marked as deprecated on galaxy?
• Should community.grafana be deprecated and removed from the Ansible package? @felixfontein am i correct that it can be done using the process for renaming collections?
• Finally we could archive the repo after ^ is done?

[rrey]

The merge was abandoned because of the CLA requirement on the Grafana side.

I added people on the project last week to help maintain the project. Hopefully it will help making the project more active. Let's give it a last chance before deprecating it.

[Andersson007]

@rrey sure, thanks for taking care of it!