RFD - Create publicly accessible chat communication channel

[Adam-D-Lewis]

Status	Open for comments 💬
Author(s)	Adam-D-Lewis
Date Created	2024-08-27

Create publicly available real time chat communication channel

Summary

The Nebari team is considering implementing a public real-time chat communication channel to improve community engagement and increase visibility into developer discussions. Currently, real time chat is only available to Quansight devs and partially accessible by some partner orgs, in a private slack channel which is used daily. This proposal aims to address the current limitations of the private chat and GitHub-based communication and provide a more accessible platform for the Nebari community.

Relevant references

• Jupyter community discussing something similar in 2023
• Zulip free for Open Source

User/Developer Benefit

• Enhanced connection with the community. Helpful for Nebari developers to get to know their users.
• Lower barrier to entry for community members compared to GitHub issues
• Improved visibility to Nebari-related discussions

Design Proposal

I suggest we implement a public real-time chat platform, with Zulip mentioned as a potential option. The chat would include:

• A developer discussion channel (write-limited to those on at least one of the Nebari Teams, readable by all)
• Maybe A PR review request channel
• A Q&A channel
• other channels as needed
• Possible integration with Slack for existing team members who currently use Slack

Alternatives or approaches considered (if any)

• Discord: Concerns raised about potential future monetization and service restrictions
• Continuing with the current private slack and only GitHub-based communication for public use

Best practices

• Redirect help questions back to GitHub issues when appropriate

User impact

• Easier access to Nebari developers and community
• Foster community growth
• More informal environment for discussions and brainstorming

Concerns

This could become a time suck for Nebari developers replacing important development work with responding to community questions.

To combat this, it would be ideal if some channels are only writeable by those on the developer team, but readable by all. This allows devs to only subscribe to those channels selectively (or none at all) if they feel notifications are disrupting their productivity. Also, ideally we can leave some post pinned or in a channel description to educate users. It's also important for users to realize that b/c it's an open source project, there is no guarantee of support. If users need guaranteed support, they can reach out to Quansight about their Nebari Services.

How to ensure that the chat remains active and doesn't become unused like previous attempts (e.g., Gitter)?

I think use will be guaranteed by archiving the current Quansight-internal Nebari development channels, to nudge developers to use the new communication medium set up by this RFD.

How to ensure developers don't divulge any confidential information in the now public channel?

This RFD includes a migration of future Nebari conversation from Quansight internal chat to the new public chat. How can we guarantee that private information is not mentioned in the new medium? I feel that educating Quansight developers that the new chat is public would be sufficient. However, if concerns remain I'd propose that this new chat medium be accessible to Quansight only for the e.g. first month. At the end of the month, we can search to see if any confidential information was posted. If not, it's likely not an issue. There is also the possibility of creating a bot to monitor for keywords in the slack channels and notify Quansight staff if anyone uses any of those keywords in the public channel, then Quansight staff could check and delete the info if necessary, but of course that is more work to set up.

Ideally bugs will be recorded as github issues and questions recorded in Github Discussions. How do we ensure this happens?

I've seen Slack integrations by Marvin by Prefect where a priviliged user can write /issue on a slack thread and a bot will open a github issue and transfer the details there. Perhaps something similar could be done on whatever platform we choose (e.g. Zulip, Discord) for issues and questions being moved to github discussions. This type of integration would lower the barrier to move those things over, but is effort to set up.

Who will be responsible for maintaining and moderating the public channel?

Ideally, there would be a community team that manages this, but I don't think we're large enough to warrant creating that team explicitly at the moment so I'd say all of the core team should be admins on the chat. The maintenance team will respond in the chat on a daily rotation.

Unresolved questions

• Which platform to choose?
  • Top contenders are Zulip and Gitter/Matrix at the moment, but others can be considered
• Does Zulip's slack integration work well?
  • slack to zulip - https://zulip.com/integrations/doc/slack
  • zulip to slack - maybe https://integrately.com/integrations/slack/zulip
• Can Zulip create publicly readable channels, but limit who can write to channels?

[jaimergp]

Does Zulip's slack integration work well?

I've seen this in use with Napari/CZI. We had a private room in Zulip that enabled two-way communication with a private Slack channel

[Adam-D-Lewis]

Thanks @jaimergp, if you know how they're doing that, that would be helpful. The slack integration from zulip appears to be unidirectional.

[jaimergp]

There's a bot account that seems to be "copy-pasting" messages back and forth, but I don't have much details. What I see from the Zulip side is something like this:

Zulip user:
  This is a text message
Slack integration bot:
  - This is a user from Slack: Another text message
  - Yet another user from Slack: More stuff

Edit: it looks a lot like this: https://zulip.com/integrations/doc/slack

[Adam-D-Lewis]

There is also the option to use Slack free and then export all the messages to Zulip for keeping history which I've seen the Julia community suggest. I don't know if they ended up actually doing that or not though.

[dcmcand]

My top two concerns are that this could either

1. turn into a time suck - we don't have any dedicated DevRel, community manager, or support folks who could be tasked with maintaining this, so we would need to figure out what kind of time this would take and whose responsibility it is.
2. This ends up not being used at all. I would not want the time and effort to set this up to be wasted.

I know you have commented on both of these issues above, but I think clarity around who owns the impenmentation and communication here is important. We would also need to make an effort to direct people to this channel.

I do think it is a good idea, and I don't like that most of the communications for an open source project are currently locked into a private slack.

For the platform, I would favor the free Slack since we are all very used to using Slack and it wouldn't require adopting a new tool from anyone. I know there are limitations from the free slack as far as message retention, but I think that is reasonable given the fact that it is an open source project. It will still represent a large step forward it openness and clarity of communications.

[pavithraes]

Thanks for opening a detailed discussion. While I like the idea, I don't think we're at the stage to do this yet.

My primary concern about a new space is further fragmenting Nebari discussions. Currently, Nebari discussions happen in several spaces: Quansight's various internal Slack channels, JATIC slack & repositories, various Nebari repository issues & discussions, internal and community meetings. conda-store is deeply coupled with Nebari, and all the conda-store spaces often have some Nebari discussions as well. Keeping all information in sync and up-to-date, and maintaining GitHub as the source of truth, takes effort. We do try (especially shoutout to @kcpevey here) but I think we have a lot of scope for improvement.

Additionally, I don't think the benefits outweigh the costs (time & effort in setting up and managing the new space) here:

Lower barrier to entry for community members compared to GitHub issues

I disagree, and think this is a subjective. From past experience, people interested in a project first check out the repo and issues/PRs. Joining a Slack space (or any other sync platform) is the second step. I'll note it does help folks take the next step in becoming active community members, and we can identify and support folks on this journey. However, I think GitHub is the first step, and participating and following-up on discussions here is a higher priority.

Improved visibility to Nebari-related discussions

This can be a potential benefit only if the preliminary steps are satisfied - migration of current discussion, continued usage and moderations of the new space. Furthermore, real-time chat does not capture background & context. If the space is intended for new community members, I'd argue that we should encourage them to communicate on a platform like GitHub where they can get a clearer picture with all the necessary context. Again, this assumes our GitHub repo/issues/discussions capture all information and decisions. I think we do want that to be the case for Nebari, and we can prioritize working towards that.

Aside, I spent ~1y moderating Wikimedia's Zulip space for newcomers & we tried it for Bokeh for a few months. It's an excellent tool, but has a learning curve for the primary moderator (unless things have changed recently) who'll set up hygiene practices+workflows. Additionally, I suspect folks might default to going back to Slack channels as the more familiar option if they aren't already in the practice of keeping up with other discussion forums and find Zulip's discussion paradigm to be too different. If we do this now or in the future, I'd suggest starting with Slack as the platform familiar to most of the current community members.

[Adam-D-Lewis]

We discussed this in the Quansight internal Nebari meeting today when more concerns about the time involved to maintain a channel open to everyone. One alternative that was discussed was to have a publicly readable Nebari slack for members of the Nebari teams. Others can read them, but not post. Maybe still put some non postable channels directing people to github issues/discussions (e.g. a non-usable Q&A channel which only has a comment saying "Go post on Github Discussions! Here's the link." The benefit is that the development discussion which sometimes happens in Nebari could at least be put in a publicly readable place. Any interaction, discussion from others would still happen on Github.

[dharhas]

Looking at https://app.gitter.im/#/room/#jupyterhub_jupyterhub:gitter.im

one other issue in a public channel which can become a time suck is dealing with spam.

Also, slack is a bad tool for this. It is fairly hard to make an "open" channel on an enterprise account and the "free slack org" that scipy etc uses for conferences is a bad idea.

1. you need to log into yet another slack workspace with multiple channels
2. Losing all your messages every 10k posts defeats the purpose of having conversations and decisions tracked

github issues/discussion have the advantage that conversations are tracked well for posterity even if they lack some of the immediacy of a real time conversational channel.

[Adam-D-Lewis]

I'll close this since it seems Rejected for now.

[pdurbin]

@Adam-D-Lewis psst https://github.com/pdurbin/slopi-communication 😄

Our chat is over at https://dataverse.zulipchat.com

Some related reading: https://opensource.com/open-organization/17/11/transparency-dataverse-project