-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OEP-57: Core Repositories #312
Changes from all commits
b237ba1
a3f228f
d3c7651
8bb6128
c74b896
c7f8aef
9787eae
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,166 @@ | ||||||||
=========================== | ||||||||
OEP-57: Core Repositories | ||||||||
=========================== | ||||||||
|
||||||||
.. list-table:: | ||||||||
:widths: 25 75 | ||||||||
|
||||||||
* - OEP-57 | ||||||||
- :doc:`OEP-57 <oep-0057-core-repositories>` | ||||||||
* - Title | ||||||||
- Core Repositories | ||||||||
* - Last Modified | ||||||||
- 2022-03-XX | ||||||||
* - Authors | ||||||||
- Kyle McCormick <[email protected]> | ||||||||
* - Arbiter | ||||||||
- Joe Example <[email protected]> | ||||||||
* - Status | ||||||||
- Draft | ||||||||
* - Type | ||||||||
- Process | ||||||||
* - Created | ||||||||
- 2022-03-XX | ||||||||
* - Review Period | ||||||||
- 2022-XX-XX - 2022-XX-XX | ||||||||
|
||||||||
.. contents:: | ||||||||
:local: | ||||||||
:depth: 3 | ||||||||
|
||||||||
|
||||||||
Abstract | ||||||||
-------- | ||||||||
|
||||||||
TODO | ||||||||
|
||||||||
.. note:: | ||||||||
|
||||||||
This OEP is a big ol' work-in-progress. Feel free to leave early feedback, but please keep in mind that it's still a rough outline. Lots of this stuff is my stream-of-consciousness. It's not representative of tCRIL's or anyone else's plans/opinions. -Kyle | ||||||||
|
||||||||
|
||||||||
Motivation | ||||||||
---------- | ||||||||
|
||||||||
There is a desire to make the `openedx GitHub organization`_ easier to manage and understand. To do this, we must be judicious about introducing new repositories and consider removing some existing repositories. At the same time, contributors to new repositories are eager to share their work with the Open edX community and, understandably, often desire for the upstream of their repository to become reflected under the GitHub organization that bears the Open edX project's name. | ||||||||
|
||||||||
To best accomodate both these interests in a fair and inclusive way, this OEP aims to: | ||||||||
|
||||||||
* establish the purpose of the openedx GitHub organization, | ||||||||
* define a transparent process for the management of the organization's set of repositories, and | ||||||||
* explain how community members can prepare their repositories for inclusion in the organization. | ||||||||
|
||||||||
Definition of Core Repositories | ||||||||
------------------------------- | ||||||||
|
||||||||
* We define the *Core repositories of Open edX* (or in this context, "the Core") as the entire set of public repositories hosted in the `openedx GitHub organization`_. | ||||||||
|
||||||||
* Phrased another way: A repository is in the Core if and only if it is public and in the openedx GitHub organization. | ||||||||
|
||||||||
* Terminology notes: | ||||||||
|
||||||||
* The choice of *Core* here is inspired by the Core Contributor program, which exists in part to grant community members access to repositories in the openedx GitHub organization. | ||||||||
|
||||||||
* This does not preclude other, smaller "cores" of Open edX (such as a *learning core*, *core utilities*, etc.) from existing within the Core repositories. | ||||||||
|
||||||||
* **What repositories are in the Core?** | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I realize that these criteria leave a gray area between repositories that must be in the core, but don't follow the standards for repositories that may be in the core. I have some thoughts on how to reconcile this but am curious what others think. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think I see what you're saying, but, if something is already a dependency of an Open edX release, it can't just be removed, so this feels to me like a grandfathered-in situation. I could see at line 91 adding something like:
|
||||||||
|
||||||||
* A repository *must* be in the Core if it is directly included in `latest Open edX release`_. | ||||||||
|
||||||||
* A repository *must* be in the Core if: | ||||||||
|
||||||||
* it is listed as a non-optional base dependency of a repository included in the latest Open edX release *and* | ||||||||
|
||||||||
* it is *not* maintained as a third-party package that is unspecific to Open edX, such as Django or Python Requests. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: I think "unspecific" means vague/not well defined, and that what we'd want here is "not specific to Open edX". (Or maybe "nonspecific"? I never use that one.) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||
|
||||||||
* In addition to the repositories that must be in the Core, a repository *may* be in the Core if: | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instead of saying that there is a Core with some repos that must be in Core and some that may be in Core, with (Core == openedx GitHub org), would it be useful to define tiers of repos that will end up in the openedx GitHub org? Something like:
I realize that's somewhat at odds with the notion of aligning this language to "Core Contributors". I'm wondering if we have other kinds of information we want to convey in how we group these repositories–how actively they're developed, for instance. We've also maintained a number of forks of external open source projects over the years for upgrade or compatibility reasons, and I think it's confusing to lump that in as "Core". There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am not sure if directly aligning this with the "core contributors" language is a good idea if there isn't a 1:1 mapping. I like Dave's tiers idea, or even some qualitative schema like github topics. I think we would need to clean up and track our topic schema for sure, but it could provide a nice way to, say, group together a set of MFEs that are part of a consolidated effort to deprecate something (and are thus optional but collectively required if you remove the deprecated app). I'm thinking specifically of the instructor dash MFE project that 2U and OpenCraft will be collaborating on. |
||||||||
|
||||||||
* it is of interest to more than one entity the Open edX community (TODO: this needs better criteria); and | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||
|
||||||||
* it has assigned `maintainers`_ who strive to: | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This only applies to code repos. We also have project repos (working group backlogs, etc). |
||||||||
|
||||||||
* review community code contributions, | ||||||||
* participate in community discussions related to the repository, and | ||||||||
* keep the repository compliant with the accepted Open edX Proposals (OEPs). | ||||||||
Comment on lines
+82
to
+84
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe I should drop just these requirements, and instead make sure they're in the list of maintenance responsibilities that OEP-0055 lays out. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree. I would say in the top-level bulled
|
||||||||
|
||||||||
* We declare that all access to write, configure, or otherwise administer Core repositories shall be granted through the `Core Contributor Program`_ and managed by its Program Administrators. | ||||||||
|
||||||||
* We declare that the addition or removal of repositories from the Core should by reviewed by existing Core Contributors. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As far as the process goes - we've been having trouble getting more than 30-40% of any given core contributors to participate in any CC program survey or standup we arrange. I think it would be good to, like the CC nomination process, define a minimum number of votes. The process should be public, where anyone can weigh in but only CCs get votes. Time-bounded is good. Would we specifically want maintainers to weigh in (who are already CCs)? Defining a rough template of how a repo "applies" for admission may be nice: repo name, purpose, core development team, who in the community is using it, plans for future development, and plan for maintainership all come to mind. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if we need to just define a couple of general mechanisims for community voting and then we can refer to them in all the OEPs where we need that kind of input. So far, we've got:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think having two voting mechanisms, "affirmative consent +N" defined as requiring N positive votes and no negatives, & "lazy assent: X days" as a time-bounded default acceptance of X days given no negatives, makes sense to me! What do you think would be the best way to define these? The absolute shortest OEP ever? 😁 |
||||||||
|
||||||||
* TODO: Define a review process. | ||||||||
|
||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. per Sarina's comment
Suggested change
|
||||||||
|
||||||||
Reasoning | ||||||||
--------- | ||||||||
|
||||||||
Many other repositories exist as part of the Open edX community. Some of these are forks of Core repositories, and others are upstream repositories in their own right. These various non-Core repositories are vital to different Open edX community members. Furthermore, the existence of non-Core repositories in general is important to the health of the Open edX project, which thrives on collaboration, experimentation, and extension. | ||||||||
|
||||||||
However, from these many repositories, we wish to enumerate a smaller set which: | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||
|
||||||||
* is a good target for enhancement, documentation, and technical investment; | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: before the bullets you use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I used singular verbs because There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, got it. Yeah for some reason it does read awkwardly (to me); I actually think then the problem is the You can also ignore me if you disagree. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree but also agree it's not a blocker. Offered a suggestion if you want to fold it in, @kdmccormick |
||||||||
|
||||||||
* defines a logical boundary within which it is fair to enforce Open edX Proposals; | ||||||||
|
||||||||
* defines an administrative boundary within which the Core Contributor Program applies; | ||||||||
|
||||||||
* makes the Open edX project more comprehensible by collecting its most critical components; | ||||||||
|
||||||||
* (TODO: should I add something about tCRIL stewardship/ownership?); and | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. along what lines were you thinking? |
||||||||
|
||||||||
* encompasses the first-party aspect of the Open edX release supply chain, identifying repositories that need to be: | ||||||||
|
||||||||
* monitoried for discloures under the `Open edX security policy`_, | ||||||||
* regularly updated with security patches, | ||||||||
* covered by automated unit tests and static analysis, and | ||||||||
* ready to deploy to production at any time. | ||||||||
|
||||||||
|
||||||||
Considerations | ||||||||
kdmccormick marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
-------------- | ||||||||
|
||||||||
|
||||||||
* Resolving legacy processes and access policies | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: I would make these subsection, not nested bullets.
|
||||||||
|
||||||||
* The process for reviewing addition/removal of repos via the Core Contributor program isn't ready to be spun up yet or even well-defined. In the meantime, tCRIL has been making these judgement calls. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But won't this document define that process? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah. I should have included the whole Considerations section in the PR description, not the OEP. It's meant to be a conversation with the folks previewing this OEP, not contents of the final OEP itself. |
||||||||
* 2U OCM and some of tCRIL retain historical access to many Core repositories outside of the Core Contributor program process. There is a plan to bring those folks under the Core Contributor umbrella, but it will take time. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm hopeful all of tCRIL will have gone through the CC nomination process before this lands (so tCRIL folx - get that nomination thread going!) |
||||||||
* tCRIL is currently using a few public repositories in the openedx organization for day-to-day work, such as ``openedx/tcril-engineering``. Upon acceptance of this OEP, they would need to make a new GitHub organization for those repositories. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤔 migrating hundreds of GH issues will be a ton of fun! |
||||||||
|
||||||||
* Cleaning up the Core | ||||||||
kdmccormick marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
* Many repositories in the openedx GitHub organization do not currently meet the Core criteria. We will need to review the organization contents, and for several repos, transfer them out or invest in them. | ||||||||
kdmccormick marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
* Even so, the definition of the Core still captures a large number of repositories, probably larger than we want our "Core" to be. We could reduce it by loosening the package dependencies of many services to being optional instead of strictly required. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if it will be valuable to have a second org for the looser but still close to core repos. For example repos tha aren't required but that are defacto the way you do X if you don't have other opinions? Even if X plugs into an interface. The idea of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it might be easier to use topics to tier the repos into different categories. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I just feel like it would be painful to maintain sync with users, groups, settings and such across a second org. Or dealing with moving repos around between those orgs. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We only use topics on a few repos now, and we use them in an inconsistent manner. But we could be more disciplined about it and make it something that folks could reasonably filter on, e.g. https://github.com/orgs/openedx/repositories?q=topic%3Afrontend&type=all&language=&sort= There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it's worth holding off on a specific solution for this to see how things shake out with the maintainers and core changes. It might be that each contributing org maintains its own separate collection, or that the peripheral libraries fall clearly into guilds by topic. I'm just spitballing here but basically I think we should shift these core things and then see what the new emergent patterns are. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One small update on this idea--as 2U starts thinking about pilots for OEP-55 I'm realizing that one of our pain points is the way that our internal "ownership" spreadsheet scope is defined, which is basically "anything on the openedx or edx orgs". Arguably that definition will not scale, but I also can't think of a good alternative. Maybe org-based maintainership gets encoded as a tag? e.g. I can filter for all the repos currently maintained by 2U. That would allow us to continue to automate some of our ownership stuff without having to do weird things like duplicate the entire maintainer roster in our spreadsheet (which currently only lists teams internal to 2U). |
||||||||
|
||||||||
* Ensuring the Core contains everything it should | ||||||||
|
||||||||
* There are four repositories directly included in Maple that aren't in the openedx organization. We'll either need to remove them from the release or transfer them into openedx. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where are we at on this for Nutmeg? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nutmeg master branches were created on Monday. There are four MFEs in the edx org included in Nutmeg, and two other repos that should probably be removed from the tagging process. |
||||||||
* This will take a while, but we'll need to look at the list of packages that are depended upon by released repositories. If any of them are Open edX code but outside the openedx organization, we'll need to move them into openedx, or refactor the dependencies so that they're optional. | ||||||||
|
||||||||
|
||||||||
|
||||||||
Motivation | ||||||||
---------- | ||||||||
|
||||||||
TODO | ||||||||
|
||||||||
|
||||||||
Alternatives Considered | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The big picture of this OEP as I understand it is:
I'm wondering if it's a good idea to join those two ideas together. As Dave commented, orgs imply access administrations, and as Sarina pointed out, there will be data to migrate if some repos have to be moved. Couldn't we find other ways of clearly indicating what is Core? As written, this OEP will require making yet another organization (or maybe more than one?) The phrase "open edx" refers to a software platform, but also a project. Why shouldn't the openedx organization include repos that are in the project but not the platform? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It might make the proposal more concrete if we could see a dry-run of what would have to move. What repos need to move into the openedx org, what repos would need to move out, and where they would go. |
||||||||
----------------------- | ||||||||
|
||||||||
TODO | ||||||||
|
||||||||
|
||||||||
Change History | ||||||||
-------------- | ||||||||
|
||||||||
2022-03-XX | ||||||||
========== | ||||||||
|
||||||||
* Document created in `initial pull request`_. | ||||||||
|
||||||||
|
||||||||
.. _openedx GitHub organization: https://github.com/openedx | ||||||||
.. _security policy: https://github.com/openedx/edx-platform/security/policy | ||||||||
.. _latest Open edX release: ./oep-0010-proc-openedx-releases.rst | ||||||||
.. _maintainers: ./oep-0055-project-maintainers.rst | ||||||||
.. _Core Contributor Program: ./oep-0054-core-contributors.rst | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Q: does this relative linking work in the RTD site? I've only seen explicit links in other OEPs. |
||||||||
.. _initial pull request: https://github.com/openedx/open-edx-proposals/pull/312 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: According to the OEP spec, you should be putting
proc
in the document name (oep-0057-proc-core-repositories
, see https://github.com/openedx/open-edx-proposals/tree/master/oeps/processes - all (except oep-1 lolol, and oep-54 where I also missed the notice) and https://open-edx-proposals.readthedocs.io/en/latest/processes/oep-0001.html#oep-header-preamble)