Butter Robot: "What is my purpose?"
Rick Sanchez: "You pass butter".
This is a Clojure application designed to keep in sync a main repository and a secondary repository which is a sub-part (most commonly a folder) of the first, similarly to a git submodule, but with none of the hassle.
In fact, with this bot you can keep working on your main repository with the same workflow, but can still separate a part of it (and maybe open source it).
This is especially useful in case of monorepos.
At WorksHub, we use a single repository for our application, which consists of a Clojurescript frontend and Clojure backend. We recently wanted to open source our frontend and tried to use git submodules
. This resulted in a very clunky workflow, because often features and requests spanned both frontend and backend, so we needed to open PRs in two separate repositories, and syncing them was not very easy (especially the pinning of the version).
From these difficulties, we changed approach and gravitated towards a sync-based workflow that satisfies the following criteria:
- Two separate repositories (one private, with backend and frontend, and one public with frontend code only)
- Our workflow is impacted as little as possible (i.e. the bot should work mostly automatically)
- Changes from one repo are propagated in the other, and authorship is kept intact
- A main repository (defined from now on as
server-repo
) on GitHub - A secondary repository (defined from now on as
client-repo
) on GitHub - One GitHub account (with
auth
token) to use as "actor" for all the synchronization (therefore it needs write access to both repositories).
- Configure the environment variables (as documented at the bottom of this README)
- Run
flow-bot
in a server somewhere - On GitHub, go to
client-repo
> Settings > Webhooks and click on "Add webhook" - Enter the URL (and the port, if necessary) of your instance of
flow-bot
in "Paylod URL" - Set
Content-type
toapplication/json
- Select "Send me everything"
- Ensure the "Active" checkbox is checked
- Click on "Add webhook"
- Repeat steps 3-8 for the
server-repo
Whenever there is an open PR on client-repo
:
- The application waits for a PR comment that contains the
magic-string
(in our exampleOK TO MERGE
) - The application checks out the PR branch on
client-repo
- The application then checks out
master
onserver-repo
, creates a new branch calledclient-$i
where$i
is the PR number on GitHub - The application copies over the
client
folder fromclient-repo
to onserver-repo
- The application commits the changes to
server-repo
attributing the commit authorship to the original committer of the PR - The application creates a new PR on
server-repo
Whenever a PR that originated from client-repo
gets merged in server-repo
:
- The application keeps track of the original author in local app-state
- The application comments over the
client-repo
PR, informing the original committer that their PR was finally merged upstream - The application tries to close the PR on
client-repo
(Note: this is currently not working, GitHub does respond to our API call but the PR is not closed, requiring manual intervention)
Whenever there is a new commit on master
branch on server-repo
:
- The application does a checkout of
master
onclient-repo
and copies over theclient
folder fromserver-repo
toclient-repo
- The application commits the changes to
client-repo
, giving correct attribution. - The commit is attributed to the author of the first commit in the merged PR. In this way, if the changes was originally from
client-repo
, we attribute the authorship to the correct contributor.
This program uses GNU version of xargs
. If you want to hack on flow-bot are running under macOS, make sure to execute
brew install findutils
and follow the instructions on screen to add the GNU versions of those programs on your machine.
This is necessary because the version of xargs that comes shipped with macOS does not support the -r
argument used by the script
The program needs the following environment variables, supplied, for example, with a .lein-env
file
{:server-org "test-org-integration"
:server-repo "app"
:client-org "test-org-integration"
:client-repo "client"
:client-folder "client"
:magic-string "OK TO MERGE"
:git-user "GitUser"
:git-email "[email protected]"
:git-token "SUPERSECRETTOKEN"
:port "3000"}