DevOps & Integration
One of the student of this project has an active devops P3 project, and is using this project as a test of his (in)capacity in all forms of github automation.
All the automation scripts are done using github actions. Although it runs on linux, wich is not as superior as windows OS-wise, github actions are a very effective and efficient ways of automating the boring tasks.
Although GitHub is dominating in the open source software paradigm, gitlab has a lot to offer. Especially, it has a lot of project management and gestion features that are sometimes lacking in github.
One of these well-liked gitlab features is related to the git workflow that we employ in this repository.
This workflow is very simple :
Where feature is a branch corresponding to an issue in the git. This workflow is very convenient because it fits very well with a good work package schema defining in detail the jobs to do. These jobs are fragmented as issues created at the start of the project.
Each feature is a branch that directly corresponds to its issue in github, said issue describing a unit of work to be done by the dev team.
To use this workflow at its full capacity, the work units should be as atomic and coherent as possible.
But in gitHub, there is no way to easily create and link a branch from an issue. Thath's where Create Issue Branch (CIB) comes into place !
CIB is a module to add to a repository, via app or via an action (.yml). We took the second option, as we like to have our actions as close to the code as possible.
For the action installation, CIB needs two .yml (action files) :
- The action file, under .github/workflow. It can be named however we want, because it is the description of the action to do (with its trigger)
- The configuration file, if we want to customize our CIB (we do). It must always be at .github/issue-branch.yml for the bot to recognize it
The action file describes the github action and the triggers for it.
As we use CIB on comment, we put the triggers of the action on the issues assignation and comments (on:). The trigger on assignations is not usefull right now, but recommended as the other mod of CIB is to create a branch on issue assignation (and we would like the bot to work if we change mode).
The action simply calls the CIB github (uses : X/[email protected]) to create the issue.
The bot can be configured to change some of its mecanics.
We use two configurations :
- The mode is set to chatops. The default mode create the branch on issue assignation. With chatops, it creates the branch on a "/cib" comment
- The source branch of the created branches are the "default" branch of the repository by standard. However, our default branch is main, and we would like to create feature branches from develop. As such, we tell the bot that all the issues with a label matching the pattern "*" (so, all the issues as far as we are concerned) are to be created from the "develop" branch. Another usefull configuration would have been the naming pattern for the created branches, but we find the standard pattern very good (#issue-name)
We can now call the CIB bot by commenting /cib under any issue
The bot responds by confirming the creation of the branch.
At the commentary, and as we say in the trigger part of the .yml, the action is called on github :
As pull request are more prevalent in the github ecosystem than simple merges from git, we made a little addendum to the CIB action.
This addendum allows the action to close issues on corresponding pull requests.
The only requirement is that the issue name contains the number of the branch as "issue-n°", or that it is created with the vanilla naming scheme of the CIB.
We use the vanilla version, so it is no problem for all branches created from the /cib command.
- The action now listens (has a trigger 'on:') to closed pull requests
- The configuration file indicate to the runner that it must closes corresponding issues on pull requests
Continuous Integration is a part of DevOps consisting in automatizing all sort of tests during the developpement of the app. For Caravel, we decided to go with automatized unit testing.
We decided yet again to use github action instead of external solutions (jenkins, travis-CI, circle-CI, etc...) because :
- Github actions are tighlty coupled with dockers and containers (as seen with mySQL)
- Github actions are executed on-the-fly on github shared runners
- The action files are in the .github/workflow folder - automation & CI/CD pipeline stay with the code (efficiency, maintainability...)
Github Action offers a simple solution to unit testing for laravel. It uses mySQL to setup a light database and launch all unit tests.
However, the team is not sure that SQLite would fit all our requirement as the project grow. We would also like to test on the same DB that we use for developpement & deployement.
From the base of the SQLite action file, we added the use of MYSQL.
Github offers "services". They are dockers (with images from dockerhub in our case) started parallel to our main runner, and automatically open for communication.
The runner create a service with the dockerhub MYSQL image. We then create the DB, and wait for it to respond to pings.
The runner then access to the DB using the password stocked on our github repository as secret. The secret from the github repository is encrypted, and will be known only from the runner. The github secret matches the .env.example file used to create the mysql DB. It is evident that this password is only used for testing - not for developpement nor deployement.
Credits to "Test automation for Laravel 7 and MySQL with GitHub Actions " for the MYSQL setup exemple.
Both actions are kept :
- The "laravel" simple action using SQLite, because as it is proposed by github, it is more reliable
- The "Caravel CI" complete action using MYSQL, because it is our work (less reliable) but more fit for our project
To test them in a very simple setting, we pushed a double passing unit test on the develop branch. Then, we pushed one of them failing to see if the actions responded correctly.
So far, the tests are working and triggered each time a branch is pushed. The devs will make sur not to merge a failing branch on develop or master (and will be punished accordingly if they make so).
To be done once the servers are assigned