init

.github/workflows/container-ci.yml

@@ -0,0 +1,54 @@
+name: Containerized CI
+run-name: Container CI, triggered by ${{ github.event_name }}
+on: [push, pull_request]
+jobs:
+    plot:
+            name: Plot
+            runs-on: ubuntu-latest
+            steps:
+                - name: Check out repository code
+                  uses: actions/checkout@v4
+                - name: Set up Python 3.10
+                  uses: actions/setup-python@v5
+                  with:
+                    python-version: '3.10'
+                - name: Install dependencies
+                  run: |
+                    python -m pip install --upgrade pip
+                    pip install -r requirements.txt
+                - name: Run pylint
+                  run: |
+                    pylint --disable=import-outside-toplevel mandelbrot.py
+                - name: Generate the mandelbrot set
+                  run: |
+                    python mandelbrot.py
+                - name: Output as Artifact
+                  uses: actions/upload-artifact@v4
+                  with:
+                    name: mandelbrot
+                    path: mandelbrot.pdf
+                    retention-days: 1
+                    if-no-files-found: error
+    texlive:
+        runs-on: ubuntu-latest
+        needs: plot
+        container: 
+          image: texlive/texlive:latest
+        steps:
+        - name: Checkout Repository
+          uses: actions/checkout@v4
+        - name: Download artifact from run-matplotlib
+          uses: actions/download-artifact@v4
+          with:
+            name: mandelbrot
+            path:        
+        - name: Compile LaTeX document
+          run: |
+            latexmk -pdf main.tex
+        - name: Save matplotlib plot as atifact
+          uses: actions/upload-artifact@v4
+          with:
+            name: pdf
+            path: main.pdf
+            retention-days: 1
+            if-no-files-found: error

.github/workflows/hello-ci.yml

@@ -0,0 +1,14 @@
+name: Hello CI
+run-name: Hello ${{ github.actor }}, this action was triggered by ${{ github.event_name }}
+on: [push, pull_request]
+jobs:
+    hello_job:
+        name: hello
+        runs-on: ubuntu-latest
+        steps:
+            - run: echo "Hello CI, running on ${{ runner.os }}"
+            - name: Check out repository code
+              uses: actions/checkout@v4
+            - name: list files
+              run: |
+                ls ${{ github.workspace }}

.github/workflows/python-ci.yml

@@ -0,0 +1,32 @@
+name: First Python CI
+run-name: Python CI, triggered by ${{ github.event_name }}
+on: [push, pull_request]
+jobs:
+    plot:
+        name: Plot
+        runs-on: ubuntu-latest
+        steps:
+            - name: Check out repository code
+              uses: actions/checkout@v4
+            - name: Set up Python 3.10
+              uses: actions/setup-python@v5
+              with:
+                python-version: '3.10'
+            - name: Install dependencies
+              run: |
+                python -m pip install --upgrade pip
+                pip install -r requirements.txt
+            - name: Run pylint
+              run: |
+                pylint --disable=import-outside-toplevel mandelbrot.py
+            - name: Generate the mandelbrot set
+              run: |
+                python mandelbrot.py --pdf
+            - name: Output as Artifact
+              uses: actions/upload-artifact@v4
+              with:
+                name: mandelbrot
+                path: mandelbrot.pdf
+                retention-days: 1
+                if-no-files-found: error
+

.gitignore

@@ -0,0 +1,9 @@
+book
+venv
+
+*.aux
+*.log
+*.fdb_latexmk
+*.fls
+*.pdf
+

README.md

@@ -0,0 +1,14 @@
+
+# deRSE24: Getting up and Running with your first CI Pipeline
+
+[mdbook](https://deRSE-ci.beuscher.dev)
+
+## Prerequisites
+
+- GitHub Account
+- local instance of git, linked with GitHub account - if not: use [github.dev](https://github.dev)
+- text editor (vs code, vim, ...)
+
+## Usefull Tools
+
+- [actionlint](https://rhysd.github.io/actionlint/)

SUMMARY.md

@@ -0,0 +1,8 @@
+# Summary
+
+- [Introduction](./intro.md)
+  - [A git primer](./git.md)
+  - [Other useful tools](./tools.md)
+- [A Hello World CI](./hello-ci.md)
+- [Python and Artifacts](./python.md)
+- [Containerization](./container.md)

book.toml

@@ -0,0 +1,6 @@
+[book]
+authors = ["Maximilian Beuscher"]
+language = "en"
+multilingual = false
+src = ""
+title = "Getting up and running with your First CI/CD pipeline"

container.md

@@ -0,0 +1,43 @@
+# Running containers in an Action
+
+There is the possibility that a lot of environments needed cannot or only be created in a time consuming manner in one of the provided runners. While one possibility is to use a selfhosted runner a far more straight forward solution could be to use containerization. Containers can provide an easy way to setup consistent environments and there is already a long list of pre build containers available (eg. on [docker.hub](https://docker.hub).
+
+It is also possitble to use a `dockerfile` to define a custom container environment, but in most cases it is sufficient to use a prebuild one.
+
+## Practical Example: A LaTeX Container 
+
+In this example we will use the `texlive/texlive:latest` container to compile a simple latex document.  
+
+The sample document can be downloaded through
+
+```sh
+curl -L https://derse-ci.beuscher.dev/main.tex -o main.tex
+```
+
+Continuing our previous examples, we create a new job `texlive:`. 
+
+GitHub provides already a straight forward way of creating a docker environment on the `ubuntu-latest` image by adding a `container` key and specify the container.
+
+```yml
+{{#include .github/workflows/container-ci.yml:32:39}}
+```
+
+After checking out the repository we need to download the previously saved artifact using an action provided by GitHub>
+
+```yml
+{{#include .github/workflows/container-ci.yml:40:44}}
+```
+
+Finally we invoke `latexmk`, starting the compilation and create an Artifact for the `main.pdf`:
+
+```yml
+{{#include .github/workflows/container-ci.yml:45:47}}
+```
+
+The container part of the final workflow should be similar to:
+
+```yml
+{{#include .github/workflows/container-ci.yml:48:}}
+```
+
+Note that since the - in terms of file size - large container needs to be downloaded from [docker.hub](https://docker.hub) this workflow will run for some time.

git.md

@@ -0,0 +1,88 @@
+# A short Git primer
+
+A more complete guide to Git can be fount here:
+
+- [SoftwareCarpentry Git Intro](https://swcarpentry.github.io/git-novice/index.html)
+
+## Installation
+
+On Linux `git` should already be installed, on MacOs or Windows, refer to [git downloads](https://git-scm.com/downloads)
+
+### Setting up the credentials
+
+```sh
+git config --global user.name "Maximilian Beuscher"
+git config --global user.email "[email protected]"
+```
+
+### SSH key setup
+
+```sh
+cd ~/.ssh
+```
+
+```sh
+ssh-keygen -t ed25519 -C "[email protected]"
+```
+
+When prompted for a file name just accept with `enter` to get the default naming, then `cat` the public key and copy it to the clipboard.
+
+```sh
+cat ~/.ssh/id_ed25519.pub
+```
+
+Go to [Account > Keys](https://github.com/settings/keys) and create a new SSH Key.
+
+Check if ssh is configured correctly:
+
+```sh
+ssh -T [email protected]
+```
+
+### Adding a Remote
+
+When cloning the repository from GitHub the remote should be already configured correctly, this can be verified with
+
+```sh
+git remote -v
+```
+
+if not, modify the following command with your user and repository:
+
+```sh
+git remote add origin [email protected]:beuscher/repo.git
+```
+
+git also allows to use https, requiring a slightly different "syntax" for the remote origin.
+
+## Git Command Cheat Sheet
+
+Add a file to the git repository:
+
+```sh
+git add *
+```
+
+Check the current status of the repository, it will list for example uncommitted changes
+
+```sh
+git status
+```
+
+Commit changes
+
+```sh
+git commit -m [commit message]
+```
+
+Push changes to remote
+
+```sh
+git push origin main
+```
+
+Pull changes fro remote
+
+```sh
+git pull origin main
+```

hello-ci.md

@@ -0,0 +1,108 @@
+# Hello CI
+
+## Setup a git repository and add a remote
+
+First we start on GitHub.com and create a new repository by selecting on the top right **New** > **Repository** and choose a suitable name. Next we select **SSH** instead of **HTTPS** and copy the address to the clipboard.
+
+We then switch to our local command line and start by creating a new folder and `cd` into it:
+
+```sh
+mkdir first-ci
+cd first-ci
+```
+
+then we locally initialize a new git repository, ensure we are on branch `main` and add the remote "origin":
+
+```sh
+git init
+git checkout -b main
+git remote add origin [email protected]:beuscher/repo.git
+```
+
+one can verify this by running
+
+```sh
+git remote -v
+```
+
+then we push our sample files to remote using
+
+```sh
+git push origin main
+```
+
+Now we can start setting up our first CI pipeline.
+
+## A first workflow file
+
+Work flows are defined using `.yml` files located in the  `.github/workflows/` directory, where each workflow is defined in its own file. Thus we create a new workflow `hello-ci` by creating a file named `hello-ci.yml`.
+
+```sh
+mkdir .github/workflows/
+touch .github/workflows/hello-ci.yml
+```
+
+For each workflow some basic parameters are required, while `name` and `run-name` are technically optional (name would default to the file name of that action) they still should be set.
+The `run-name` parameter specifies the name of a workflow run. When triggered by a pull or push event it will default to the commit message.
+Here we will directly make use of a feature named Contexts, which allows us to access information about the repository, the current workflow run itself, the triggering event, etc. In this case we will use it to define the `run-name` in such a way that it will include who and which event triggered the run.
+
+One of the most important key:value pairs to be set is the `on: []` which defines which events will triggers this workflow, which we will set to `push, pull_request`.
+To allow for a manually triggered workflow the `workflow_dispatch` event must be set. It is also possible to define more complex events like the creation of an issue or to restrict the tigger conditions by depending on changes to a subdirectory, but this is out of scope for this tutorial.
+
+The final top level key we need to set is `jobs:`. It groups all jobs of the workflow together. For our workflow to actually do something we will need to add a job.
+
+```yml
+{{#include .github/workflows/hello-ci.yml::4}}
+```
+
+## The Hello CI job
+
+We start by creating our first job with the name `hello` and the job id `hello_job`.
+
+```yml
+{{#include .github/workflows/hello-ci.yml:5:6}}
+```
+
+Further jobs (dependent or independent of other) can be defined by adding further named keys under `jobs:`.
+
+When then have to choose a runner for our job or in other words: We need to choose a platform and OS. In most cases this will be `ubuntu-latest`.
+
+```yml
+{{#include .github/workflows/hello-ci.yml:5:7}}
+```
+
+GitHub also provides default runners for windows and mac (`windows-latest` and `macos-latest`) but one should note here, that those are billed with a factor x2, x10 on minutes.
+Further there is also the option of self hosted runners.
+
+### Steps
+
+Each job is split up in multiple steps. In each step one can directly specify a command to run or run a script. Each step can also have a name. For a lot of common tasks like to checkout/clone the repository into virtual machine GitHub provides default actions. In our case we specify `actions/checkout@v4` which will allow us to access the contents of this repository.
+We also `ls` to check that the checkout worked as expected.
+
+```yml
+{{#include .github/workflows/hello-ci.yml:8:}}
+```
+
+these actions are essentially GitHub repositories containing robust scripts performing these tasks.
+
+Now one could use `actionlint` to check for syntax errors.
+
+## a first pipeline run
+
+The contents of `hello_ci.yml` should now be something like:
+
+```yml
+{{#include .github/workflows/hello-ci.yml}}
+```
+
+Finally Sync with GitHub:
+
+```sh
+git add .github/workflows/hello-ci.yml
+git commit -m "hello-ci.yml"
+git push origin main
+```
+
+### view run on GitHub
+
+In our repository (on [github.com](github.com)) when go check the Actions tab and select a workflow run to view details.