How to Use DevStream in an Air-gapped Environment

[daniel-hutao]

1. Background

We can currently only use dtm where the Internet can be connected, but a lot of company environments are air-gapped. I wanna find a way to use dtm in an air-gapped environment.

2. Some of the Major Materials I've Identified

1. dtm and dtm plugins
2. Container images
3. Helm charts
4. Something is needed inside the plugins(tools), such as Jenkins needs its plugins

Let's give the formulas together.

2.1 dtm and dtm plugins

• Provide a script, it can recognize the dtm version, plugin category, plugin CPU-arch, etc.

Anyway, we need to download the dtm and dtm plugins on the mac/Windows PC, and then run them on an air-gapped Linux environment.

2.2 Container Images

The image problem is not easy to solve, and we need to consider a bit more aspects, such as:

1. multi-nodes Kubernetes cluster. (Copy images to all hosts maybe not be easy)
2. some resource controllers(k8s Deployment, DaemonSet, etc.) may set the imagePullPolicy to Always, such as Jenkins. There are even some Deployments that are managed by a k8s Operator and dtm can't change the imagePullPolicy configuration through the API. (Prepare all images without an image repo isn't a good way)
3. The docker daemon or containerd may support HTTP or HTTPS only. (Deploy a Harbor before dtm apply isn't a good way, for the HTTPS cluster, certificate configuration may require some complicated operations.)

Therefore, we need to make some assumptions/requirements:

The air-gapped environment has an image repo already.

Then, what dtm needs to do is to make each plugin support the specified image name, for example: https://example.io:9000/jenkins/jenkins:2.346.3-jdk11

2.3 Helm Charts

That's easy, we could do it in 2 steps:

1. Provide a script like the above, it can download the charts properly;
2. Let dtm support the local chart installation.

2.4 Jenkins Plugins (Not dtm Plugins) or Other Things Need to Download

We identify the required stuff for each plugin and then tell the users how to get it manually through documentation?

Let's discuss these formulas together and give a to-do list after. @devstream-io/devstream

[IronCore864]

Points 2.1 to 2.3 look good.

The only problem I think is the Jenkins plugins. I think manually downloading the plugins and then using official CLI tools to install them is the way to go. See more here:

https://github.com/jenkinsci/plugin-installation-manager-tool

[daniel-hutao]

The coding progress is tracked by #1084

[steinliber]

2.4 we can download these plugins before, then install them in jenkins-cli. I think this can be done by script