Add services support

[pauldoomgov]

Addresses https://github.com/GSA-TTS/devtools-program/issues/47

• Launches services as separate applications with the same initial name as the "worker" instances
• Creates a route in the apps.internal domain to allow the main worker to connect to the service(s)
• Adds a network policy to allow access from the worker to the service(s)
• Allows each worker to have separate services, avoiding collision
• Cleans up after itself like a responsible runner

Here is some selected output using this .gitlab-ci.yml showing an "echo" service being contacted from an main worker step:

[cf-driver] Using SSH to connect to glrw-r34277582-p56364419-c0-j7131131154 and run steps$ curl http://${SERVICE_PREFIX}echo.apps.internal:8080
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0{
  "path": "/",
  "headers": {
    "host": "glrw-r34277582-p56364419-c0-j7131131154-svc-echo.apps.internal:8080",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "glrw-r34277582-p56364419-c0-j7131131154-svc-echo.apps.internal",
  "ip": "10.255.19.225",
  "ips": [],
  "protocol": "http",
  "query": {},
  "subdomains": [
    "glrw-r34277582-p56364419-c0-j7131131154-svc-echo"
  ],
  "xhr": false,
  "os": {
    "hostname": "0178ff2e-18a1-45e5-55ba-09ce"
  },
  "connection": {}

For context, the relevant .gitlab-ci.yml:

variables:
  SERVICE_PREFIX: "glrw-r$CI_RUNNER_ID-p$CI_PROJECT_ID-c$CI_CONCURRENT_PROJECT_ID-j$CI_JOB_ID-svc-"
stages:
- test
echo-test-job:
  stage: test
  services:
  - name: mendhak/http-https-echo:latest
    alias: echo
  script:
  - curl http://${SERVICE_PREFIX}echo.apps.internal:8080

[zjrgov]

This is very neat! LGTM

[pburkholder]

Just an update on where I am with reviewing this. A few months ago I worked through the tutorial "DevOps with GitLab CI Course - Build Pipelines and Deploy to AWS" -- and I finally got around to cleaning up the leftover resources on AWS and cloud.gov for this, and eliminating the bitrot. So it now deploys to https://peterb-gl-proud-wombat-qc.app.cloud.gov/ when using the GitLab instance runners.

But when I switch from GitLab Instance Runners to the cloud.gov Runner-Manager in peterb-runner the deployment fails trying to use the CF-CLI docker image:

[cf-driver] Installing dependencies into runner-38480029-project-54365529-concurrent-0-7150019953
[cf-driver] Ensuring git, git-lfs, and curl are installed
Required packages missing and I do not know what to do about it
ERROR: Job failed: exit status 1

... so, just naively switching from instance runners to project runners on cloud.gov is not turnkey. I'll work on this more in the morning.

[pauldoomgov]

Just an update on where I am with reviewing this. A few months ago I worked through the tutorial "DevOps with GitLab CI Course - Build Pipelines and Deploy to AWS" -- and I finally got around to cleaning up the leftover resources on AWS and cloud.gov for this, and eliminating the bitrot. So it now deploys to https://peterb-gl-proud-wombat-qc.app.cloud.gov/ when using the GitLab instance runners.

But when I switch from GitLab Instance Runners to the cloud.gov Runner-Manager in peterb-runner the deployment fails trying to use the CF-CLI docker image:

[cf-driver] Installing dependencies into runner-38480029-project-54365529-concurrent-0-7150019953
[cf-driver] Ensuring git, git-lfs, and curl are installed
Required packages missing and I do not know what to do about it
ERROR: Job failed: exit status 1

... so, just naively switching from instance runners to project runners on cloud.gov is not turnkey. I'll work on this more in the morning.

Created issue to track this: #9