Cheatsheet for working with Concourse CI
This cheatsheet lists various useful tips and tricks to use with Concourse CI and thus omits the very basics such as logging in or setting pipeline from yaml file.
# for bash, put the following on your .bashrc
source <(fly completion --shell bash)
# for zsh, put the following on your .zshrc
source <(fly completion --shell zsh)# lists all the accessible pipelines regardless of what team you're on in the target
fly -t <your_target> ps -afly -t <your_target> workers --detailsfly -t <your_target> containers
# above command also provides handle id which you can use to intercept
fly -t <your_target> intercept --handle <handle_id> # eg. handle_id - 5f588b86-116b-4e7c-5bef-811dad839539# simple example to connect to a particular pipeline's specific job
fly -t <your_target> intercept --job <PIPELINE>/<JOB_NAME>
# simple example to connect to a particular pipeline's specific job and run specific binary
# this is useful to specify custom shell such as sh in alpine or any arbitrary command during intercept
fly -t <your_target> intercept --job <PIPELINE>/<JOB_NAME> <your_custom_binary>fly -t <your_target> curl /api/v1/info
> {"version":"6.1.0","worker_version":"2.2","external_url":"https://concourse.example.com"}
fly -t <your_target> curl /api/v1/builds
> json_array_of build_listsThis is often useful for open-source projects so that the build pipeline is visible for unauthenticated users.
fly -t <your_target> expose-pipeline --pipeline <YOUR_PIPELINE>Concourse has a handy fly command to format your pipeline in a "canonical" form.
Omitting -w / --write would print formatted pipeline on stdout instead.
This is useful when you've made a mess of your yaml configuration formatting.
fly format-pipeline -c <YOUR_PIPELINE.yml> -wBy default, concourse sets team on a target when you login and its cumbersome
to manage multiple targets just to be working on a different team space. Fly
commands support --team so you can specify another team name in a single target.
However, this is a work in progress as tracked HERE
so make sure you check the progress in above issue or by checking the help information.
- Use
privileged: truein the task config.
- task: some-task
privileged: true- You can use icon name from Material Design icon to get nice little icon by your resource.
- resources:
- name: my-image
type: registry-image
icon: dockerYou can use YAML anchor to re-use configuration blocks and remove duplicates.
Often times, you can use file directive but still the anchor syntax can be useful.
# this is an example from concourse docs
# the following repetitive blocks can be shortened using yaml anchor syntax
large_value:
do_the_thing: true
with_these_values: [1, 2, 3]
duplicate_value:
do_the_thing: true
with_these_values: [1, 2, 3]
# look how we anchor a block with &anchor_name syntax and reference it with *anchor_name
large_value: &my_anchor
do_the_thing: true
with_these_values: [1, 2, 3]
duplicate_value: *my_anchorThe same anchor syntax can be used to merge yaml objects. On the example below, you can see how we can avoid duplicate AWS ECR configuration by using anchor.
aws-ecr-config: &aws-ecr-config
aws_role_arn: ((ecr-role-arn))
aws_region: ((ecr-region))
aws_access_key_id: ((ecr-access-key-id))
aws_secret_access_key: ((ecr-secret-access-key))
resources:
- name: elixir-1.8.2
type: registry-image
source:
repository: engineering/elixir
tag: 1.8.2
<<: *aws-ecr-config
- name: elixir-1.10.1
type: registry-image
source:
repository: engineering/elixir
tag: 1.10.1
<<: *aws-ecr-configHere's another example:
jobs:
- name: job-1
plan:
- &task-1
task: task-1
config:
platform: linux
- &task-2
task: task-2
config:
platform: linux
- name: job-2
- *task-1
- *task-2# you can configure and override default container limits
# cpu - max amount of CPU available to task container, measured in shares
# memory - max amount of memory available to task container
# 0 means unlimited
jobs:
- name: container-limits-job
plan:
- task: task-with-container-limits
config:
platform: linux
image_resource:
type: mock
source: {mirror_self: true}
container_limits:
cpu: 512
memory: 1GB
run:
path: sh
args: ["-c", "echo hello"]While yaml tricks are nice for smaller pipelines, complex pipelines
benefit from a well-defined structure. Check out the command schema
and task step config
that shows an alternative file that allows you to point to a .yml
containing the task config.
A good starting example follows:
techgaun at techgaun in /home/techgaun/projects/rpi-ha
$ tre
1 .
2 └── .ci
3 ├── dockerfiles
4 │ ├── Dockerfile.alpine
5 │ ├── Dockerfile.buster
6 │ └── Dockerfile.distroless
7 ├── pipelines
8 │ ├── k8-build.yml
9 │ └── swarm-build.yml
10 ├── scripts
11 │ ├── build
12 │ ├── init
13 │ └── test
14 └── tasks
15 ├── build.yml
16 ├── e2e.yml
17 └── test.yml
18
19 5 directories, 11 files- You can get a badge for your pipeline with the following URL:
/api/v1/teams/{team}/pipelines/{pipeline}/badge
Example from Concourse CI itself:
# snippet for above SVG
[](https://ci.concourse-ci.org/teams/main/pipelines/concourse)
- You can get a badge for your pipeline's specific jobs with the following URL:
/api/v1/teams/{team}/pipelines/{pipeline}/jobs/{job}/badge
Example from Concourse CI itself:
# snippet for above SVG
[](https://ci.concourse-ci.org/teams/main/pipelines/concourse/jobs/unit)
- Additionally, Concourse API supports a team's pipeline in a CCMenu compatible XML file.
/api/v1/teams/{team}/cc.xml
- Concourse CI Pipeline Editor - Provides validation and content assist for Concourse CI pipeline and task configuration yml files
- Concourse-Vis - A plugin to preview Concourse pipelines in Atom.
- Concourse CI Pipeline Dashboard - Example dashboard/pipelines
- Concourse Internals - Deeper understanding of Concourse
- Concourse Tutorial by Stark & Wayne - A great introduction to Concourse
- Pipelines Used by Concourse Team - A collection of pipelines used by the Concourse Team
- Sample Concourse Pipeline Examples - Fixtures from Concourse source code to understand pipelines better