GitHub_Actions

GitHub Actions

There are three current workflows in GitHub actions that are triggered by either pushes or tags / releases.

Create and upload release

Triggered by a tag matching any of the following regexes:

• 'v*' # Push events to matching v*, i.e v1.0, v20.15.10 etc
• 'pre-v*' # Push events to matching pre-v, i.ve pre-v1.0, pre-v20.15.10
• 'latest'

This creates a new 'release' bundle for download from the releases page

Update autocompletion

The autocompletion scripts are generated by the appspec / shell completions repo. Autocompletion is awesome and you will need it. Autocompletions are now auto-updated with GitHub Actions.

Sync tools and workflows

This is how tools and workflows are synced to ICA. It requires some work to set up. This workflow runs when a file ending in .cwl changes on GitHub.

Setting up the secrets

• ICA_BASE_URL:
  • Is hardly a secret
  • Set to `https://aps2.platform.illumina.com"
• ICA_ACCESS_TOKENS_JSON:
  • Base64 encoded
  • Uses the following json hierarchy:

    {
        "PROJECT_NAME": {
            "ICA_ACCESS_TOKEN": "<TOKEN>"
        },
        "PROJECT_NAME_2": {
            "ICA_ACCESS_TOKEN": "<TOKEN>"
        }
    } 
    

  • You can use the following lines of code to manually create the base64 json string - this has the advantages that tokens created in CWL-ICA do NOT have access to data but only WES and TES scopes.
  • GH Actions does not allow curly braces in env vars which is why we must first encode to base64.
  • The encoded secrets json is masked from the workflow logs however it is important to note that encoded does NOT equal encrypted. The string can be merely decoded with base64 -d

  # Globals
  API_KEY='INSERT_API_KEY_HERE'
  
  declare -A PROJECT_DICT=( \
    ["collab-illumina-dev_workflows"]="dddd6c29-24d3-49f4-91c0-7e818b3c0a21" \
    ["development_workflows"]="0df0356d-3637-48a5-80d1-a924642a6556" \
    ["production_workflows"]="fdd48e11-cdcc-46a9-b5ac-dee3a4c5f19d" \
  )
  
  # Initialise tokens array
  declare -A PROJECT_TOKENS
  
  # Collect personal access token
  personal_access_token="$( \
    curl \
      --silent \
      --location \
      --fail \
      --request POST \
      --url "${ICA_BASE_URL}/v1/tokens" \
      --header "Accept: application/json" \
      --header "X-API-Key: ${API_KEY}" \
      --header "Content-Length: 0" | \
    jq --raw-output \
      '.access_token' \
  )"
  
  # Create token for each project
  for project in "${!PROJECT_DICT[@]}"; do
    # Create Token
    project_id="${PROJECT_DICT["${project}"]}"
    PROJECT_TOKENS+=(["${project}"]="$( \
      curl \
        --fail \
        --silent \
        --location \
        --request "POST" \
        --url "${ICA_BASE_URL}/v1/tokens" \
        --header "Accept: application/json" \
        --header "Content-Length: 0" \
        --header "X-API-Key: ${API_KEY}" \
        --header "Authorization: Bearer ${personal_access_token}" \
        --get \
        --data "cid=${project_id}" \
        --data "scopes=$( \
          jq --null-input --raw-output \
            '
              [
                "TES.RUNS.READ",
                "TES.TASKS.CREATE",
                "TES.TASKS.DELETE",
                "TES.TASKS.GRANT",
                "TES.TASKS.READ",
                "TES.TASKS.UPDATE",
                "TES.VERSIONS.CREATE",
                "TES.VERSIONS.DELETE",
                "TES.VERSIONS.GRANT",
                "TES.VERSIONS.READ",
                "TES.VERSIONS.UPDATE",
                "WES.RUNS.READ",
                "WES.SIGNALS.CREATE",
                "WES.SIGNALS.DELETE",
                "WES.SIGNALS.GRANT",
                "WES.SIGNALS.READ",
                "WES.SIGNALS.UPDATE",
                "WES.VERSIONS.CREATE",
                "WES.VERSIONS.DELETE",
                "WES.VERSIONS.GRANT",
                "WES.VERSIONS.READ",
                "WES.VERSIONS.UPDATE",
                "WES.WORKFLOWS.CREATE",
                "WES.WORKFLOWS.DELETE",
                "WES.WORKFLOWS.GRANT",
                "WES.WORKFLOWS.READ",
                "WES.WORKFLOWS.UPDATE"
              ] |
              join(",")
            ' \
        )" | \
        jq --raw-output \
          '.access_token' \
      )"
    )
  done
  
  # Create object of tokens
  jq --null-input --raw-output \
    --arg production_workflows_token "${PROJECT_TOKENS["production_workflows"]}" \
    --arg development_workflows_token "${PROJECT_TOKENS["development_workflows"]}" \
    --arg collab_illumina_dev_workflows_token "${PROJECT_TOKENS["collab-illumina-dev_workflows"]}" \
    '
      {
        "PRODUCTION_WORKFLOWS": {
          "ICA_ACCESS_TOKEN": $production_workflows_token
        },
        "COLLAB-ILLUMINA-DEV_WORKFLOWS": {
          "ICA_ACCESS_TOKEN": $collab_illumina_dev_workflows_token
        },
        "DEVELOPMENT_WORKFLOWS": {
          "ICA_ACCESS_TOKEN": $development_workflows_token
        }
      } |
      tojson |
      @base64
    '