BOSH CPI Certification

This repository contains additional tests above and beyond unit and integration tests. This is meant to complement the existing tests, not to replace.

• What are we testing?
  • BOSH CPI releases
• Why are we testing?
  • To provide a high level of confidence that new releases of BOSH, BOSH stemcells, and BOSH CPIs work together properly
• When will the certification pipeline run?
  • Whenever any of the following are released: (trigger: true)
    1. bosh-release
    2. bosh-<iaas>-cpi-release
    3. stemcell
    4. Once a day.
• How are we testing?
  • Testing for certification consists of the following test scenarios:
    1. BATs are run for every flavor of stemcells (ubuntu-trusty and centos-7)
    2. A BOSH director upgrade test is run from a previous version family stemcell (e.g. 3363.x to 3421.latest)
    3. Specific IaaS end-2-end tests, if necessary

BATs - BOSH Acceptance Tests

0. Set up infrastructure environment from scratch. We use Terraform resource whenever and wherever we can. It works beautifully.
1. Deploy BOSH director with latest bosh-release, IaaS-specific cpi-release, and stemcell
2. Run BATs
3. Tear down BOSH director
4. Tear down infrastructure environment

BOSH Director Upgrade Test

0. Set up infrastructure environment from scratch, we use Terraform resource when and wherever we can. It works beautifully.
1. Deploy BOSH director with old bosh-release (255.4), IaaS-specific cpi-release (X), and stemcell (3363.x)
2. Deploy certification-release
3. Redeploy bosh director with latest bosh-release, IaaS-specific, cpi-release, and stemcell
4. Redeploy certification-release with --recreate
5. Tear down BOSH director
6. Tear down infrastructure environment

The folder structure looks like this:

.
├── iaas
    ├── bats
    |   └── bats-spec.yml
    ├── certification
    |   └── cloud-config-ops.yml
    ├── ops
    |   └── custom-cpi-release.yml
    └── pipeline.yml

iaas/bats/bats-spec.yml: Manifest with all IaaS-specific configurations for BATs. Make a PR to add the IaaS-specific template to the BATs template folder.

iaas/certification/cloud-config-ops.yml: Ops file to add IaaS-specific properties to the certification cloud-config

iaas/ops/custom-cpi-release.yml: Used by the certification pipeline to upload a specific BOSH CPI release version. See example below:

- type: replace
  path: /releases/name=bosh-<iaas>-cpi
  value:
    name: bosh-<iaas>-cpi
    url: ((cpi_release_uri))

iaas/pipeline.yml: Your Concourse certification pipeline YAML configuration. See example.

BOSH CPIs

• vSphere
• AWS
• AlibabaCloud
• vCloud
• GCP
• OpenStack
• Azure
• SoftLayer
• RackHD
• Photon

Others:

• VirtualBox
• Docker
• Warden