Initial open source release of Chef Automate

Chef Automate provides a unified view into infrastructure managed
by Chef, Inspec, and Habitat.

* How do I use Chef Automate?

To get started using Chef Automate, please see the official
documentation at

https://automate.chef.io/

* How do I contribute to Chef Automate?

You can contribute to Chef Automate by filing bugs or requesting
new features important to you via GitHub, improving
documentation, and contributing code.

To contribute code, the best place to get started is via our
Habitat-based development environment. How to use that
environment is described in `dev-docs/DEV_ENVIRONMENT.md`.

* What about the git history?

We have opted to truncate the git history of this project. The
git history has been preserved and if you find yourself needing
more context for a particular part of the code, please don't
hesitate to open a github issue and ask. We are sorry for any
friction this causes in the near term.

* What are the licensing terms?

Unless otherwise noted, all code in the repository is offered
under version 2.0 of the Apache License. The full text of the
license is in `LICENSE`

Co-Authored-By: Adam Jacob <[email protected]>
Co-Authored-By: Alex Pop <[email protected]>
Co-Authored-By: baumanj <[email protected]>
Co-Authored-By: Blake Johnson <[email protected]>
Co-Authored-By: Brandy Austin <[email protected]>
Co-Authored-By: Brenna Darroch <[email protected]>
Co-Authored-By: Bryan McLellan <[email protected]>
Co-Authored-By: Christian Nunciato <[email protected]>
Co-Authored-By: Christoph Hartmann <[email protected]>
Co-Authored-By: Christopher Webber <[email protected]>
Co-Authored-By: Dan DeLeo <[email protected]>
Co-Authored-By: David McCown <[email protected]>
Co-Authored-By: David Nelson <[email protected]>
Co-Authored-By: Davida Marion <[email protected]>
Co-Authored-By: Davy McAleer <[email protected]>
Co-Authored-By: Dominik Richter <[email protected]>
Co-Authored-By: Elizabeth Uselton <[email protected]>
Co-Authored-By: Elliott Davis <[email protected]>
Co-Authored-By: gpeers <[email protected]>
Co-Authored-By: hannah-radish <[email protected]>
Co-Authored-By: Irving Popovetsky <[email protected]>
Co-Authored-By: JJ Asghar <[email protected]>
Co-Authored-By: Jay Mundrawala <[email protected]>
Co-Authored-By: Jeremiah Snapp <[email protected]>
Co-Authored-By: Jeremy Miller <[email protected]>
Co-Authored-By: jmink <[email protected]>
Co-Authored-By: Jon Morrow <[email protected]>
Co-Authored-By: Joshua Padgett <[email protected]>
Co-Authored-By: Julian C. Dunn <[email protected]>
Co-Authored-By: Kartik Null Cating-Subramanian <[email protected]>
Co-Authored-By: Kevin Reedy <[email protected]>
Co-Authored-By: Kimberly Garmoe <[email protected]>
Co-Authored-By: Kyleen <[email protected]>
Co-Authored-By: Lance Finfrock <[email protected]>
Co-Authored-By: Maggie Walker <[email protected]>
Co-Authored-By: Marc A. Paradise <[email protected]>
Co-Authored-By: Mark Anderson <[email protected]>
Co-Authored-By: Mark Mzyk <[email protected]>
Co-Authored-By: Martin Scott <[email protected]>
Co-Authored-By: Matthew Peck <[email protected]>
Co-Authored-By: Michael Sorens <[email protected]>
Co-Authored-By: Mike Krasnow <[email protected]>
Co-Authored-By: mjingle <[email protected]>
Co-Authored-By: Nathan L Smith <[email protected]>
Co-Authored-By: Nick Brewer <[email protected]>
Co-Authored-By: Nolan Davidson <[email protected]>
Co-Authored-By: Paul Welch <[email protected]>
Co-Authored-By: Pete Higgins <[email protected]>
Co-Authored-By: Prajakta Purohit <[email protected]>
Co-Authored-By: Rachel Adler <[email protected]>
Co-Authored-By: Rick Marry <[email protected]>
Co-Authored-By: Ryan Cragun <[email protected]>
Co-Authored-By: Ryan Hass <[email protected]>
Co-Authored-By: Ryan Keairns <[email protected]>
Co-Authored-By: Salim Afiune <[email protected]>
Co-Authored-By: Scott Christopherson <[email protected]>
Co-Authored-By: Scott Hain <[email protected]>
Co-Authored-By: Seth Chisamore <[email protected]>
Co-Authored-By: Seth Falcon <[email protected]>
Co-Authored-By: Shadae Holmes <[email protected]>
Co-Authored-By: Shanku Niyogi <[email protected]>
Co-Authored-By: sjvreddy <[email protected]>
Co-Authored-By: Stephan Renatus <[email protected]>
Co-Authored-By: Stephen Delano <[email protected]>
Co-Authored-By: Steven Danna <[email protected]>
Co-Authored-By: susan evans <[email protected]>
Co-Authored-By: Thomas Cate <[email protected]>
Co-Authored-By: Tim Flechtner <[email protected]>
Co-Authored-By: Tom Duffield <[email protected]>
Co-Authored-By: Trevor Bramble <[email protected]>
Co-Authored-By: Tyler Ball <[email protected]>
Co-Authored-By: Tyler Cloke <[email protected]>
Co-Authored-By: Victoria Jeffrey <[email protected]>
Co-Authored-By: Yvonne Lam <[email protected]>

Signed-off-by: Steven Danna <[email protected]>

.buildkite/hooks/pre-command

@@ -0,0 +1,22 @@
+#!/bin/bash
+
+set -eu
+
+# We've now seen cases where origin/master on the build hosts can get
+# out of date. This causes us to build components unnecessarily.
+# Fetching it here hopefully will prevent this situation.
+echo "Fetching origin/master"
+git fetch origin master
+
+
+# By default, Buildkite pulls down HEAD. If we're on a pull-request, pull down
+# the merged head: https://github.com/buildkite/agent/blob/master/bootstrap/bootstrap.go#L698
+if [[ "$BUILDKITE_PULL_REQUEST" != "false" ]]; then
+  echo "Switching to refspec 'refs/pull/$BUILDKITE_PULL_REQUEST/merge'"
+  git fetch origin +refs/pull/$BUILDKITE_PULL_REQUEST/merge
+  git checkout -qf FETCH_HEAD
+fi
+
+# Count retries as BK annotations; don't make all jobs explode if the script
+# is removed.
+[[ -x "scripts/count_retries" ]] && scripts/count_retries

.buildkite/hooks/pre-exit

@@ -0,0 +1,11 @@
+#!/bin/bash
+
+set -eu
+
+echo "Attempting to clean up docker containers for buildkitejob=$BUILDKITE_JOB_ID"
+
+matching_containers=$(docker ps --filter "label=buildkitejob=$BUILDKITE_JOB_ID" --format "{{.ID}}")
+if [ -n "$matching_containers" ]; then
+    echo "Killing docker containers $matching_containers"
+    docker kill $matching_containers
+fi

.envrc

@@ -0,0 +1,40 @@
+# Make sure that we have specified the HAB_STUDIO_SECRET_GITHUB_TOKEN variable
+if [[ "x$HAB_STUDIO_SECRET_GITHUB_TOKEN" == "x" ]]; then
+  echo -e "\n\nERROR: Please update your shell environment and set the HAB_STUDIO_SECRET_GITHUB_TOKEN environment variable."
+  echo -e "If you don't have one, you can create a token at https://github.com/settings/tokens. It should have all the scopes under repo.\n\n"
+fi
+
+# Add extra ports you need for development. (ex. 9200 for elasticsearch)
+ports_to_export=(
+  2000 # automate-gateway
+  443  # automate-load-balancer
+  8989 # workflow git server
+  2345 # Delve Server (Go debugger)
+  10141 # Elasticsearch port
+  10140 # NATS internal-to-a2 port
+  4222 # NATS external facing port
+  10109 # automate-prometheus
+)
+for p in "${ports_to_export[@]}"
+do
+  [[ $HAB_DOCKER_OPTS != "" ]] && HAB_DOCKER_OPTS=" ${HAB_DOCKER_OPTS}"
+  HAB_DOCKER_OPTS="-p ${p}:${p}${HAB_DOCKER_OPTS}"
+done
+export HAB_DOCKER_OPTS
+
+if [[ -f ".dev/secrets-env.sh" ]]; then
+   source ".dev/secrets-env.sh"
+fi
+
+# Avoid starting the supervisor automatically
+# We will be injecting the config via the .studiorc
+export HAB_STUDIO_SUP=false
+
+# Read in aws creds from environment
+export HAB_STUDIO_SECRET_AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}"
+export HAB_STUDIO_SECRET_AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}"
+
+# Read in azure creds from environment
+export HAB_STUDIO_SECRET_AZURE_CLIENT_ID="${AZURE_CLIENT_ID}"
+export HAB_STUDIO_SECRET_AZURE_CLIENT_SECRET="${AZURE_CLIENT_SECRET}"
+export HAB_STUDIO_SECRET_AZURE_TENANT_ID="${AZURE_TENANT_ID}"

.expeditor/a1-migration.pipeline.yml

@@ -0,0 +1,25 @@
+steps:
+
+  #
+  # A1 -> A2 migration test
+  #
+
+  #
+  # Run this on buildkite via https://buildkite.com/chef/chef-slash-a2-master-a1-migration-slash-dev
+  #
+
+  - label: A1 -> A2 migration test
+    command:
+      - scripts/nightly_migration.sh
+    env:
+      HAB_ORIGIN: chef
+    timeout_in_minutes: 30
+    expeditor:
+      secrets:
+        A1_LICENSE:
+          path: secret/a2/delivery_license
+          field: license
+      executor:
+        linux:
+          single-use: true
+          privileged: true

.expeditor/announce-release.sh

@@ -0,0 +1,43 @@
+#!/bin/bash
+
+set -eou pipefail
+
+# Download the manifest
+curl -o manifest.json "https://packages.chef.io/manifests/${EXPEDITOR_TARGET_CHANNEL}/automate/latest.json"
+# Extract the build version from the manifest
+build_version=$(jq -r -c ".build"  manifest.json)
+# Download the release-notes for our specific build
+curl -o release-notes.md "https://packages.chef.io/release-notes/automate/${build_version}.md"
+
+topic_title="Automate 2 version $build_version Released!"
+topic_body=$(cat <<EOH
+We are delighted to announce the availability of version $build_version of Chef Automate 2.
+
+$(cat release-notes.md)
+
+---
+## How to Upgrade
+
+By default Chef Automate 2 will [automatically upgrade](https://automate.chef.io/docs/install/#upgrades) to this new version. If you have disabled automatic upgrades you can manually initiate an upgrade by running:
+
+\`\`\`
+chef-automate upgrade run
+\`\`\`
+
+As always, we welcome your feedback and invite you to contact us directly or share your [feedback online](https://www.chef.io/feedback/). Thanks for using Chef Automate 2!
+EOH
+)
+
+# category 9 is "Chef Release Announcements": https://discourse.chef.io/c/chef-release
+
+curl -X POST https://discourse.chef.io/posts \
+  -H "Content-Type: multipart/form-data" \
+  -F "api_username=chef-ci" \
+  -F "api_key=$DISCOURSE_API_TOKEN" \
+  -F "category=9" \
+  -F "title=$topic_title" \
+  -F "raw=$topic_body"
+
+# Cleanup
+rm manifest.json
+rm release-notes.md

.expeditor/basic-a1.pipeline.yml

@@ -0,0 +1,25 @@
+steps:
+
+  #
+  # basic A1 -> A2 migration (chef-server on one node, A1 on a second node)
+  #
+
+  #
+  # Run this on buildkite via https://buildkite.com/chef/chef-slash-a2-master-a2-basic-a1-slash-dev
+  #
+
+  - label: basic A1 -> A2 migration test
+    command:
+      - scripts/nightly_basic_a1.sh
+    env:
+      HAB_ORIGIN: chef
+    timeout_in_minutes: 30
+    expeditor:
+      secrets:
+        A1_LICENSE:
+          path: secret/a2/delivery_license
+          field: license
+      executor:
+        linux:
+          single-use: true
+          privileged: true

.expeditor/buildkite/README.md

@@ -0,0 +1 @@
+These files are in the "buildkite" folder because they will likely be extracted out as Buildkite plugin built-ins in the future.

.expeditor/buildkite/automate-chef-io-deploy.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -euo pipefail
+
+export CHANNEL="${EXPEDITOR_TARGET_CHANNEL:-dev}"
+
+cd terraform/automate-chef-io-deploy
+make apply

.expeditor/buildkite/cypress.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+set -euo pipefail
+
+cd /workdir/e2e
+
+instances_to_test=( "a2-iamv2-local-fresh-install-${CHANNEL}.cd.chef.co" "a2-iamv2-local-inplace-upgrade-${CHANNEL}.cd.chef.co" )
+
+for instance in ${instances_to_test[*]}
+do
+  echo "--- Executing Cypress tests against $instance"
+  export CYPRESS_BASE_URL="https://$instance"
+  export CYPRESS_RECORD_KEY="$CYPRESS_RECORD_KEY"
+
+  cypress run --record
+done

.expeditor/buildkite/hugo_lint.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+
+set -euo pipefail
+
+LINT_STATUS="$(grep -r -I --color=auto -o --with-filename -n -P '[^\x00-\x7F]' ./components/automate-chef-io/content/docs &> /dev/null ; echo $?)"
+
+if [ "$LINT_STATUS" == 1 ]; then
+  echo "Success!"
+  exit 0
+else
+  echo "Failure!"
+  grep -r -I --color=auto -o --with-filename -n -P '[^\x00-\x7F]' ./components/automate-chef-io/content/docs
+  if [ "$LINT_STATUS" == 0 ]; then
+    exit 1
+  else
+    exit $LINT_STATUS
+  fi
+fi

.expeditor/buildkite/inspec.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+set -euo pipefail
+
+echo -e "$CHEF_CI_SSH_PRIVATE_KEY" > chef-ci-ad-ssh
+
+instances_to_test=$(curl --silent https://a2-${CHANNEL}.cd.chef.co/assets/data.json | jq --raw-output 'map(select(.tags[] | contains ("chef-automate-cli"))) | .[] .fqdn')
+
+for instance in ${instances_to_test[*]}
+do
+  echo "--- Executing a2-deploy-smoke profile against $instance"
+
+  cat >./attrs.yml <<EOH
+---
+target_host: $instance
+EOH
+
+  inspec exec inspec/a2-deploy-smoke --sudo --target ssh://chef-ci@$instance -i chef-ci-ad-ssh --attrs attrs.yml
+done

.expeditor/buildkite/terraform.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+
+set -euo pipefail
+
+# This block translates the "channel" into the appropriate set of VPC settings
+# used in terraform/Makefile. These settings still rely on old Workflow-isms,
+# which is why the TF_ENVIRONMENT name doesn't match up with the CHANNEL.
+if [ "$CHANNEL" == "unstable" ]; then
+  export TF_ENVIRONMENT=acceptance
+elif [ "$CHANNEL" == "dev" ]; then
+  export TF_ENVIRONMENT=union
+elif [ "$CHANNEL" == "acceptance" ]; then
+  export TF_ENVIRONMENT=delivered
+else
+  puts "We do not currently support deploying channel $CHANNEL"
+  exit 1
+fi
+
+export LOGS_URL="$BUILDKITE_BUILD_URL"
+
+cd terraform/test-environments
+make deploy

.expeditor/buildkite/test-license-usage.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+
+set -euo pipefail
+
+# This block translates the "channel" into the appropriate set of VPC settings
+# used in terraform/Makefile. These settings still rely on old Workflow-isms,
+# which is why the TF_ENVIRONMENT name doesn't match up with the CHANNEL.
+if [ "$CHANNEL" == "unstable" ]; then
+  export TF_ENVIRONMENT=acceptance
+elif [ "$CHANNEL" == "dev" ]; then
+  export TF_ENVIRONMENT=union
+elif [ "$CHANNEL" == "acceptance" ]; then
+  export TF_ENVIRONMENT=delivered
+else
+  puts "We do not currently support deploying channel $CHANNEL"
+  exit 1
+fi
+
+export LOGS_URL="$BUILDKITE_BUILD_URL"
+
+cd terraform/test-license-usage
+make apply