Chore: Enable PR testing in Drone (grafana#26189)

* Add Drone configuration Signed-off-by: Arve Knudsen <[email protected]> * Add more steps Signed-off-by: Arve Knudsen <[email protected]> * Add more steps Signed-off-by: Arve Knudsen <[email protected]> * Build front-end before testing it Signed-off-by: Arve Knudsen <[email protected]> * Upgrade grafana/build-container Signed-off-by: Arve Knudsen <[email protected]> * Add packaging step Signed-off-by: Arve Knudsen <[email protected]> * Trigger on push Signed-off-by: Arve Knudsen <[email protected]> * Remove some steps Signed-off-by: Arve Knudsen <[email protected]> * Enable steps Signed-off-by: Arve Knudsen <[email protected]> * Install Dockerize Signed-off-by: Arve Knudsen <[email protected]> * Use node image for test-frontend Signed-off-by: Arve Knudsen <[email protected]> * Increase number of test workers Signed-off-by: Arve Knudsen <[email protected]> * Make plugin installation depend on frontend tests Signed-off-by: Arve Knudsen <[email protected]> * Make integration tests depend on frontend tests Signed-off-by: Arve Knudsen <[email protected]> * Use grafana/build-container also for front-end tests Signed-off-by: Arve Knudsen <[email protected]> * Upgrade dependencies in order to fix front-end tests Signed-off-by: Arve Knudsen <[email protected]> * Depend on es-check Signed-off-by: Arve Knudsen <[email protected]> * Dont' depend on tests before building front-end Signed-off-by: Arve Knudsen <[email protected]> * Add more steps Signed-off-by: Arve Knudsen <[email protected]> * Fix packaging Signed-off-by: Arve Knudsen <[email protected]> * Simplify Signed-off-by: Arve Knudsen <[email protected]> * Try to build images Signed-off-by: Arve Knudsen <[email protected]> * Fix e2e tests Signed-off-by: Arve Knudsen <[email protected]> * Remove steps Signed-off-by: Arve Knudsen <[email protected]> * Install netcat Signed-off-by: Arve Knudsen <[email protected]> * Include golangci-lint with grafana/build-container Signed-off-by: Arve Knudsen <[email protected]> * Build storybook and docs website Signed-off-by: Arve Knudsen <[email protected]> * Fix e2e tests Signed-off-by: Arve Knudsen <[email protected]> * Use build image with root user Signed-off-by: Arve Knudsen <[email protected]> * Drop CircleCI dependencies Signed-off-by: Arve Knudsen <[email protected]> * Fix e2e tests Signed-off-by: Arve Knudsen <[email protected]> * Fix e2e under Drone Signed-off-by: Arve Knudsen <[email protected]> * Execute e2e server separately Signed-off-by: Arve Knudsen <[email protected]> * Use own plugin for building Docker images Signed-off-by: Arve Knudsen <[email protected]> * Use Starlark to configure Drone Signed-off-by: Arve Knudsen <[email protected]> * Add enterprise steps to pipeline Signed-off-by: Arve Knudsen <[email protected]> * Add more enterprise steps to pipeline Signed-off-by: Arve Knudsen <[email protected]> * Maintain Yarn cache Signed-off-by: Arve Knudsen <[email protected]> * Build enterprise Docker images Signed-off-by: Arve Knudsen <[email protected]> * Build Ubuntu Docker images Signed-off-by: Arve Knudsen <[email protected]> * Refactor Signed-off-by: Arve Knudsen <[email protected]> * Add Postgres integration test Signed-off-by: Arve Knudsen <[email protected]> * Add MySQL integration test Signed-off-by: Arve Knudsen <[email protected]> * Fix integration tests Signed-off-by: Arve Knudsen <[email protected]> * Parameterize integration test DB connections Signed-off-by: Arve Knudsen <[email protected]> * Categorize integration tests Signed-off-by: Arve Knudsen <[email protected]> * Use grabpl integration-tests Signed-off-by: Arve Knudsen <[email protected]> * Remove unintended change Signed-off-by: Arve Knudsen <[email protected]> * Drone: Disable Ubuntu Docker images for PR pipeline Signed-off-by: Arve Knudsen <[email protected]> * Regenerate yarn.lock Signed-off-by: Arve Knudsen <[email protected]> * Upgrade grabpl Signed-off-by: Arve Knudsen <[email protected]> * Restore Yarn cache before installing in grafana-enterprise Signed-off-by: Arve Knudsen <[email protected]> * Use separate pipelines for OSS and enterprise Signed-off-by: Arve Knudsen <[email protected]> * Let OSS builds depend on tests Signed-off-by: Arve Knudsen <[email protected]> * Restore Go cache before building back-end Signed-off-by: Arve Knudsen <[email protected]> * Reduce number of variants built for PRs Signed-off-by: Arve Knudsen <[email protected]> * Fix building of Docker images Signed-off-by: Arve Knudsen <[email protected]> * Drone: Simplify logic Signed-off-by: Arve Knudsen <[email protected]> * Drone: Use Starlark Signed-off-by: Arve Knudsen <[email protected]> * Drone: Fix syntax error Signed-off-by: Arve Knudsen <[email protected]> * Convert .drone.star to YAML Signed-off-by: Arve Knudsen <[email protected]> * Upgrade AWS Go SDK Signed-off-by: Arve Knudsen <[email protected]> * Drone: Fix Go linting Signed-off-by: Arve Knudsen <[email protected]> * Undo irrelevant changes Signed-off-by: Arve Knudsen <[email protected]> * Revert "Undo irrelevant changes" This reverts commit 5152f65. * Undo irrelevant changes Signed-off-by: Arve Knudsen <[email protected]> * e2e: Support Circle Signed-off-by: Arve Knudsen <[email protected]> * Remove unused script Signed-off-by: Arve Knudsen <[email protected]> * TypeScript fixes Signed-off-by: Arve Knudsen <[email protected]> * TypeScript fixes Signed-off-by: Arve Knudsen <[email protected]> * Remove unused script Signed-off-by: Arve Knudsen <[email protected]> * More Drone support Signed-off-by: Arve Knudsen <[email protected]> * Remove unused script Signed-off-by: Arve Knudsen <[email protected]> * Fix build on Circle Signed-off-by: Arve Knudsen <[email protected]> * Remove TODO comment Signed-off-by: Arve Knudsen <[email protected]>
james-deee · Jul 10, 2020 · d28d495 · d28d495
1 parent 1391575
commit d28d495
Show file tree

Hide file tree

Showing 25 changed files with 1,009 additions and 206 deletions.
diff --git a/.drone.star b/.drone.star
diff --git a/.drone.yml b/.drone.yml
@@ -0,0 +1,351 @@
+---
+kind: pipeline
+type: docker
+name: test-pr-oss
+
+platform:
+  os: linux
+  arch: amd64
+
+steps:
+- name: install-deps
+  image: grafana/build-container:1.2.21
+  commands:
+  - curl -fLO https://grafana-downloads.storage.googleapis.com/grafana-build-pipeline/v$${GRABPL_VERSION}/grabpl
+  - chmod +x grabpl
+  - mkdir -p bin
+  - mv grabpl bin
+  - curl -fLO https://github.com/jwilder/dockerize/releases/download/v$${DOCKERIZE_VERSION}/dockerize-linux-amd64-v$${DOCKERIZE_VERSION}.tar.gz
+  - tar -C bin -xzvf dockerize-linux-amd64-v$${DOCKERIZE_VERSION}.tar.gz
+  - rm dockerize-linux-amd64-v$${DOCKERIZE_VERSION}.tar.gz
+  - yarn install --frozen-lockfile --no-progress
+  - cp -r $(yarn cache dir) yarn-cache
+  environment:
+    DOCKERIZE_VERSION: 0.6.1
+    GRABPL_VERSION: 0.4.15
+
+- name: lint-go
+  image: grafana/build-container:1.2.21
+  commands:
+  - make lint-go
+  environment:
+    CGO_ENABLED: 1
+  depends_on:
+  - install-deps
+
+- name: codespell
+  image: grafana/build-container:1.2.21
+  commands:
+  - "echo -e \"unknwon\nreferer\nerrorstring\neror\niam\" > words_to_ignore.txt"
+  - codespell -I words_to_ignore.txt docs/
+  depends_on:
+  - install-deps
+
+- name: shellcheck
+  image: grafana/build-container:1.2.21
+  commands:
+  - curl -fLO http://storage.googleapis.com/grafana-downloads/ci-dependencies/shellcheck-v$${VERSION}.linux.x86_64.tar.xz
+  - echo $$CHKSUM shellcheck-v$${VERSION}.linux.x86_64.tar.xz | sha512sum --check --strict --status
+  - tar xf shellcheck-v$${VERSION}.linux.x86_64.tar.xz
+  - mv shellcheck-v$${VERSION}/shellcheck /usr/local/bin/
+  - rm -rf shellcheck-v$${VERSION}*
+  - ./bin/grabpl shellcheck
+  environment:
+    CHKSUM: beca3d7819a6bdcfbd044576df4fc284053b48f468b2f03428fe66f4ceb2c05d9b5411357fa15003cb0311406c255084cf7283a3b8fce644c340c2f6aa910b9f
+    VERSION: 0.7.1
+  depends_on:
+  - install-deps
+
+- name: build-backend
+  image: grafana/build-container:1.2.21
+  commands:
+  - rm -rf $(go env GOCACHE) && cp -r go-cache $(go env GOCACHE)
+  - ./bin/grabpl build-backend --github-token "$${GITHUB_TOKEN}" --edition oss --build-id $DRONE_BUILD_NUMBER --variants linux-x64,linux-x64-musl,osx64,win64
+  environment:
+    GITHUB_TOKEN:
+      from_secret: github_token
+  depends_on:
+  - install-deps
+  - lint-go
+  - test-backend
+
+- name: build-frontend
+  image: grafana/build-container:1.2.21
+  commands:
+  - rm -rf $(yarn cache dir) && cp -r yarn-cache $(yarn cache dir)
+  - ./bin/grabpl build-frontend --no-install-deps --github-token "$${GITHUB_TOKEN}" --edition oss --build-id $DRONE_BUILD_NUMBER
+  environment:
+    GITHUB_TOKEN:
+      from_secret: github_token
+  depends_on:
+  - install-deps
+  - test-frontend
+
+- name: test-backend
+  image: grafana/build-container:1.2.21
+  commands:
+  - go test -covermode=atomic ./pkg/...
+  - ./bin/grabpl integration-tests
+  - cp -r $(go env GOCACHE) go-cache
+  depends_on:
+  - install-deps
+  - lint-go
+
+- name: test-frontend
+  image: grafana/build-container:1.2.21
+  commands:
+  - rm -rf $(yarn cache dir) && cp -r yarn-cache $(yarn cache dir)
+  - yarn run prettier:check
+  - yarn run packages:typecheck
+  - yarn run typecheck
+  - yarn run test
+  depends_on:
+  - install-deps
+
+- name: build-plugins
+  image: grafana/build-container:1.2.21
+  commands:
+  - rm -rf $(yarn cache dir) && cp -r yarn-cache $(yarn cache dir)
+  - ./bin/grabpl build-plugins --edition oss --no-install-deps
+  depends_on:
+  - install-deps
+  - lint-go
+
+- name: package
+  image: grafana/build-container:1.2.21
+  commands:
+  - . scripts/build/gpg-test-vars.sh && ./bin/grabpl package --github-token "$${GITHUB_TOKEN}" --edition oss --build-id $DRONE_BUILD_NUMBER --variants linux-x64,linux-x64-musl,osx64,win64
+  environment:
+    GITHUB_TOKEN:
+      from_secret: github_token
+  depends_on:
+  - build-backend
+  - build-frontend
+  - build-plugins
+  - test-backend
+  - test-frontend
+  - codespell
+  - shellcheck
+
+- name: end-to-end-tests-server
+  image: grafana/build-container:1.2.21
+  detach: true
+  commands:
+  - ./e2e/start-server
+  depends_on:
+  - package
+
+- name: end-to-end-tests
+  image: grafana/ci-e2e:12.18-1
+  commands:
+  - rm -rf $(yarn cache dir) && cp -r yarn-cache $(yarn cache dir)
+  - ./node_modules/.bin/cypress install
+  - ./e2e/wait-for-grafana
+  - ./e2e/run-suite
+  environment:
+    HOST: end-to-end-tests-server
+  depends_on:
+  - end-to-end-tests-server
+
+- name: publish-storybook
+  image: grafana/build-container:1.2.21
+  commands:
+  - rm -rf $(yarn cache dir) && cp -r yarn-cache $(yarn cache dir)
+  - yarn storybook:build
+  depends_on:
+  - package
+
+- name: build-docs-website
+  image: grafana/docs-base:latest
+  commands:
+  - mkdir -p /hugo/content/docs/grafana
+  - cp -r docs/sources /hugo/content/docs/grafana/latest
+  - cd /hugo && make prod
+
+- name: copy-packages-for-docker
+  image: grafana/build-container:1.2.21
+  commands:
+  - cp dist/*.tar.gz packaging/docker/
+  depends_on:
+  - package
+
+- name: build-docker-images
+  image: grafana/drone-grafana-docker:0.2.0
+  settings:
+    archs: amd64
+    dry_run: true
+    edition: oss
+  depends_on:
+  - copy-packages-for-docker
+
+- name: postgres-integration-test
+  image: grafana/build-container:1.2.21
+  commands:
+  - apt-get update
+  - apt-get install -yq postgresql-client
+  - ./bin/dockerize -wait tcp://postgres:5432 -timeout 120s
+  - psql -p 5432 -h postgres -U grafanatest -d grafanatest -f devenv/docker/blocks/postgres_tests/setup.sql
+  - rm -rf $(go env GOCACHE) && cp -r go-cache $(go env GOCACHE)
+  - go clean -testcache
+  - ./bin/grabpl integration-tests --database postgres
+  environment:
+    GRAFANA_TEST_DB: postgres
+    PGPASSWORD: grafanatest
+    POSTGRES_HOST: postgres
+  depends_on:
+  - test-backend
+  - test-frontend
+
+- name: mysql-integration-test
+  image: grafana/build-container:1.2.21
+  commands:
+  - apt-get update
+  - apt-get install -yq default-mysql-client
+  - ./bin/dockerize -wait tcp://mysql:3306 -timeout 120s
+  - cat devenv/docker/blocks/mysql_tests/setup.sql | mysql -h mysql -P 3306 -u root -prootpass
+  - rm -rf $(go env GOCACHE) && cp -r go-cache $(go env GOCACHE)
+  - go clean -testcache
+  - ./bin/grabpl integration-tests --database mysql
+  environment:
+    GRAFANA_TEST_DB: mysql
+    MYSQL_HOST: mysql
+  depends_on:
+  - test-backend
+  - test-frontend
+
+services:
+- name: postgres
+  image: postgres:12.3-alpine
+  environment:
+    POSTGRES_DB: grafanatest
+    POSTGRES_PASSWORD: grafanatest
+    POSTGRES_USER: grafanatest
+
+- name: mysql
+  image: mysql:5.6.48
+  environment:
+    MYSQL_DATABASE: grafana_tests
+    MYSQL_PASSWORD: password
+    MYSQL_ROOT_PASSWORD: rootpass
+    MYSQL_USER: grafana
+
+trigger:
+  event:
+  - pull_request
+
+---
+kind: pipeline
+type: docker
+name: test-pr-enterprise
+
+platform:
+  os: linux
+  arch: amd64
+
+steps:
+- name: install-deps
+  image: grafana/build-container:1.2.21
+  commands:
+  - curl -fLO https://grafana-downloads.storage.googleapis.com/grafana-build-pipeline/v$${GRABPL_VERSION}/grabpl
+  - chmod +x grabpl
+  - mkdir -p bin
+  - mv grabpl bin
+  - curl -fLO https://github.com/jwilder/dockerize/releases/download/v$${DOCKERIZE_VERSION}/dockerize-linux-amd64-v$${DOCKERIZE_VERSION}.tar.gz
+  - tar -C bin -xzvf dockerize-linux-amd64-v$${DOCKERIZE_VERSION}.tar.gz
+  - rm dockerize-linux-amd64-v$${DOCKERIZE_VERSION}.tar.gz
+  - yarn install --frozen-lockfile --no-progress
+  - cp -r $(yarn cache dir) yarn-cache
+  environment:
+    DOCKERIZE_VERSION: 0.6.1
+    GRABPL_VERSION: 0.4.15
+  when:
+    repo:
+    - grafana/grafana
+    - aknuds1/grafana
+
+- name: build-backend
+  image: grafana/build-container:1.2.21
+  commands:
+  - ./bin/grabpl build-backend --github-token "$${GITHUB_TOKEN}" --edition enterprise --build-id $DRONE_BUILD_NUMBER --variants linux-x64,linux-x64-musl,osx64,win64
+  environment:
+    GITHUB_TOKEN:
+      from_secret: github_token
+  when:
+    repo:
+    - grafana/grafana
+    - aknuds1/grafana
+  depends_on:
+  - install-deps
+
+- name: build-frontend
+  image: grafana/build-container:1.2.21
+  commands:
+  - rm -rf $(yarn cache dir) && cp -r yarn-cache $(yarn cache dir)
+  - ./bin/grabpl build-frontend --no-install-deps --github-token "$${GITHUB_TOKEN}" --edition enterprise --build-id $DRONE_BUILD_NUMBER
+  environment:
+    GITHUB_TOKEN:
+      from_secret: github_token
+  when:
+    repo:
+    - grafana/grafana
+    - aknuds1/grafana
+  depends_on:
+  - install-deps
+
+- name: build-plugins
+  image: grafana/build-container:1.2.21
+  commands:
+  - rm -rf $(yarn cache dir) && cp -r yarn-cache $(yarn cache dir)
+  - ./bin/grabpl build-plugins --edition enterprise --no-install-deps
+  when:
+    repo:
+    - grafana/grafana
+    - aknuds1/grafana
+  depends_on:
+  - install-deps
+
+- name: package
+  image: grafana/build-container:1.2.21
+  commands:
+  - . scripts/build/gpg-test-vars.sh && ./bin/grabpl package --github-token "$${GITHUB_TOKEN}" --edition enterprise --build-id $DRONE_BUILD_NUMBER --variants linux-x64,linux-x64-musl,osx64,win64
+  environment:
+    GITHUB_TOKEN:
+      from_secret: github_token
+  when:
+    repo:
+    - grafana/grafana
+    - aknuds1/grafana
+  depends_on:
+  - build-backend
+  - build-frontend
+  - build-plugins
+
+- name: copy-packages-for-docker
+  image: grafana/build-container:1.2.21
+  commands:
+  - cp dist/*.tar.gz packaging/docker/
+  when:
+    repo:
+    - grafana/grafana
+    - aknuds1/grafana
+  depends_on:
+  - package
+
+- name: build-docker-images
+  image: grafana/drone-grafana-docker:0.2.0
+  settings:
+    archs: amd64
+    dry_run: true
+    edition: enterprise
+  when:
+    repo:
+    - grafana/grafana
+    - aknuds1/grafana
+  depends_on:
+  - copy-packages-for-docker
+
+trigger:
+  event:
+  - pull_request
+
+...
diff --git a/.editorconfig b/.editorconfig
@@ -25,3 +25,6 @@ trim_trailing_whitespace = false
 [Makefile]
 indent_style = tab
 indent_size = 2
+
+[*.star]
+indent_size = 4
diff --git a/e2e/run-suite b/e2e/run-suite
@@ -1,13 +1,17 @@
 #!/bin/bash
+set -xeo pipefail
 
 . e2e/variables
 
+HOST=${HOST:-$DEFAULT_HOST}
+PORT=${PORT:-$DEFAULT_PORT}
+
 echo -e "Starting Cypress scenarios"
 
 CMD="start"
 PARAMS=""
 SLOWMO=0
-URL=${BASE_URL:-"http://$DEFAULT_HOST:$DEFAULT_PORT"}
+URL=${BASE_URL:-"http://$HOST:$PORT"}
 SUITE=${SUITE:-$DEFAULT_SUITE}
 
 if [ "$1" == "debug" ]; then
@@ -23,6 +27,6 @@ fi
 
 cd packages/grafana-e2e
 
-yarn $CMD --env BASE_URL=$URL,CIRCLE_SHA1=$CIRCLE_SHA1,SLOWMO=$SLOWMO \
+yarn $CMD --env BASE_URL=$URL,SLOWMO=$SLOWMO \
   --config integrationFolder=../../e2e/$SUITE/specs,screenshotsFolder=../../e2e/$SUITE/screenshots,videosFolder=../../e2e/$SUITE/videos,fileServerFolder=./cypress,viewportWidth=1920,viewportHeight=1080,trashAssetsBeforeRuns=false \
   $PARAMS
diff --git a/package.json b/package.json
@@ -13,7 +13,7 @@
     "e2e:debug": "./e2e/start-and-run-suite debug",
     "e2e:dev": "./e2e/start-and-run-suite dev",
     "jest": "jest --notify --watch",
-    "jest-ci": "mkdir -p reports/junit && export JEST_JUNIT_OUTPUT_DIR=reports/junit && jest --ci --reporters=default --reporters=jest-junit --maxWorkers 2",
+    "jest-ci": "mkdir -p reports/junit && export JEST_JUNIT_OUTPUT_DIR=reports/junit && jest --ci --reporters=default --reporters=jest-junit -w ${TEST_MAX_WORKERS:-100%}",
     "lint": "eslint public/app e2e/suite1 public/test --ext=.js,.ts,.tsx",
     "lint:fix": "yarn lint --fix",
     "packages:build": "lerna run clean && lerna run build --ignore @grafana-plugins/input-datasource",