Skip to content

Commit

Permalink
Feature/create command (#97)
Browse files Browse the repository at this point in the history
* feat: infra code heavily WIP

* feat: infra phase

* fix: after rebase

* chore: refactoring after rebase

* fix: added license banner

* chore: refactoring on eks cluster create cmd

* fix: missing folder creation

* fix: linting

* feat: added dryrun

* feat: moved furyfile inside struct in eks/clusters

chore: removed deprecated func calls to ioutil Read/WriteFile

* fix: post rebase

* chore: changed order of download/validate

* chore: first step of the distribution config rework

* chore: update deps

* feat: add config structs validation

* fix: tests after introducing validation

* chore: refactor pre-create cluster code

* chore: refactor 'create cluster' logic to remove the need to use other app services

* chore: rename variable

* feat: first draft of kubernetes phase

* fix: after rebase

* fix: added experimental conf to terraform

* feat: kubernetes phase is working

* feat: taking vars from infra output

* refactor: embed base into phase files

* fix: go imports

* fix: add missing license banners, temporarily deactivate some linting issues

* chore: improve tools error messages

* fix: bats integration test

* chore: temporarily silenced a couple linting issues

* feat: add create config command

* feat: add short description for create config command

* fix: rebase issue

* fix: default distribution version

* chore: add missing license banners

* feat: replace bats tests with ginkgo ones, add workdir flag

* feat: add test for download deps, refactor e2e tests

* feat: tag all test files with build tag, separate integration test data from e2e's

* feat: add test for create config command

* chore: add missing license banners

* chore: update deps

* fix: validate deps test, refactor code to have cmds output for free

* feat: bump golang in ci to 1.19.1

* feat: fix integration test data path

* fix: restore netrc file in test pipeline steps

* chore: fix create cluster command code style

* chore: remove old code

* chore: remove packr go library

* chore: remove old deps from manifests

* chore: bring back remaining logic from internal/app to cmd

* chore: update deps

* chore: refactor 'create cluster' logic to remove the need to use other app services

* fix: unhandled nil on Walk function

* chore: added provisioners dir to embed + refactor

* chore: refactor eksCluster v1alpha2 to make the code more self-contained

* chore: remove pointless use of generics

* chore: further refactor of cluster creator factory

* chore: refactor logic out of cluster creator phases to cmd runners

* chore: refactor tools and ekscluster phases

* chore: refactor 'x' packages within their own, dedicated folder

* chore: refactor 'base' struct to 'phase'

* feat: added expensive test on cluster create

* fix: tests after rebase

* feat: add local code coverage visualization

* feat: refine test targets

* feat: add manual confirmation in makefile before 'expensive' test targets

* feat: add tools target to simplify development

* chore: sort tools in makefile

* chore: remove unused tools in makefile

* fix: ansible bin not found

* fix: e2e tests on CI

* fix: wrong env var

* feat: add tests

* feat: add missing tool

* chore: add more tests

* fix: add semver's TesSameMinorStr and TestSamePatchStr

* feat: merge furyctl and defaults in distribution phase

* fix: linting

* fix: bug on _helpers.tpl not found

* fix: remove double code coverage report opening in makefile

* chore: added test to NewDefaultModelFromStruct

* chore: add metadata key to merged config

* chore: upgrade to new schema

* fix: test to reflect new furyctl file

* feat: dostrobution terraform step wip

* fix: distribution config file wrong paths

* chore: removed technical debt

* feat: added terraform step in phase distribution

* feat: inject into terraform wip

* chore: updated furyctl-defautls in tests

* feat: WIP distro manifests

* fix: post distro review

* fix: updated from new distro

* chore: refactoring + tests

* fix: tests + bugs

* chore: dump template updated

* fix: e2e tests

* Chore/enable linters (#105)

* feat: first linters round

* fix: linting after rebase

* fix: unit tests

* fix: nolint needed

* fix: wrapcheck

* chore: removed commented line from rules

* fix: linting goerr113

* chore: added revive linter

* fix: wrong args on kubectl

Co-authored-by: Alessio Pragliola <[email protected]>

* feat: refactor to enable deletion (#112)

* feat: refactor to enable deletion

* feat: delete command

* feat: phase in delete

* feat: check pending resources / retry resource apply

* fix: linting

* chore: added log infos

* feat: log to file and stdout

* feat: delete ingresses

* fix: tests + linting

* feat: error thrown when cfg already exists (#116)

* Feat: switch from current to home dir (#117)

* feat: switch to home dir in cmd

* feat: changed dir from current workdir to homedir

* fix: e2e tests

* feat: copy kubeconfig to current workdir

* feat: added copy ovpn file + changed phase dirs to not hidden

* feat: added --log flag (#119)

* feat: added --log flag

* fix: linting

* fix: e2e testing

* fix: wrong permission + improve flag description

* Feature/download bin default path (#118)

* feat: changed bin folder structure

* fix: e2e testing

* fix: unit tests

* Apply suggestions from code review

Co-authored-by: Claudio Beatrice <[email protected]>

* chore: removed misleading errors from cmds

* fix: post change requests

* fix: removed debug flag when not needed

* Update internal/dependencies/tools/tool.go

Co-authored-by: Claudio Beatrice <[email protected]>

* feat: added bin-path flag to commands that need it

Co-authored-by: Claudio Beatrice <[email protected]>

* feat: add analytics tracker and events

* chore: rename DistroDetails in ClusterDetails

* chore: small refactor,remove vscode's config file

* chore: improve command tracking

* feat: add analytics enable/disable capability

* fix: enable analytics flag

* fix: rebase issues

* chore: fix rebase issue

* feat: huge refactor,add more analytics logic, add flush

* chore: add suggested changes

* fix: license banner

* chore: optimize code

* feat: add mixpanel token's secret

* fix: rebase issues

* fix: cluster's error message

Co-authored-by: Claudio Beatrice <[email protected]>

* fix: download deps error message

Co-authored-by: Claudio Beatrice <[email protected]>

* chore: disable analytics in e2e tests, ignore any logs

* chore: replace fatal with error's level

* chore: replace error with panic to handle the error

* chore: Missing NL

Co-authored-by: Claudio Beatrice <[email protected]>

* chore: change log level

Co-authored-by: Claudio Beatrice <[email protected]>

* chore: rename GuardEvent to StopEvent

* chore: refactor

* feat: add cmdutil, wrap internal errors

* fix: license banner

* feat: add parsing flags to the cluster deletion

* fix: wrong flag name

* chore: update json schema validator from v1 to v5

* chore: update deps

* chore: update code after deps upgrade

* feat: improve missing binary deps error message

* chore: small refactor

* fix: miss go build test unit tag

* feat: change retry message and change to debug log level

* feat: implement move from hard-coded furyctl.yaml template to downloading it from the distro (#125)

* feat: implement move from hard-coded furyctl.yaml template to downloading it from the distro

* chore: rework distribution.GetTemplatePath function, fix tests

* feat: make ginkgo tests output more verbose

* chore: remove useless test file

* chore: add test file result to gitignore

* fix: update var references in phase templates

* fix: add random suffix to keyprefix used to store tf state in tests

* chore: typo in missing flag

Co-authored-by: Alessio Pragliola <[email protected]>

* feat: skip phase flag added

* feat: removed orphaned resources

* feat: added more expensive tests

* fix: missing NL at end of files

* chore: refactor create cluster command to reduce cyclomatic complexity

* chore: refine e2e tests descriptions

* chore: fix linting issues

* feat: add awscli's tool and runner

* chore: adapt tools validation with the new kfd schema

* chore: update with the new kfd schema, fix auto-connect flag

* feat: update gitignore

* chore: add new line at the end of file

* chore: update with the new kfd changes

* fix: remove unused methods implementations

* chore: improve the logic

* chore: format and fix lint issues

* fix: broken tests

* chore: point fury-distribution dep to the latest fury-next branch of the distro

* fix: add missing aws fake binary, avoid double channel closing.

* fix: add missing create cluster e2e test fake binaries

* fix: make create cluster e2e test fake binaries executable

* chore: try fixing e2e test

* chore: revert experiment with openvpn base path

* feat: add flag to skip deps validation, rename skip-download flag

* chore: fix linting issues

* chore: update deps

* chore: add ellipsis to a string

* chore: debug e2e test

* chore: debug e2e test

* chore: remove e2e tests debug

* chore: re-enable all e2e tests again

* feat: add aws-cli to e2e-test image in CI

* chore: replace deprecated go methods in tests

* chore: update ginkgo and gomega deps

* feat: improve ginkgo tests output

* fix: add missing dollar escaping in drone CI manifest

* fix: move openvpn tool version check after infra folder setup during cluster creation

* feat: use a tmp file for the patched furyctl.yaml instead of the original one

* chore: restore WorkDir for furyagent runner in infra phase of cluster creation

* feat: moved source from fury-distribution to templates/distribution

* chore: bump fury version, update template folders, replace panic with log+exit

* chore: removed ApiServerEndpointAccess from phase kubernetes

* chore: updated fields from new schema

* fix: wrap logic in main to correctly handle defers and process exit codes

* chore: remove unused code

* chore: bump deps

* chore: update tests to match the new schema

* fix: handle dry-run of cluster creation kubernetes phase gracefully

* chore: handle optional values

* chore: reverted removal of .Spec.Kubernetes.ApiServerEndpointAccess

* chore: update test schemas

* fix: wrong cidr range

* fix: linting

* chore: better error msg vpcID

* chore: reformat code

* chore: update fury-distribution dep

* chore: handle apiServerEndpointAccess as a pointer

* chore: update distro schemas

* chore: refine node templates parsing to better handle missing vars detection

* chore: update furyctl.yaml files in e2e tests to match latest schema

* fix: unique fields in template's nodes parser + skip variable nodes

* fix: TestVariableNode unit test

* chore: update fury-distribution dep

* fix: make help text for version flag in config create command a bit more clear

* feat: introduce support for distribution relative paths in distro downloader

* chore(gomod): clean up go sum

* feat: refine deps validation command

* fix: set the right env var name in info message

* fix: improve help text a bit

* fix: improve help text a bit

* fix: linting issues and tests

* fix: improve help text

* chore: improve OpenVPN connection messages

* Feature/first wave adjustments (#131)

* feat: improved log messages

task list:
- add debug message when waiting for cluster resources removal
- append '...' to Deleting x phase messages
- when deleting a cluster or deleting just phase infrastructure print a WARN telling the user to kill the openvpn process

* feat: bump golang version to 1.19.4

* feat: added kubeconfig flag

* fix: linting

* chore: refactor phases

* fix: e2e testing in CI

following workaround found on: sgerrand/alpine-pkg-glibc#185

* fix: wrong order in apk add flags (?)

* fix: e2e testing

* feat: added mandatory condition to kubeconfig flag in create cmd

* feat: added mandatory condition to kubeconfig flag in delete cmd

* chore: moved up --kubeconfig validation

* chore: better UX from Ramiro's review

Co-authored-by: Ramiro Algozino <[email protected]>

* chore: UX improvement on delete cluster missing kubeconfig

Co-authored-by: Ramiro Algozino <[email protected]>

* chore: improved missing kubeconfig msg + revert to warn

* chore: adjust delete cluster missing kubeconfig flag msg

Co-authored-by: Ramiro Algozino <[email protected]>

* feat: renamed bucket to bucketName (#146)

* feat: removed furyctl binary version check (#147)

* feat: handle unsupported phases (#149)

* feat: add repo check

* chore: add retries threshold, improve ux

* chore: small refactor

* fix: switch git prefix

* fix: modules download not working on branches

* fix: linting on deps download

* feat: added dry run to cluster delete (#154)

* feat: added dry run to infra/kube phase

* feat: added dry run to distro phase

* chore: added debug mode to exec in dry-run

* feat: improved ux in log messages

* chore: remove logrus timestamp format

* chore: force debuglevel as workaround for now

* fix: changed wrong url in help --distro-location

* feat: added dry-run logic in phase distribution to create cluster cmd (#157)

* feat: added dry-run logic to distribution phase in create cluster

* chore: removed log message

* chore: removed kubectl apply --dry-run

* feat: store furyctl.yaml in a cluster's secret

* feat: add missing license banner and an info message

* chore: use creator kubeconfig path

* chore: replace kube client with kubectl runner

* chore: small refactor

* fix: wrong config path

* chore: refactor

* fix: don't store secret if dryrun is enabled

* chore: refactor dryRun injection

* fix: problem with kubeconfig / secret file

* chore: added unit tests + refactoring to kubex

Co-authored-by: omissis <[email protected]>
Co-authored-by: Luigi <[email protected]>
Co-authored-by: Luigi Barbato <[email protected]>
Co-authored-by: Ramiro Algozino <[email protected]>
  • Loading branch information
5 people committed Apr 19, 2023
1 parent 878ca70 commit 0d15d8c
Show file tree
Hide file tree
Showing 350 changed files with 26,815 additions and 10,350 deletions.
66 changes: 56 additions & 10 deletions .drone.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ name: main

steps:
- name: prepare
image: quay.io/sighup/golang:1.19.0
image: quay.io/sighup/golang:1.19.4
depends_on:
- clone
pull: always
Expand All @@ -17,12 +17,11 @@ steps:
- go mod download

- name: license
image: quay.io/sighup/golang:1.19.0
image: quay.io/sighup/golang:1.19.4
depends_on:
- clone
pull: always
commands:
- go install github.com/google/[email protected]
- make license-check
environment:
CGO_ENABLED: 0
Expand All @@ -31,7 +30,7 @@ steps:
GOTMPDIR: /drone/src/.go/tmp

- name: lint
image: quay.io/sighup/golang:1.19.0
image: quay.io/sighup/golang:1.19.4
depends_on:
- prepare
pull: always
Expand All @@ -44,7 +43,7 @@ steps:
GOTMPDIR: /drone/src/.go/tmp

- name: test-unit
image: quay.io/sighup/golang:1.19.0
image: quay.io/sighup/golang:1.19.4
depends_on:
- prepare
pull: always
Expand All @@ -57,13 +56,11 @@ steps:
GOTMPDIR: /drone/src/.go/tmp

- name: test-integration
image: quay.io/sighup/golang:1.19.0
image: quay.io/sighup/golang:1.19.4
depends_on:
- prepare
commands:
- echo $${NETRC_FILE} > /root/.netrc
- git clone --depth 1 --branch v1.8.0 https://github.com/bats-core/bats-core.git
- ./bats-core/install.sh /usr/local
- make test-integration
environment:
CGO_ENABLED: 0
Expand All @@ -73,13 +70,59 @@ steps:
NETRC_FILE:
from_secret: NETRC_FILE

- name: test-e2e
image: quay.io/sighup/golang:1.19.4
depends_on:
- prepare
commands:
- apk add --update binutils curl groff openssh-client
# Install aws-cli
- curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub
- curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/$${GLIBC_VER}/glibc-$${GLIBC_VER}.apk
- curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/$${GLIBC_VER}/glibc-bin-$${GLIBC_VER}.apk
- curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/$${GLIBC_VER}/glibc-i18n-$${GLIBC_VER}.apk
- apk add --force-overwrite --virtual .glibc glibc-$${GLIBC_VER}.apk glibc-bin-$${GLIBC_VER}.apk glibc-i18n-$${GLIBC_VER}.apk
- /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8
- curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64-$${AWSCLI_VERSION}.zip -o awscliv2.zip
- unzip awscliv2.zip
- aws/install
- apk add gcompat || true
# Setup SSH
- mkdir /root/.ssh
- echo "$${GITHUB_SSH}" | tr -d '\r' | sed -e '$a\' > /root/.ssh/id_rsa
- echo $${NETRC_FILE} > /root/.netrc
- chmod 600 /root/.ssh/id_rsa
- touch /root/.ssh/known_hosts
- chmod 600 /root/.ssh/known_hosts
- ssh-keyscan -H github.com > /etc/ssh/ssh_known_hosts 2> /dev/null
# Run tests
- make test-e2e
environment:
CGO_ENABLED: 0
GOCACHE: /drone/src/.go/cache
GOMODCACHE: /drone/src/.go/modcache
GOTMPDIR: /drone/src/.go/tmp
AWSCLI_VERSION: 2.8.12
GLIBC_VER: 2.35-r0
AWS_ACCESS_KEY_ID:
from_secret: AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY:
from_secret: AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION:
from_secret: AWS_REGION
FURYCTL_MIXPANEL_TOKEN:
from_secret: FURYCTL_MIXPANEL_TOKEN
NETRC_FILE:
from_secret: NETRC_FILE
GITHUB_SSH:
from_secret: GITHUB_SSH

- name: build
image: ghcr.io/goreleaser/goreleaser:v1.11.4
depends_on:
- prepare
pull: always
commands:
- go install github.com/gobuffalo/packr/v2/[email protected]
- git reset --hard
- git fetch --tags
- make build
Expand All @@ -92,6 +135,8 @@ steps:
GOCACHE: /drone/src/.go/cache
GOMODCACHE: /drone/src/.go/modcache
GOTMPDIR: /drone/src/.go/tmp
FURYCTL_MIXPANEL_TOKEN:
from_secret: FURYCTL_MIXPANEL_TOKEN
GITHUB_TOKEN:
from_secret: GITHUB_TOKEN

Expand All @@ -103,7 +148,6 @@ steps:
- test-unit
- test-integration
commands:
- go install github.com/gobuffalo/packr/v2/[email protected]
- git reset --hard
- git fetch --tags
- make release
Expand All @@ -116,6 +160,8 @@ steps:
GOCACHE: /drone/src/.go/cache
GOMODCACHE: /drone/src/.go/modcache
GOTMPDIR: /drone/src/.go/tmp
FURYCTL_MIXPANEL_TOKEN:
from_secret: FURYCTL_MIXPANEL_TOKEN
GITHUB_TOKEN:
from_secret: GITHUB_TOKEN

Expand Down
15 changes: 13 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,19 @@
coverage.out
coverage.*
coverprofile.out
vendor
bin
.idea
*-packr.go
furyctl
demo
dist/
.infrastructure
.kubernetes
.distribution
test/data/e2e/**/target
test/e2e/e2e.test

.vscode
*.log
go.work
go.work.sum
.env
1 change: 0 additions & 1 deletion .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ project_name: furyctl
before:
hooks:
- go mod tidy
- packr2 build
builds:
- env:
- CGO_ENABLED=0
Expand Down
Loading

0 comments on commit 0d15d8c

Please sign in to comment.