Skip to content
This repository has been archived by the owner on Nov 2, 2024. It is now read-only.

Commit

Permalink
chore: refactor to nix linters
Browse files Browse the repository at this point in the history
  • Loading branch information
ajgon committed Aug 12, 2024
1 parent 20e2bcc commit bca978b
Show file tree
Hide file tree
Showing 15 changed files with 233 additions and 83 deletions.
37 changes: 11 additions & 26 deletions .github/workflows/linter.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,18 @@ jobs:
with:
token: "${{ steps.app-token.outputs.token }}"

- name: Setup OpenTofu
uses: opentofu/setup-opentofu@ae80d4ecaab946d8f5ff18397fbf6d0686c6d46a # v1.0.3
- name: Install nix
uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
with:
nix_path: nixpkgs=channel:nixos-unstable
extra_nix_config: |
access-tokens = github.com=${{ steps.app-token.outputs.token }}
- name: Enable develop shell
uses: nicknovitski/nix-develop@a2060d116a50b36dfab02280af558e73ab52427d # v1.1.0

- name: Setup TFLint
uses: terraform-linters/setup-tflint@19a52fbac37dacb22a09518e4ef6ee234f2d4987 # v4.0.0
- name: Run linters
run: pre-commit run --all-files

- name: Setup Homebrew
uses: Homebrew/actions/setup-homebrew@master
Expand All @@ -39,28 +46,6 @@ jobs:
shell: bash
run: brew install helm kubeconform yq

- name: Init TFLint
run: tflint --init
env:
GITHUB_TOKEN: "${{ steps.app-token.outputs.token }}"

- name: Run ShellCheck
uses: ludeeus/action-shellcheck@00cae500b08a931fb5698e11e79bfbd38e612a38 # 2.0.0

- name: Run tofu formatter
run: tofu fmt -check -diff -recursive tofu/

- name: Run tofu lint
run: tflint -f compact

- name: Run yamllint
uses: karancode/yamllint-github-action@fdef6bc189425ecc84cc4543b2674566c0827053 # v2.1.1
with:
yamllint_strict: true
yamllint_comment: true
env:
GITHUB_ACCESS_TOKEN: "${{ steps.app-token.outputs.token }}"

- name: Run task lint:egress-comment
run: ./.taskfiles/Lint/egress-comment-job.sh

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/lychee.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jobs:
--json number \
| jq --raw-output '.[0].number' \
)
echo "issue-number=${issue_number}" >> $GITHUB_OUTPUT
echo "issue-number=${issue_number}" >> "$GITHUB_OUTPUT"
echo "${issue_number}"
- name: Create or Update Issue
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,4 @@ talenv*

# nix
result
.pre-commit-config.yaml
5 changes: 5 additions & 0 deletions .markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
MD013:
# Number of characters
line_length: 120
tables: false
11 changes: 6 additions & 5 deletions .taskfiles/Lint/yaml-json-schema-job.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env bash

SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
TMP_DIR="$(mktemp -d)"

# search for manifests without JSON schema links
yaml_files="$(sh -c "find . -name '*.y*ml' -not -name \"$(yq '.ignoreNames|join("\" -not -name \"")' "${SCRIPT_DIR}/../../.yaml-json-schema")\" -not -path ./\"$(yq '.ignorePaths|join("\" -not -path ./\"")' "${SCRIPT_DIR}/../../.yaml-json-schema")\"")"
Expand All @@ -15,17 +16,17 @@ for file in $yaml_files; do
fi
fi
# shellcheck disable=SC2016
if ! yq -s '"/tmp/test_split_" + $index' "${file}" 2> /dev/null; then
cp "${file}" /tmp/test_split_0.yml
if ! yq -s '"${TMP_DIR}/test_split_" + $index' "${file}" 2> /dev/null; then
cp "${file}" "${TMP_DIR}/test_split_0.yml"
fi
if grep -Hoc '# yaml-language-serve' /tmp/test_split_* | grep -q ':0$'; then
if grep -Hoc '# yaml-language-serve' "${TMP_DIR}/test_split_"* | grep -q ':0$'; then
if [ $error == 0 ]; then
echo "Found YAML files without valid JSON schema manifest links:"
fi
error=1
echo "${file}"
else
for split in /tmp/test_split_*; do
for split in "${TMP_DIR}/test_split_"*; do
if [ -z "${IGNORE_SCHEMA_FETCH}" ]; then
schemaUrl="$(grep '# yaml-language-serve' "${split}" | head -n 1 | awk -F= '{print $2}')"
if [ -z "$schemaUrl" ]; then
Expand All @@ -46,7 +47,7 @@ for file in $yaml_files; do
fi
done
fi
rm -rf /tmp/test_split_*
rm -rf "${TMP_DIR}/test_split_"*
done

if [ $error == 1 ]; then
Expand Down
1 change: 1 addition & 0 deletions .yaml-json-schema
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ ignorePaths:
- talos/*/tal*.yaml

ignoreNames:
- .markdownlint.yaml
- protection.tmpl.yaml
- "*.sops.y*ml"
- "values.*y*ml"
40 changes: 30 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<!-- markdownlint-disable MD013 MD033 MD041 -->
<div align="center">
<img src="docs/img/k8shappy.png">
<img src="docs/img/k8shappy.png" alt="kubepepe">
<br>
<sup><sup>
Art by <a href="https://twitter.com/SkeletalGadget">@SkeletalGadget</a>
Expand Down Expand Up @@ -31,40 +32,53 @@ _... automated via [ArgoCD](https://argoproj.github.io/cd/), [Renovate](https://
[![Power-Usage](https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.rzegocki.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dcluster_power_usage&style=flat-square&label=Power)](https://github.com/kashalls/kromgo/)

</div>
<!-- markdownlint-enable MD013 MD033 -->

---

## 📖 Overview

This is a repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code (IaC) and GitOps practices using tools like [OpenTofu](https://opentofu.org/), [Kubernetes](https://kubernetes.io), [ArgoCD](https://argoproj.github.io/cd/), [Renovate](https://github.com/renovatebot/renovate) and [GitHub Actions](https://github.com/features/actions).
This is a repository for my home infrastructure and Kubernetes cluster.
I try to adhere to Infrastructure as Code (IaC) and GitOps practices using tools like [OpenTofu](https://opentofu.org/),
[Kubernetes](https://kubernetes.io), [ArgoCD](https://argoproj.github.io/cd/), [Renovate](https://github.com/renovatebot/renovate)
and [GitHub Actions](https://github.com/features/actions).

---

## ⛵ Kubernetes

### Installation

This semi hyper-converged cluster runs [Talos Linux](https://talos.dev), an immutable and ephemeral Linux distribution built for [Kubernetes](https://kubernetes.io), deployed on bare-metal [Intel NUCs](https://www.intel.com/content/www/us/en/products/details/nuc.html). [Rook](https://rook.io) then provides my workloads with persistent block, and file storage; while a seperate server provides file storage for my media.
This semi hyper-converged cluster runs [Talos Linux](https://talos.dev), an immutable and ephemeral Linux distribution
built for [Kubernetes](https://kubernetes.io), deployed on bare-metal [Intel NUCs](https://www.intel.com/content/www/us/en/products/details/nuc.html).
[Rook](https://rook.io) then provides my workloads with persistent block, and file storage;
while a seperate server provides file storage for my media.

### Core Components

- [actions-runner-controller](https://github.com/actions/actions-runner-controller): Self-hosted Github runners.
- [cilium](https://cilium.io): Internal Kubernetes networking plugin.
- [cert-manager](https://cert-manager.io): Creates SSL certificates for services in my Kubernetes cluster.
- [external-dns](https://github.com/kubernetes-sigs/external-dns): Automatically manages DNS records from my cluster in a cloud DNS provider.
- [ingress-nginx](https://github.com/kubernetes/ingress-nginx): Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer.
- [external-dns](https://github.com/kubernetes-sigs/external-dns): Automatically manages DNS records from my cluster
in a cloud DNS provider.
- [ingress-nginx](https://github.com/kubernetes/ingress-nginx): Ingress controller for Kubernetes using NGINX as
a reverse proxy and load balancer.
- [rook](https://rook.io): Distributed block storage for peristent storage.
- [spegel](https://github.com/XenitAB/spegel): Stateless cluster local OCI registry mirror.
- [vault](https://www.vaultproject.io/): Safe and encrypted storage for all Kubernetes secrets.
- [volsync](https://github.com/backube/volsync): Backup and recovery of persistent volume claims.

### GitOps

[ArgoCD](https://argoproj.github.io/cd/) watches the clusters in my [kubernetes](./kubernetes/) folder (see Directories below) and makes the changes to my clusters based on the state of my Git repository.
[ArgoCD](https://argoproj.github.io/cd/) watches the clusters in my [kubernetes](./kubernetes/) folder
(see Directories below), and makes the changes to my clusters based on the state of my Git repository.

The way ArgoCD works for me here is it will recursively search the `kubernetes/clusters/${cluster}` folder and deploys all `application.yaml` manifests. I follow "app of apps" pattern, so cluster apps can include other apps, which can be shared between clusters, and which live under `kubernetes/apps` directory.
The way ArgoCD works for me here is it will recursively search the `kubernetes/clusters/${cluster}` folder,
and deploys all `application.yaml` manifests. I follow "app of apps" pattern, so cluster apps can include other apps,
which can be shared between clusters, and which live under `kubernetes/apps` directory.

[Renovate](https://github.com/renovatebot/renovate) watches my **entire** repository looking for dependency updates, when they are found a PR is automatically created. When some PRs are merged Flux applies the changes to my cluster.
[Renovate](https://github.com/renovatebot/renovate) watches my **entire** repository looking for dependency updates.
When they are found a PR is automatically created. When some PRs are merged ArgoCD applies the changes to my cluster.

### Directories

Expand All @@ -84,7 +98,9 @@ This Git repository contains the following directories under [Kubernetes](./kube

## ☁️ Cloud Dependencies

While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup. This saves me from having to worry about two things. (1) Dealing with chicken/egg scenarios and (2) services I critically need whether my cluster is online or not.
While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup.
This saves me from having to worry about two things. (1) Dealing with chicken/egg scenarios and (2) services I critically
need whether my cluster is online or not.

| Service | Use | Cost |
|-------------------------------------------|----------------------------------------------------------------|----------------|
Expand Down Expand Up @@ -113,6 +129,7 @@ While most of my infrastructure and workloads are self-hosted I do rely upon the

## ⭐ Stargazers

<!-- markdownlint-disable MD013 MD033 -->
<div align="center">

<a href="https://star-history.com/#deedee-ops/home-ops&Date">
Expand All @@ -124,12 +141,15 @@ While most of my infrastructure and workloads are self-hosted I do rely upon the
</a>

</div>
<!-- markdownlint-enable MD013 MD033 -->

---

## 🤝 Gratitude and Thanks

Thanks to all the people who donate their time to the [Home Operations](https://discord.gg/home-operations) Discord community. Be sure to check out [kubesearch.dev](https://kubesearch.dev/) for ideas on how to deploy applications or get ideas on what you may deploy.
Thanks to all the people who donate their time to the [Home Operations](https://discord.gg/home-operations) Discord community.
Be sure to check out [kubesearch.dev](https://kubesearch.dev/) for ideas on how to deploy applications
or get ideas on what you may deploy.

---

Expand Down
2 changes: 2 additions & 0 deletions docs/INSTALL.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# INSTALL

## Prepare

### Zap disks for CEPH
Expand Down
93 changes: 92 additions & 1 deletion flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit bca978b

Please sign in to comment.