Skip to content

Commit

Permalink
Merge pull request #56 from MEhrn00/main
Browse files Browse the repository at this point in the history
Add documentation hosting on Github pages
  • Loading branch information
t94j0 authored Apr 25, 2024
2 parents 7ff1675 + 98ab8bf commit e2c4ea9
Show file tree
Hide file tree
Showing 27 changed files with 616 additions and 179 deletions.
71 changes: 71 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Build documentation

on:
push:
branches: [main]

# Only trigger workflow when documentation files are changed
paths:
- 'docs/**'
- 'mkdocs.yml'
- '.github/workflows/docs.yml'

# Prevent this workflow from running concurrently with the helm-release.yml workflow
concurrency:
group: "pages"
cancel-in-progress: false

jobs:
docs:
runs-on: ubuntu-latest

permissions:
pages: write
id-token: write

environment:
name: github-pages

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python virtualenv
run: |
pip install --upgrade pip
python -m venv env
source env/bin/activate
pip install -r docs/requirements.txt
- name: Build documentation
run: |
source env/bin/activate
mkdocs build
- name: Add existing Helm repository index.yml file
env:
GH_TOKEN: ${{ github.token }}
run: |
PAGES_URL=$(gh api \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/${{ github.repository }}/pages \
| jq -r '.html_url')
if [[ "$PAGES_URL" != "null" ]]; then
HTTP_STATUS=$(curl -sL -w '%{http_code}' "${PAGES_URL%/}/index.yaml" -o site/index.yaml)
if [[ "$HTTP_STATUS" != "200" ]]; then
rm site/index.yaml
fi
fi
- name: Setup Github pages
uses: actions/configure-pages@v4

- name: Create Github pages artifact
uses: actions/upload-pages-artifact@v3
with:
path: site

- name: Deploy documentation to Github pages
uses: actions/deploy-pages@v4
152 changes: 102 additions & 50 deletions .github/workflows/helm-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,16 @@ on:
default: 'helm'
type: string

env:
PACKAGE_DIR: dist


# Only allow one instance of this workflow to run at a time
# Prevent this workflow from running concurrently with the docs.yml workflow
concurrency:
group: "pages"
cancel-in-progress: true
cancel-in-progress: false

jobs:
verify:
name: Verify release

runs-on: ubuntu-latest

steps:
Expand Down Expand Up @@ -70,38 +69,42 @@ jobs:
false
fi
release:
docs:
name: Build documentation
needs: verify

# Provision a Github token with repository and pages write permissions
permissions:
contents: write
pages: write
id-token: write

# Use the github-pages environment. The actions/deploy-pages workflow fails with a
# "Invalid environment node id" error if an environment is not specified.
# https://github.com/actions/deploy-pages/issues/271
environment:
name: github-pages

runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Configure git
- name: Setup Python virtualenv
run: |
git config user.name "$GITHUB_ACTOR"
git config user.email "[email protected]"
pip install --upgrade pip
python -m venv env
source env/bin/activate
pip install -r docs/requirements.txt
- name: Create a git tag for the release
uses: EndBug/add-and-commit@v9
- name: Build documentation
run: |
source env/bin/activate
mkdocs build
- name: Store built documentation artifacts
uses: actions/upload-artifact@v4
with:
message: "Nemesis v${{ inputs.version }}"
push: true
tag: "v${{ inputs.version }}"
name: docs
path: site

helm:
name: Package Helm charts
needs: verify

runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Helm
env:
Expand All @@ -121,17 +124,15 @@ jobs:
done
done
- name: Package Helm charts
- name: Create Chart packages
env:
PACKAGE_DIR: ${{ env.PACKAGE_DIR }}
CHARTS_DIR: ${{ inputs.charts_dir }}
run: |
mkdir -p $PACKAGE_DIR
find $CHARTS_DIR -maxdepth 2 -mindepth 2 -type f -name "Chart.yaml" -printf '%h\n' | xargs -I % bash -c "helm package -d $PACKAGE_DIR %"
mkdir -p dist
find $CHARTS_DIR -maxdepth 2 -mindepth 2 -type f -name "Chart.yaml" -printf '%h\n' | xargs -I % bash -c "helm package -d dist %"
- name: Pull in previous index.yaml file if it exists
env:
PACKAGE_DIR: ${{ env.PACKAGE_DIR }}
GH_TOKEN: ${{ github.token }}
run: |
PAGES_URL=$(gh api \
Expand All @@ -141,46 +142,97 @@ jobs:
| jq -r '.html_url')
if [[ "$PAGES_URL" != "null" ]]; then
HTTP_STATUS=$(curl -sL -w '%{http_code}' "${PAGES_URL%/}/index.yaml" -o ${PACKAGE_DIR}/index.yaml)
HTTP_STATUS=$(curl -sL -w '%{http_code}' "${PAGES_URL%/}/index.yaml" -o dist/index.yaml)
if [[ "$HTTP_STATUS" != "200" ]]; then
rm ${PACKAGE_DIR}/index.yaml
rm dist/index.yaml
fi
fi
- name: Update Helm repository index.yaml file
env:
PACKAGE_DIR: ${{ env.PACKAGE_DIR }}
CHART_BASE_URL: ${{ github.server_url }}/${{ github.repository }}/releases/download/v${{ inputs.version }}
run: |
if [ -f ${PACKAGE_DIR}/index.yaml ]; then
helm repo index $PACKAGE_DIR --merge ${PACKAGE_DIR}/index.yaml --url $CHART_BASE_URL
if [ -f dist/index.yaml ]; then
helm repo index dist --merge dist/index.yaml --url $CHART_BASE_URL
else
helm repo index $PACKAGE_DIR --url $CHART_BASE_URL
helm repo index dist --url $CHART_BASE_URL
fi
- name: Create Github release with the Helm charts
env:
PACKAGE_DIR: ${{ env.PACKAGE_DIR }}
VERSION: v${{ inputs.version }}
GH_TOKEN: ${{ github.token }}
run: gh release create ${VERSION} -R ${{ github.repository }} -t "Nemesis $VERSION" -n "Nemesis $VERSION release" $PACKAGE_DIR/*.tgz
- name: Store Helm chart artifacts
uses: actions/upload-artifact@v4
with:
name: charts
path: dist

- name: Remove packaged Helm charts
env:
PACKAGE_DIR: ${{ env.PACKAGE_DIR }}
run: rm -f ${PACKAGE_DIR}/*.tgz
release:
name: Publish and release files
needs:
- verify
- docs
- helm

# Provision a Github token with repository and pages write permissions
permissions:
contents: write
pages: write
id-token: write

# Use the github-pages environment. The actions/deploy-pages workflow fails with a
# "Invalid environment node id" error if an environment is not specified.
# https://github.com/actions/deploy-pages/issues/271
environment:
name: github-pages

runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Configure git
run: |
git config user.name "$GITHUB_ACTOR"
git config user.email "[email protected]"
- name: Create a git tag for the release
uses: EndBug/add-and-commit@v9
with:
message: "Nemesis v${{ inputs.version }}"
push: true
tag: "v${{ inputs.version }}"

- name: Download documentation site files
uses: actions/download-artifact@v4
with:
name: docs
path: site

- name: Download Helm chart files
uses: actions/download-artifact@v4
with:
name: charts
path: dist

- name: Merge Chart index.yaml file with documentation files
run: mv dist/index.yaml site/index.yaml

- name: Setup Github pages
uses: actions/configure-pages@v4

- name: Create Github pages artifact
uses: actions/upload-pages-artifact@v3
with:
path: ${{ env.PACKAGE_DIR }}
path: site

- name: Deploy Helm chart repository to Github pages
- name: Deploy Github pages site
uses: actions/deploy-pages@v4

- name: Create Github release with the Helm charts
env:
VERSION: v${{ inputs.version }}
GH_TOKEN: ${{ github.token }}
run: gh release create ${VERSION} -R ${{ github.repository }} -t "Nemesis $VERSION" -n "Nemesis $VERSION release" dist/*.tgz

- name: Remove Github release and tag on failure
continue-on-error: true
if: ${{ failure() }}
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ __pycache__
nemesis.config
config.yml
submit_to_nemesis.yaml
submit/
submit/
site
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="img/nemesis_white.png" alt="Nemesis" style="width: 800px;" />
<img src="docs/images/nemesis_white.png" alt="Nemesis" style="width: 800px;" />
</p>
<hr />

Expand Down Expand Up @@ -33,15 +33,15 @@ Built on Kubernetes with scale in mind, our goal with Nemesis was to create a ce

Nemesis aims to automate a number of repetitive tasks operators encounter on engagements, empower operators’ analytic capabilities and collective knowledge, and create structured and unstructured data stores of as much operational data as possible to help guide future research and facilitate offensive data analysis.

# Setup / Installation
## Setup / Installation
Follow the [quickstart guide](docs/quickstart.md)

Or see the full [setup instructions](docs/setup.md)

# Usage
## Usage
See the [Nemesis Usage Guide](docs/usage_guide.md).

# Contributing / Development Environment Setup
## Contributing / Development Environment Setup
See [development.md](./docs/development.md)

## Further Reading
Expand All @@ -54,7 +54,7 @@ See [development.md](./docs/development.md)
| *On (Structured) Data* | Jul 26, 2023 | https://posts.specterops.io/on-structured-data-707b7d9876c6 |


# Acknowledgments
## Acknowledgments

Nemesis is built on large chunk of other people's work. Throughout the codebase we've provided citations, references, and applicable licenses for anything used or adapted from public sources. If we're forgotten proper credit anywhere, please let us know or submit a pull request!

Expand Down
8 changes: 4 additions & 4 deletions docs/access_nemesis.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ If you use Minikube, by default, services are not exposed anywhere outside of th

In the examples below, the following assumptions are made:
- Minikube server IP: `192.168.230.42`.
- Nemesis's [`nemesisHttpServer` option](../helm/nemesis/values.yaml) is configured to be `https://192.168.230.42:7443/`
- Nemesis's [`nemesisHttpServer` option](https://github.com/SpecterOps/Nemesis/blob/main/helm/nemesis/values.yaml) is configured to be `https://192.168.230.42:7443/`

To quickly setup an SSH port forward, you can use the [minikube_port_forward.sh](../scripts/minikube_port_forward.sh) script:
To quickly setup an SSH port forward, you can use the [minikube_port_forward.sh](https://github.com/SpecterOps/Nemesis/blob/main/scripts/minikube_port_forward.sh) script:
```bash
cd Nemesis/scripts/
./minikube_port_forward.sh 7443
Expand Down Expand Up @@ -65,7 +65,7 @@ There's many ways you can do this (kubectl, SSH local port forward, Socat, IP ta
**SSH**
Using an SSH local port forward is our preferred method right now as it's simple to setup and proven reliable.

Let's say you configure the [`nemesisHttpServer` option](../helm/nemesis/values.yaml#L8) to listen on port `<HOST_IP>:7443`. Running the following command on the k8s host will expose the Minikube's endpoint externally (output in Step 1) using an SSH local port forward:
Let's say you configure the [`nemesisHttpServer` option](https://github.com/SpecterOps/Nemesis/blob/main/helm/nemesis/values.yaml#L8) to listen on port `<HOST_IP>:7443`. Running the following command on the k8s host will expose the Minikube's endpoint externally (output in Step 1) using an SSH local port forward:
```bash
ssh -N -L <HOST_IP>:7443:192.168.49.2:30123
```
Expand All @@ -84,4 +84,4 @@ sudo setcap CAP_NET_BIND_SERVICE=+eip $(which kubectl)
```

## Accessing Nemesis via Docker Desktop
Nemesis can run locally Docker Desktop. In that case, once Nemesis is deployed, you can access the nginx endpoint at `https://localhost/`.
Nemesis can run locally Docker Desktop. In that case, once Nemesis is deployed, you can access the nginx endpoint at `https://localhost/`.
Loading

0 comments on commit e2c4ea9

Please sign in to comment.