Skip to content

Commit

Permalink
Merge branch 'main' into fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
dtroyer-salad authored Nov 28, 2023
2 parents 7e4c239 + 6d74be1 commit aadeb44
Show file tree
Hide file tree
Showing 7 changed files with 180 additions and 21 deletions.
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ The SaladCloud Virtual Kubelet Provider creates a _virtual node_ in your K8s clu

To the K8s API, it looks like a real node. However, when you schedule a pod on the virtual node, a container group deployment is created using the SaladCloud API instead of running the pod on a node in the K8s cluster. The container group deployment runs the pod on a remote, GPU-enabled node on the SaladCloud network.

## Demo

This was used in a presentation at KubeCon2023, the script and pod spec file for the QR code workload are in
the [demo](demo) directory.

## Development

Follow the steps below to get started with local development.
Expand Down
32 changes: 32 additions & 0 deletions demo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Salad Cloud Virtual Kubelet Demo

These files were originally used in dtroyer's KubeCon2023 talk.

## Run the Demo

This outlines running the demo on Docker Desktop's Kubernetes implementation:

* Set up the environment with a .env file (or equivalent) similar to this one:
```bash
export LOG_LEVEL=INFO
export SCE_API_KEY=<api-key>
export SCE_ORGANIZATION_NAME=salad
export SCE_PROJECT_NAME=demo
export NAMESPACE=saladcloud-demo
export NODE_NAME=${SCE_PROJECT_NAME}
```
* Start with a fresh Kubernetes environment. Only the docker-desktop conrol plane should be displayed
with `kubectl get node` and `kubectl get pod`. `demo.sh status` will run both of those commands
at once.
* Run the virtual kubelet via its Helm chart using `demo.sh start`.
* Run `demo.sh status` again to see that the virtual kubelet is registered as an agent
* Start the QR code workload with `kubectl apply qr.yaml`.
* Run `demo.sh status` again to see two additional pods listed that are the requested container groups.
The pod names should match the Container Groups in the Salad Portal (NOTE: with a prefix!)
* Note that Kubernetes pods are mapped to SCE Container Groups 1:1, specifying 2 replicas on the YAML spec
will result in two Contarer Groups being created. At this time there is not a mechanism to specify the
Container Group replicas and it will always be 1.
* Once a Container Group is shown running, grab the assigned URL from the portal and paste it into a browser.
Commence generating crazy QR codes that look like city skylines or a plaid flannel shirt.
* Run `kubectl delete -f qr.yaml` to stop the Container Groups.
* Run `demo.sh stop` to stop the virtual kubelet pod.
61 changes: 61 additions & 0 deletions demo/demo.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#!/bin/bash
# demo.sh - Control demo saladcloud-virtual-kublet instance in K8s cluster

# Typically a .env file similar to the following is used to configure the virtual kubelet
# export LOG_LEVEL=INFO
# export SCE_API_KEY=<api-key>
# export SCE_ORGANIZATION_NAME=salad
# export SCE_PROJECT_NAME=demo
# export NAMESPACE=saladcloud-demo
# export NODE_NAME=${SCE_PROJECT_NAME}

TOP_DIR=$(cd $(dirname "$0") && pwd)

# Script lives in a subdirectory of the top repo dir
pushd $TOP_DIR/.. >/dev/null

IMAGE_TAG=${IMAGE_TAG:-latest}
NAMESPACE=${NAMESPACE:-saladcloud-demo}
NODE_NAME=${NODE_NAME:-demo}

if [[ "$1" == "start" ]]; then
shift
CMD=" \
helm install \
--create-namespace \
--namespace ${NAMESPACE} \
--set salad.organizationName=${SCE_ORGANIZATION_NAME} \
--set salad.projectName=${SCE_PROJECT_NAME} \
--set provider.image.tag=${IMAGE_TAG} \
--set provider.nodename=${NODE_NAME}-vk \
${NODE_NAME} \
./charts/virtual-kubelet"
echo $CMD
$CMD \
--set salad.apiKey=${SCE_API_KEY} \
--set provider.logLevel=${LOG_LEVEL}

elif [[ "$1" == "stop" ]]; then
shift
helm uninstall \
--namespace ${NAMESPACE} \
${NODE_NAME}
elif [[ "$1" == "status" ]]; then
echo ""
echo "$ kubectl get node"
kubectl get node
echo ""
echo "$ kubectl --namespace ${NAMESPACE} get pod"
kubectl --namespace ${NAMESPACE} get pod
elif [[ "$1" == "logs" ]]; then
podname=$(kubectl --namespace ${NAMESPACE} get pod|awk "/${NODE_NAME}/ { print \$1 }")
kubectl --namespace ${NAMESPACE} logs $podname
elif [[ "$1" == "apply" ]]; then
# Launch the qr-code
kubectl apply -f demo/qr.yaml
elif [[ "$1" == "delete" ]]; then
# Delete the qr-code
kubectl delete -f demo/qr.yaml
fi

popd >/dev/null
59 changes: 59 additions & 0 deletions demo/qr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: qr-code-demo
name: qr-code
namespace: saladcloud-demo
spec:
replicas: 2
selector:
matchLabels:
app: qr-code-demo
template:
metadata:
annotations:
salad.com/country-codes: us
salad.com/networking-protocol: "http"
salad.com/networking-port: "1234"
salad.com/networking-auth: "false"
salad.com/gpu-classes: "dec851b7-eba7-4457-a319-a01b611a810e"
# salad.com/gpu-classes: "cb6c1931-89b6-4f76-976f-54047320ccc6"
labels:
app: qr-code-demo
spec:
containers:
- image: saladtechnologies/stable-fast-qr-code:latest-baked
name: qr-code
resources:
limits:
cpu: 2
memory: 8192
env:
- name: HOST
value: "*"
- name: PORT
value: "1234"
startupProbe:
exec:
command: [ "curl", "--fail", "http://localhost:1234/hc" ]
initialDelaySeconds: 60
failureThreshold: 60
periodSeconds: 10
livenessProbe:
exec:
command: [ "curl", "--fail", "http://localhost:1234/hc" ]
initialDelaySeconds: 60
failureThreshold: 60
periodSeconds: 10
nodeSelector:
kubernetes.io/role: agent
type: virtual-kubelet
os:
name: linux
restartPolicy: Always
tolerations:
- key: virtual-kubelet.io/provider
operator: Equal
value: saladcloud
effect: NoSchedule
2 changes: 1 addition & 1 deletion docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1

FROM golang:1.21.3-alpine3.18 AS build
FROM golang:1.21.4-alpine3.18 AS build
WORKDIR /app
ARG VERSION_FLAGS=-ldflags=

Expand Down
12 changes: 6 additions & 6 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ require (
github.com/prometheus/client_model v0.5.0
github.com/sirupsen/logrus v1.9.3
github.com/virtual-kubelet/virtual-kubelet v1.10.0
k8s.io/api v0.28.2
k8s.io/api v0.28.4
)

require (
Expand Down Expand Up @@ -37,7 +37,7 @@ require (
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-cmp v0.5.9 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/uuid v1.3.1
github.com/google/uuid v1.4.0
github.com/gorilla/mux v1.8.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 // indirect
Expand All @@ -56,7 +56,7 @@ require (
github.com/prometheus/client_golang v1.16.0 // indirect
github.com/prometheus/common v0.44.0 // indirect
github.com/prometheus/procfs v0.10.1 // indirect
github.com/spf13/cobra v1.7.0
github.com/spf13/cobra v1.8.0
github.com/spf13/pflag v1.0.5 // indirect
github.com/stoewer/go-strcase v1.2.0 // indirect
go.etcd.io/etcd/api/v3 v3.5.9 // indirect
Expand Down Expand Up @@ -89,15 +89,15 @@ require (
google.golang.org/genproto v0.0.0-20230526203410-71b5a4ffd15e // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230530153820-e85fd2cbaebc // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230530153820-e85fd2cbaebc // indirect
google.golang.org/grpc v1.55.0 // indirect
google.golang.org/grpc v1.56.3 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apimachinery v0.28.2
k8s.io/apimachinery v0.28.4
k8s.io/apiserver v0.28.1 // indirect
k8s.io/client-go v0.28.2 // indirect
k8s.io/client-go v0.28.4
k8s.io/component-base v0.28.1 // indirect
k8s.io/klog/v2 v2.100.1 // indirect
k8s.io/kms v0.28.1 // indirect
Expand Down
30 changes: 16 additions & 14 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.110.0 h1:Zc8gqp3+a9/Eyph2KDmcGaPtbKRIoqq4YTlL4NMD0Ys=
cloud.google.com/go/compute v1.19.0 h1:+9zda3WGgW1ZSTlVppLCYFIr48Pa35q1uG2N1itbCEQ=
cloud.google.com/go/compute v1.19.1 h1:am86mquDUgjGNWxiGn+5PGLbmgiWXlE/yNWpIpNvuXY=
cloud.google.com/go/compute/metadata v0.2.3 h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/NYTimes/gziphandler v1.1.1 h1:ZUDjpQae29j0ryrS0u/B8HZfJBtBQHjqw2rQ2cqUQ3I=
Expand All @@ -23,11 +23,12 @@ github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4 h1:/inchEIKaYC1Akx+H+gqO04wryn5h75LSazbRlnya1k=
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec=
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
Expand All @@ -39,6 +40,7 @@ github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymF
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/envoyproxy/protoc-gen-validate v0.10.1 h1:c0g45+xCJhdgFGw7a5QAfdS4byAbud7miNWJ1WwEVf8=
github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U=
github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww=
Expand Down Expand Up @@ -102,8 +104,8 @@ github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4=
github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
Expand Down Expand Up @@ -168,8 +170,8 @@ github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQD
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/soheilhy/cmux v0.1.5 h1:jjzc5WVemNEDTLwv9tlmemhC73tI08BNOIGwBOo10Js=
github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stoewer/go-strcase v1.2.0 h1:Z2iHWqGXH00XYgqDmNgQbIBxf3wrNq0F3feEy0ainaU=
Expand Down Expand Up @@ -314,8 +316,8 @@ google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyac
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.55.0 h1:3Oj82/tFSCeUrRTg/5E/7d/W5A1tj6Ky1ABAuZuv5ag=
google.golang.org/grpc v1.55.0/go.mod h1:iYEXKGkEBhg1PjZQvoYEVPTDkHo1/bjTnfwTeGONTY8=
google.golang.org/grpc v1.56.3 h1:8I4C0Yq1EjstUzUJzpcRVbuYA2mODtEmpWiQoN/b2nc=
google.golang.org/grpc v1.56.3/go.mod h1:I9bI3vqKfayGqPUAwGdOSu7kt6oIJLixfffKrpXqQ9s=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
Expand Down Expand Up @@ -347,15 +349,15 @@ gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
k8s.io/api v0.28.2 h1:9mpl5mOb6vXZvqbQmankOfPIGiudghwCoLl1EYfUZbw=
k8s.io/api v0.28.2/go.mod h1:RVnJBsjU8tcMq7C3iaRSGMeaKt2TWEUXcpIt/90fjEg=
k8s.io/api v0.28.4 h1:8ZBrLjwosLl/NYgv1P7EQLqoO8MGQApnbgH8tu3BMzY=
k8s.io/api v0.28.4/go.mod h1:axWTGrY88s/5YE+JSt4uUi6NMM+gur1en2REMR7IRj0=
k8s.io/apiextensions-apiserver v0.27.2 h1:iwhyoeS4xj9Y7v8YExhUwbVuBhMr3Q4bd/laClBV6Bo=
k8s.io/apimachinery v0.28.2 h1:KCOJLrc6gu+wV1BYgwik4AF4vXOlVJPdiqn0yAWWwXQ=
k8s.io/apimachinery v0.28.2/go.mod h1:RdzF87y/ngqk9H4z3EL2Rppv5jj95vGS/HaFXrLDApU=
k8s.io/apimachinery v0.28.4 h1:zOSJe1mc+GxuMnFzD4Z/U1wst50X28ZNsn5bhgIIao8=
k8s.io/apimachinery v0.28.4/go.mod h1:wI37ncBvfAoswfq626yPTe6Bz1c22L7uaJ8dho83mgg=
k8s.io/apiserver v0.28.1 h1:dw2/NKauDZCnOUAzIo2hFhtBRUo6gQK832NV8kuDbGM=
k8s.io/apiserver v0.28.1/go.mod h1:d8aizlSRB6yRgJ6PKfDkdwCy2DXt/d1FDR6iJN9kY1w=
k8s.io/client-go v0.28.2 h1:DNoYI1vGq0slMBN/SWKMZMw0Rq+0EQW6/AK4v9+3VeY=
k8s.io/client-go v0.28.2/go.mod h1:sMkApowspLuc7omj1FOSUxSoqjr+d5Q0Yc0LOFnYFJY=
k8s.io/client-go v0.28.4 h1:Np5ocjlZcTrkyRJ3+T3PkXDpe4UpatQxj85+xjaD2wY=
k8s.io/client-go v0.28.4/go.mod h1:0VDZFpgoZfelyP5Wqu0/r/TRYcLYuJ2U1KEeoaPa1N4=
k8s.io/component-base v0.28.1 h1:LA4AujMlK2mr0tZbQDZkjWbdhTV5bRyEyAFe0TJxlWg=
k8s.io/component-base v0.28.1/go.mod h1:jI11OyhbX21Qtbav7JkhehyBsIRfnO8oEgoAR12ArIU=
k8s.io/klog/v2 v2.100.1 h1:7WCHKK6K8fNhTqfBhISHQ97KrnJNFZMcQvKp7gP/tmg=
Expand Down

0 comments on commit aadeb44

Please sign in to comment.