Skip to content

Commit

Permalink
Merge pull request #137 from grycap/dev-srisco
Browse files Browse the repository at this point in the history 
Parameters to configure Knative autoscaling bounds
  • Loading branch information
@srisco
srisco authored Apr 6, 2022
2 parents d4398e5 + fc3114c commit bd81f4e
Show file tree
Hide file tree
Showing 11 changed files with 616 additions and 320 deletions.
8 changes: 5 additions & 3 deletions deploy/ansible/oscar-k3s.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@
kube_install_nfs_client: true
kube_deploy_dashboard: true
kube_nfs_server: '{{ ansible_host }}.localdomain'
kube_version: "1.23.5"
kube_cert_manager: true
kube_public_dns_name: '{{ dns_host }}'
kube_apiserver_options:
Expand All @@ -64,17 +65,17 @@
public_hostname_api: "{{ minio_dns_host }}"
public_hostname_console: "{{ minio_dns_host_console }}"
- role: 'grycap.kubefaas'
faas_framework: 'knative'
knative_version: "1.3.0"
master_deploy: true
functions_namespace: oscar-svc
- role: 'grycap.kubeoscar'
oscar_pass: '{{ oscar_password }}'
create_ingress: true
cert_manager_issuer: 'letsencrypt-prod'
minio_endpoint: 'https://{{ minio_dns_host }}'
minio_secret_key: '{{ minio_password }}'
master_deploy: true
openfaas_scaler_enable: true
serverless_backend: "openfaas"
serverless_backend: "knative"

- name: Configure WN(s)
hosts: wn
Expand All @@ -91,5 +92,6 @@
- role: grycap.kubernetes
kube_install_method: k3s
kube_type_of_node: 'wn'
kube_version: "1.23.5"
kube_server: "{{ groups['front'][0] }}"
kube_api_server: "{{ groups['front'][0] }}"
9 changes: 9 additions & 0 deletions docs/api.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,15 @@ components:
type: string
total_cpu:
type: string
synchronous:
type: object
properties:
min_scale:
type: integer
default: 0
max_scale:
type: integer
default: 0
token:
type: string
readOnly: true
Expand Down
40 changes: 24 additions & 16 deletions docs/fdl.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,22 +55,30 @@ storage_providers:

## Service

| Field | Description |
| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `name` </br> *string* | The name of the service |
| `image` </br> *string* | Docker image for the service |
| `alpine` </br> *boolean* | Alpine parameter to set if image is based on Alpine. If `true` a custom release of faas-supervisor will be used. Optional (default: false) |
| `script` </br> *string* | Local path to the user script to be executed in the service container |
| `memory` </br> *string* | Memory limit for the service following the [kubernetes format](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory). Optional (default: 256Mi) |
| `cpu` </br> *string* | CPU limit for the service following the [kubernetes format](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu). Optional (default: 0.2) |
| `total_memory` </br> *string* | Limit for the memory used by all the service's jobs running simultaneously. Apache YuniKorn scheduler is required to work. Same format as Memory, but internally translated to MB (integer). Optional (default: "") |
| `total_cpu` </br> *string* | Limit for the virtual CPUs used by all the service's jobs running simultaneously. Apache YuniKorn scheduler is required to work. Same format as CPU, but internally translated to millicores (integer). Optional (default: "") |
| `log_level` </br> *string* | Log level for the FaaS Supervisor. Available levels: NOTSET, DEBUG, INFO, WARNING, ERROR and CRITICAL. Optional (default: INFO) |
| `input` </br> *[StorageIOConfig](#storageioconfig) array* | Array with the input configuration for the service. Optional |
| `output` </br> *[StorageIOConfig](#storageioconfig) array* | Array with the output configuration for the service. Optional |
| `environment` </br> *[EnvVarsMap](#envvarsmap)* | The user-defined environment variables assigned to the service. Optional |
| `annotations` </br> *map[string]string* | User-defined Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to be set in job's definition. Optional |
| `labels` </br> *map[string]string* | User-defined Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to be set in job's definition. Optional |
| Field | Description |
| ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `name` </br> *string* | The name of the service |
| `image` </br> *string* | Docker image for the service |
| `alpine` </br> *boolean* | Alpine parameter to set if image is based on Alpine. If `true` a custom release of faas-supervisor will be used. Optional (default: false) |
| `script` </br> *string* | Local path to the user script to be executed in the service container |
| `memory` </br> *string* | Memory limit for the service following the [kubernetes format](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory). Optional (default: 256Mi) |
| `cpu` </br> *string* | CPU limit for the service following the [kubernetes format](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu). Optional (default: 0.2) |
| `total_memory` </br> *string* | Limit for the memory used by all the service's jobs running simultaneously. Apache YuniKorn scheduler is required to work. Same format as Memory, but internally translated to MB (integer). Optional (default: "") |
| `total_cpu` </br> *string* | Limit for the virtual CPUs used by all the service's jobs running simultaneously. Apache YuniKorn scheduler is required to work. Same format as CPU, but internally translated to millicores (integer). Optional (default: "") |
| `synchronous` </br> *[SynchronousSettings](#synchronoussettings)* | Struct to configure specific sync parameters. This settings are only applied on Knative ServerlessBackend. Optional. |
| `log_level` </br> *string* | Log level for the FaaS Supervisor. Available levels: NOTSET, DEBUG, INFO, WARNING, ERROR and CRITICAL. Optional (default: INFO) |
| `input` </br> *[StorageIOConfig](#storageioconfig) array* | Array with the input configuration for the service. Optional |
| `output` </br> *[StorageIOConfig](#storageioconfig) array* | Array with the output configuration for the service. Optional |
| `environment` </br> *[EnvVarsMap](#envvarsmap)* | The user-defined environment variables assigned to the service. Optional |
| `annotations` </br> *map[string]string* | User-defined Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to be set in job's definition. Optional |
| `labels` </br> *map[string]string* | User-defined Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to be set in job's definition. Optional |

## SynchronousSettings

| Field | Description |
| --------------------------- | -------------------------------------------------------------------------------------------- |
| `min_scale` </br> *integer* | Minimum number of active replicas (pods) for the service. Optional. (default: 0) |
| `max_scale` </br> *integer* | Maximum number of active replicas (pods) for the service. Optional. (default: 0 (Unlimited)) |

## StorageIOConfig

Expand Down
2 changes: 1 addition & 1 deletion docs/invoking.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ As detailed in the [API specification](api.md), invocation paths require the ser

## Synchronous invocations

Synchronous invocations allow obtaining the execution output as the response to the HTTP call to the `/run/<SERVICE_NAME>` path. For this, OSCAR delegates the execution to a Serverless Backend (currently only [OpenFaaS](https://www.openfaas.com/) is supported, but we plan to integrate [Knative](https://knative.dev) in the near future). Unlike asynchronous invocations that are translated into Kubernetes jobs, synchronous invocations use a "function" pod to handle requests. This is possible thanks to the [OpenFaaS Watchdog](https://github.com/openfaas/of-watchdog), which is injected into each service and is in charge of forking the process to be executed for each request received.
Synchronous invocations allow obtaining the execution output as the response to the HTTP call to the `/run/<SERVICE_NAME>` path. For this, OSCAR delegates the execution to a Serverless Backend ([Knative](https://knative.dev) or [OpenFaaS](https://www.openfaas.com/)). Unlike asynchronous invocations, that are translated into Kubernetes jobs, synchronous invocations use a "function" pod to handle requests. This is possible thanks to the [OpenFaaS Watchdog](https://github.com/openfaas/classic-watchdog), which is injected into each service and is in charge of forking the process to be executed for each request received.

![oscar-sync.png](images/oscar-sync.png)

Expand Down
42 changes: 41 additions & 1 deletion docs/local-testing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,46 @@ helm install nfs-server-provisioner nfs-ganesha-server-and-external-provisioner/

*Some Linux distributions may have [problems](https://github.com/kubernetes-sigs/kind/issues/1487#issuecomment-694920754) using the [NFS server provisioner](https://github.com/kubernetes-sigs/nfs-ganesha-server-and-external-provisioner) with kind due to its default configuration of kernel-limit file descriptors. To workaround it, please run `sudo sysctl -w fs.nr_open=1048576`.*

### Deploy Knative Serving as Serverless Backend (OPTIONAL)

OSCAR supports [Knative Serving](https://knative.dev/docs/serving/) as Serverless Backend to process [synchronous invocations](invoking.md#synchronous-invocations). If you want to deploy it in the kind cluster, first you must deploy the [Knative Operator](https://knative.dev/docs/install/operator/knative-with-operators/)

```
kubectl apply -f https://github.com/knative/operator/releases/download/knative-v1.3.1/operator.yaml
```

*Note that the above command deploys the version `v1.3.1` of the Operator. You can check if there are new versions [here](https://github.com/knative/operator/releases).*

Once the Operator has been successfully deployed, you can install the Knative Serving stack with the following command:

```
cat <<EOF | kubectl apply -f -
---
apiVersion: v1
kind: Namespace
metadata:
name: knative-serving
---
apiVersion: operator.knative.dev/v1beta1
kind: KnativeServing
metadata:
name: knative-serving
namespace: knative-serving
spec:
version: 1.3.0
ingress:
kourier:
enabled: true
service-type: ClusterIP
config:
config-features:
kubernetes.podspec-persistent-volume-claim: enabled
kubernetes.podspec-persistent-volume-write: enabled
network:
ingress-class: "kourier.ingress.networking.knative.dev"
EOF
```

### Deploy OSCAR

First, create the `oscar` and `oscar-svc` namespaces by executing:
Expand All @@ -83,7 +123,7 @@ First, create the `oscar` and `oscar-svc` namespaces by executing:
kubectl apply -f https://raw.githubusercontent.com/grycap/oscar/master/deploy/yaml/oscar-namespaces.yaml
```

Then, add the [grycap helm repo](https://github.com/grycap/helm-charts) and deploy by running the following commands replacing `<OSCAR_PASSWORD>` with a password of your choice and `<MINIO_PASSWORD>` with the MinIO rootPassword:
Then, add the [grycap helm repo](https://github.com/grycap/helm-charts) and deploy by running the following commands replacing `<OSCAR_PASSWORD>` with a password of your choice and `<MINIO_PASSWORD>` with the MinIO rootPassword, and remember to add the flag `--set serverlessBackend=knative` if you deployed it in the previous step:

```sh
helm repo add grycap https://grycap.github.io/helm-charts/
Expand Down
Loading

0 comments on commit bd81f4e

Please sign in to comment.