Hkube

Hkube - cloud-native open source framework to run distributed pipeline of algorithms

TL;DR;

$ helm repo add hkube https://hkube.org/helm
$ helm repo update
$ helm install hkube hkube/hkube  --version=0.1.2

Introduction

This chart deploys on a Kubernetes cluster using the Helm package manager.

Prerequisites

Hkube requires a working installation of kubernetes.
There are many options for local k8s installations. e.g Minikube or microk8s.

Hkube is installed using helm. Both helm 2 and the newer helm 3 should work.

Make sure kubectl is configured to your cluster.
For collecting algorithm logs, and to create builds, Hkube requires that certain pods will run in privileged security permissions. Consult your kubernetes installation to see how to do that. "--allow-privileged flag must be set to true for both the API server and the kubelet (or set global.isPrivileged=false)"

• Kubernetes v1.14+
• Dynamic storage provisioning (optional)
• Ingress controller (optional)

Installing the Chart

To install the chart with the release name hkube:

$ helm repo add hkube http://hkube.org/helm/
$ helm install hkube hkube/hkube

The command deploys on the Kubernetes cluster in the default configuration. The configuration section lists the parameters that can be configured during installation.

Tip: List all releases using helm list

Uninstalling the Chart

To uninstall/delete the hkube:

$ helm delete hkube 

The command removes all the Kubernetes components associated with the chart and deletes the release.

Configuration

The following table lists the configurable parameters of the hkube chart and their default values.

Parameter	Description	Default
global.clusterName	kubernetes cluster domain name	cluster.local
global.registry	private registry to use	""
global.registry_namespace	private registry namespace for pulling images in builds	hkube
global.registry_username	private registry username for pulling images	""
global.registry_password	private registry password for pulling images	""
global.registry_insecure	private registry insecure (http) for pulling base image in builds	false
global.registry_skip_tls_verify	private registry with self signed certificate for pulling base image in builds	false
global.image_pull_secret.name	name of imagepullsecret. If global.registry_password is not empty, it will be created	hkube-imagepullsecret
global.image_pull_secret.use_existing	use an existing imagepullsecret (doesn't create the secret)	false
global.resources_enable	If false, will not set resource request and limits on hkube pods. If true will set the values for each pod	false
global.storage.encoding	encoding for storage. can be msgpack, bson, json	msgpack
global.storage.maxCacheSize		500
global.storage.s3.access_key	s3 access and secret key. If using internal minio, must match minio.accessKey and secretKey	hkubeminiokey
global.storage.s3.secret_key		hkubeminiosecret
global.storage.s3.url		"http://hkube-minio:9000"
global.k8senv	kubernetes environment. Possible values are k8s, openshift	k8s
global.namespaced	use only namespace. Set to true in restricted environments such as openshift	false
global.isPrivileged	run worker pods as privileged containers. Makes log aggregation easier	true
global.ingress.hostname		""
global.ingress.prefix		""
global.ingress.use_regex	set to false if using nginx-ingress-controller prior to 0.21	true
global.ingress.requireTls		false
global.ingress.maxBodySize		500m
global.ingress.class		nginx
global.sidecars.fluent_bit.enable		false
global.sidecars.fluent_bit.config.name		fluent-bit-cm
global.sidecars.fluent_bit.config.create_cm		true
global.sidecars.fluent_bit.config.log_file		"*.log"
global.sidecars.fluent_bit.config.mount_path		/mnt/log
global.sidecars.fluent_bit.config.es.url		""
global.sidecars.fluent_bit.config.es.port		"9200"
global.sidecars.fluent_bit.config.es.tls		"Off"
global.sidecars.fluent_bit.config.es.tls_verify		"Off"
global.sidecars.fluent_bit.config.es.logStashPrefix		"hkube-"
global.sidecars.fluent_bit.config.es.user		""
global.sidecars.fluent_bit.config.es.password		""
global.sidecars.fluent_bit.resources.limits.cpu		100m
global.sidecars.fluent_bit.resources.limits.memory		200Mi
env.default_storage	storage mode. can be s3 or fs. Set global.storage.fs or global.storage.s3 accordingly	s3
env.node_env		production
image.pullPolicy	pull policy for images	IfNotPresent
etcd.enable		true
etcd.replicas	set number of replicas. defualt 3	3
etcd.persistentVolume.enabled		true
etcd.persistentVolume.storageClass		""
etcd.persistentVolume.storage		10Gi
etcd.resources		{}
redis-ha.enable		true
redis-ha.replicas	set number of replicas. defualt 3. If set to 1 also set min-slaves-to-write to 0	3
redis-ha.redis.config.min-slaves-to-write		1
redis-ha.redis.config.client-output-buffer-limit		"replica 1024mb 512mb 60"
redis-ha.redis.resources		{}
redis-ha.sentinel.resources		{}
redis-ha.hardAntiAffinity	We usually recommend not to specify default resources and to leave this as a conscious choice for the user. This also increases chances charts run on environments with little resources, such as Minikube. If you do want to specify resources, uncomment the following lines, adjust them as necessary, and remove the curly braces after 'resources:'. limits: cpu: 300m memory: 512Mi requests: cpu: 100m memory: 256Mi set to true for multiple nodes	false
jaeger.enable	enable jaeger and cassandra	false
jaeger.fullnameOverride		jaeger
jaeger.nameOverride		jaeger
jaeger.query.basePath		/jaeger
jaeger.query.ingress.enabled		true
jaeger.agent.deploymentKind	set the deployment kind of the jaeger-agent. options are daemonset and deployment	daemonset
jaeger.agent.deployment.replicas		1
jaeger.agent.daemonset.useHostPort	set to false to allow running in non privileged environment (e.g. openshift)	true
jaeger.cassandra.persistence.enabled		true
jaeger.cassandra.persistence.storageClass	cassandra data Persistent Volume Storage Class If defined, storageClassName: If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. (gp2 on AWS, standard on GKE, AWS & OpenStack)	null
jaeger.cassandra.config.cluster_size		1
jaeger.cassandra.config.cluster_domain		cluster.local
jaeger.external_agent_address	optional external jaeger agent address. leave blank for defaults	""
nginx-ingress.enable	install nginx-ingress-controller with hkube	false
mongodb.enable		true
mongodb.persistence.storageClass		null
mongodb.useStatefulSet		true
mongodb.auth.rootPassword		ehtyTJj4mR
mongodb.auth.user_name		root
mongodb.auth.databaseName		hkube
mongodb.podSecurityContext.enabled		false
mongodb.containerSecurityContext.enabled		false
minio.enable		true
minio.accessKey		hkubeminiokey
minio.secretKey		hkubeminiosecret
minio.persistence.enabled		true
minio.persistence.storageClass		""
minio.resources		{}
gitea.enable		true
gitea.persistence.storageClass		null
gitea.service.http.port		"3000"
gitea.gitea.config.repository.ENABLE_PUSH_CREATE_USER		true
gitea.gitea.admin.username		hkube
gitea.gitea.admin.password		"123456"
gitea.gitea.admin.email		"[email protected]"
gitea.postgresql.persistence.storageClass		null
gitea.postgresql.securityContext.enabled		false
gitea.postgresql.containerSecurityContext.enabled		false
gitea.memcached.securityContext.enabled		false
metrics.serviceMonitorSelector.release		metric
metrics.grafanaDashboardLabel.grafana_dashboard		"1"
metrics.grafana_version	set to '7' when use Grafana version >= 7, default '6'	"6"
systemversion		v2.5.24
build_secret.docker_registry	docker registry for algorithm builds. defaults to docker.io	""
build_secret.docker_namespace	docker registry namespace for algorithm builds. defaults to username	""
build_secret.docker_username	docker registry username for algorithm builds.	""
build_secret.docker_password	docker registry password for algorithm builds.	""
build_secret.docker_registry_insecure	docker registry is insecure for algorithm builds.	false
build_secret.docker_registry_skip_tls_verify	docker registry has self signed certificate for algorithm builds.	false
build_secret.npm_registry	npm registry for algorithm builds. Defaults to npmjs.com	""
build_secret.npm_token	npm registry token for algorithm builds. Mutually exclusive with npm_user and npm_password	""
build_secret.npm_user	npm registry user for algorithm builds. Mutually exclusive with npm_token	""
build_secret.npm_password	npm registry password for algorithm builds. Mutually exclusive with npm_token	""
build_secret.pip_registry	pip registry for algorithm builds. Defaults to pypi.org	""
build_secret.pip_token	pip registry token for algorithm builds. Defaults to pypi.org	""
build_secret.maven_registry		""
build_secret.maven_token		""
build_secret.maven_user		""
nodeSelector	node selector for hkube containers.	{}
site.env		{}
site.image.repository		hkube/hkube-site
site.image.tag		v1.0.1
site.ingress.enabled		true
site.ingress.path		hkube/site
site.service.port		80
site.service.type		ClusterIP
site.resources.limits.cpu		200m
site.resources.limits.memory		512Mi
site.resources.requests.cpu		100m
site.resources.requests.memory		256Mi
algorithm_operator.build_mode	build mode for automatic builds. set to openshift if running on openshift	kaniko
algorithm_operator.resources_enable	enable resource limits for algorithm operator	false
algorithm_operator.resources	Compute resources required for algorithm-operator	{"limits":{"cpu":"300m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"512Mi"}}
api_gateway.ingress.path		/hkube/api-server
api_gateway.ingress.enabled		true
task_executor.env.worker_socket_max_payload_bytes		1000000000
task_executor.env.worker_algorithm_start_timeout_ms		300000
task_executor.env.created_jobs_ttl	interval in MS, in which we verify if a job was deployed	15000
task_executor.env.worker_memory	worker container memory request in MB. Only set if resources_enable=true	512
task_executor.env.worker_cpu	worker container cpu request. Only set if resources_enable=true	0.1
task_executor.env.resources_enable	if true, sets resource requests on worker container	false
task_executor.env.interval	task executor internal processing interval in MS	"1000"
task_executor.env.default_quota_cpu	set to the number of CPUs in the cluster (or quota) when running on openshift	30
task_executor.env.default_quota_mem	set to the total amount of memory in the cluster (or quota) when running on openshift	20Gi
task_executor.env.default_quota_gpu	set to the total number of GPUs in the cluster (or quota) when running on openshift	0
task_executor.env.has_node_list	if running on openshift on labeled nodes, set to true and create a config-map named hkube-nodes with key hkube-nodes and value of the output of kubectl get nodes -o json	false
task_executor.env.use_resource_limits	set to true to set algorithm resource limit to equal resource request	false
task_executor.env.pods_output_log_folder	path where kubernetes stores the pods output logs.	'/var/lib/docker/containers'
task_executor.resources_enable		false
task_executor.resources		{"limits":{"cpu":"300m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"512Mi"}}
fullSystemVersion		v2.5.24-1678712208396

Specify each parameter using the --set key=value[,key=value] argument to helm install. For example:

$ helm install hkube hkube/hkube  --version=0.1.2 --set global.clusterName=cluster.local

Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example:

$ helm install hkube hkube/hkube  --version=0.1.2 --values values.yaml