🛑 Changes to this repository will no longer affect current HeLx 🛑
Tycho is an API, compiler, and executor for cloud native distributed systems.
- A subset of docker-compose is the system specification syntax.
- Kubernetes is the first supported orchestrator.
- Application Simplity: The Kubernetes API is reliable, extensive, and well documented. It is also large, complex, supports a range of possibilities greater than many applications need, and often requires the creation and control of many objects to execute comparatively simple scenarios. Tycho bridges the simplicity of Compose to the richness of the Kubernetes' architecture.
- Microservice: We wanted an end to end Python 12-factory style OpenAPI microservice that fits seamlessly into a Python ecosystem (which is why we did not use the excellent Kompose tool as a starting point).
- Lifecycle Management: Tycho treats distributed systems as programs whose entire lifecycle can be programmatically managed via an API.
- Pluggable Orchestrators: The Tycho compiler abstracts clients from the orchestrator. It creates an abstract syntax tree to model input systems and generates orchestrator specific artifacts.
- Policy: Tycho now generates network policy configurations governing the ingress and egress of traffic to systems. We anticipate generalizing the policy layer to allow security and other concerns to be woven into a deployment dynamically.
This work relies on these foundations:
- PIVOT: A cloud agnostic scheduler with an API for executing distributed systems.
- Kubernetes: Widely deployed, highly programmable, horizontally scalable container orchestration platform.
- Kompose: Automates conversion of Docker Compose to Kubernetes. Written in Go, does not provide an API. Supports Docker Compose to Kubernetes only.
- Docker: Pervasive Linux containerization tool chain enabling programmable infrastructure and portability.
- Docker-compose: Syntax and tool chain for executing distributed systems of containers.
- Docker Swarm: Docker only container orchestration platform with minimal adoption.
Github Actions are employed to test and publish development and main releases of tycho to pypi. These releases follow SemVer ('Major', 'Minor', 'Patch') versioning.
To create a main/master pypi package for tycho, the VERSION in tycho/__init__.py will need to be updated by the developer to the desired stable release version number.
If testing in the develop branch, editing the tycho/__init__.py file will NOT be necessary to generate a pypi package build, as the pypi-dev-upload.yml workflow will create a new tag based on day and time for your testing purposes which is uploaded upon each push to the develop branch. This ".dev" tag does not affect the develop branch code at all.
This means that a pr from feature branch to develop branch results in an automatic pypi build. If on the same day, a change to the develop branch occurs, then a new build is also generated with a differing ".dev" tag similar to tycho-api:1.12.0.dev20230221030806.
To locate the ".dev" tagged pypi build, navigate to the corresponding workflow run in the Github Actions tab, called build-dev-to-pypi then click the dropdown for Publish Package to Pypi and the link to the package will be provided within. The .dev packages are not searchable in Pypi as this would distract from stable packages of the same name and cause confusion - see pep 440.
- git clone https://github.com/helxplatform/tycho.git --branch branch_name
- python3 -m venv /path/to/venv - could be any path
- source /path/to/venv/bin/activate
- pip install -r /tycho/requirements.txt
- export PYTHONPATH={PYTHONPATH}:/path/to/tycho/
- python /tycho/tycho/api.py -d
samples/jupyter-ds/docker-compose.yaml:
---
# Docker compose formatted system.
version: "3"
services:
jupyter-datascience:
image: jupyter/datascience-notebook
entrypoint: start.sh jupyter lab --LabApp.token=
ports:
- 8888:8888
In one shell, run the API:
$ export PATH=~/dev/tycho/bin:$PATH
$ tycho api --debug
In another shell, launch three notebook instances.
$ export PATH=~/dev/tycho/bin:$PATH
$ tycho up -f sample/jupyter-ds/docker-compose.yaml
SYSTEM GUID PORT
jupyter-ds 909f2e60b83340cd905ae3865d461156 32693
$ tycho up -f sample/jupyter-ds/docker-compose.yaml
SYSTEM GUID PORT
jupyter-ds 6fc07ab865d14c4c8fd2d6e0380b270e 31333
$ tycho up -f sample/jupyter-ds/docker-compose.yaml
SYSTEM GUID PORT
jupyter-ds 38f01c140f0141d9b4dc1baa33960362 32270
Then make a request to each instance to show it's running. It may take a moment for the instances to be ready, especially if you're pulling a container for the first time.
$ for p in $(tycho status | grep -v PORT | awk '{ print $4 }'); do
url=http://$(minikube ip):$p; echo $url; wget -q -O- $url | grep /title;
done
http://192.168.99.111:32270
<title>JupyterLab</title>
http://192.168.99.111:31333
<title>JupyterLab</title>
http://192.168.99.111:32693
<title>JupyterLab</title>
Delete all running deployments.
$ tycho down $(tycho status --terse)
38f01c140f0141d9b4dc1baa33960362
6fc07ab865d14c4c8fd2d6e0380b270e
909f2e60b83340cd905ae3865d461156
And show that they're gone
$ tycho status
None running
- Install python 3.7.x or greater.
- Create a virtual environment.
- Install the requirements.
- Start the server.
python3 -m venv environmentName
source environmentName/bin/activate
pip install -r requirements.txt
export PATH=<tycho-repo-dir>/bin:$PATH
tycho api
This mode uses a local minikube instance with Tycho running outside of Minikube. This is the easiest way to add and test new features quickly.
Run minikube:
minikbue start
Run the minikube dashboard:
minikube dashboard
Run the Tycho API:
cd tycho
PYTHONPATH=$PWD/.. python api.py
Launch the Swagger interface http://localhost:5000/apidocs/.
Use the Tycho CLI client as shown above or invoke the API.
When we deploy Tycho into Minikube it is now able to get its Kubernetes API configuration from within the cluster.
In the repo's kubernetes directory, we define deployment, pod, service, clusterrole, and clusterrolebinding models for Tycho. The following interaction shows deploying Tycho into Minikube and interacting with the API.
We first deploy all Kubernetes Tycho-api artifacts into Minkube:
(tycho) [scox@mac~/dev/tycho/tycho]$ kubectl create -f ../kubernetes/
deployment.extensions/tycho-api created
pod/tycho-api created
clusterrole.rbac.authorization.k8s.io/tycho-api-access created
clusterrolebinding.rbac.authorization.k8s.io/tycho-api-access created
service/tycho-api created
Then we use the client as usual.
Starting out, Tycho's not running on the cluster:
First deploy the Tycho API
$ kubectl create -f ../kubernetes/
deployment.extensions/tycho-api created
pod/tycho-api created
clusterrole.rbac.authorization.k8s.io/tycho-api-access created
clusterrolebinding.rbac.authorization.k8s.io/tycho-api-access created
service/tycho-api created
Note, here we've edited the Tycho service def to create the service as type:LoadBalancer for the purposes of a command line demo. In general, we'll access the service from within the cluster rather than exposing it externally.
That runs Tycho:
Initialize the Tycho API's load balancer IP and node port.
$ lb_ip=$(kubectl get svc tycho-api -o json | jq .status.loadBalancer.ingress[0].ip | sed -e s,\",,g)
$ tycho_port=$(kubectl get service tycho-api --output json | jq .spec.ports[0].port)
Launch an application (deployment, pod, service). Note the --command flag is used to specify the command to run in the container. We use this to specify a flag that will cause the notebook to start without prompting for authentication credentials.
$ PYTHONPATH=$PWD/.. python client.py --up -n jupyter-data-science-3425 -c jupyter/datascience-notebook -p 8888 --command "start.sh jupyter lab --LabApp.token='
'"
200
{
"status": "success",
"result": {
"containers": {
"jupyter-data-science-3425-c": {
"port": 32414
}
}
},
"message": "Started system jupyter-data-science-3425"
}
Refreshing the GKE cluster monitoring UI will now show the service starting:
Then running
Get the job's load balancer ip and make a request to test the service.
$ job_lb_ip=$(kubectl get svc jupyter-data-science-3425 -o json | jq .status.loadBalancer.ingress[0].ip | sed -e s,\",,g)
$ wget --quiet -O- http://$job_lb_ip:8888 | grep -i /title
<title>Jupyter Notebook</title>
From a browser, that URL takes us directly to the Jupyter Lab IDE:
And shut the service down:
$ PYTHONPATH=$PWD/.. python client.py --down -n jupyter-data-science-3425 -s http://$lb_ip:$tycho_port
200
{
"status": "success",
"result": null,
"message": "Deleted system jupyter-data-science-3425"
}
This removes the deployment, pod, service, and replicasets created by the launcher.
Using the command lines above without the -s flag for server will work on GKE. That is, the client is created by first using the K8s API to locate the Tycho-API endpoint and port. It builds the URL automatically and creates a TychoAPI object ready to use.
client_factory = TychoClientFactory ()
client = client_factory.get_client ()
The "proxy_rewrite" feature ensures system-wide consistency in handling service
locations, especially when interacting with higher-level reverse proxies. By def
ining annotations in service.yaml, Ambassador's behavior is tailored, allowing
the underlying service to perceive an altered path while maintaining a consistent
location view at the system level.
- context.py: Processes external specifications, capturing "proxy_rewrite" directives, and transforms them into an internal representation.
- model.py: Forms the structural foundation of the system, accurately reflecting the "proxy_rewrite" configurations and their implications.
- service.yaml: Serves as a template for Kubernetes service definitions. When interpreted, it influences Ambassador's behavior using "proxy_rewrite" annotations, ensuring path and location consistency across the system.