The edge-api project is an API server for fleet edge management capabilities. The API server will provide Restful web services. This is a Golang project developed using Golang 1.15. Make sure you have at least this version installed.
Below you can see where the edge-api application sits in respect to the interaction with the user and the device at the edge to be managed.
┌──────────┐
│ USER │
└────┬─────┘
│
│
│
┌──────────────────────────────────────────────┼─────────────────────────────────────────────────────┐
│ │ │
│ cloud.redhat.com ┌────────▼──────────┐ │
│ │ 3-Scale │ │
│ │ API Management │ │
│ └────────┬──────────┘ │
│ │ │
│ │ │
│ │ │
│ ┌───────▼────────┐ │
│ │ edge-api │ │
│ │ application │ │
│ └───────┬────────┘ │
│ │ │
│ work requests │ playbook run results │
│ (playbooks) │ (ansible runner events) │
│ ┌─────────▼───────────┐ │
│ ┌───────────────────────┤ playbook-dispatcher │◄──────────────────────────────┐ │
│ │ │ application │ │ │
│ │ └─────────────────────┘ │ │
│ │ │ │
│ ┌────────▼────────┐ ┌─────┴────┐ │
│ │ cloud-connector │ │ ingress │ │
│ │ service │ │ service │ │
│ └────────┬────────┘ └─────▲────┘ │
│ │ │ │
└────────────┼─────────────────────────────────────────────────────────────────────────────┼─────────┘
│ │
┌────▼─────┐ │
│ MQTT │ │
│ BROKER │ │
└────┬─────┘ uploads │
│ (http) │
│ ┌───────────────────────────┐ │
│ │ │ │
│ │ Connected Host │ │
signals │ │ ┌───────────────────┐ │ │
(mqtt) │ │ │ RHC Client │ │ │
│ │ │ ┌───────────────┐ │ │ │
└────────────────────┼──►│ │playbook-worker│ ├───┼────────────────────────────┘
│ │ └────┬────▲─────┘ │ │
│ │ │ │ │ │
│ └──────┼────┼───────┘ │
│ │ │ │
│ ┌─▼────┴──┐ │
│ │ Ansible │ │
│ │ Runner │ │
│ └─────────┘ │
│ │
└───────────────────────────┘
Development of this project utilizes several tools listed below:
Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. You can install Git on your system if it's not already available using the following documentation.
Golang is the development code utilized by the edge-api application. You can get setup to develop with Golang by following the install documentation. The dependencies are handled by Go modules and specified in the go.mod file.
Python is only necessary to support the usage of Bonfire, which is used for deployment and testing of the edge-api application. It is recommended to use Python 3.6 with this project. While you may use the Python included with your Operating System you may also find tools like pyenv to be useful for maintaining multiple Python versions. Currently, the development dependencies are obtained using pipenv. Pipenv creates a Python virtual environment with the dependencies specified in the Pipfile and the virtual environment created is unique to this project, allowing different projects to have different dependencies.
Minikube provides a local single node Kubernetes cluster for development purposes. You can find setup information for minikube in the following Get Started! docs. Before starting your cluster you will need to make several configuration updates noted in the Clowder documentation.
Clowder is a kubernetes operator designed to make it easy to deploy applications running on the cloud.redhat.com platform in production, testing and local development environments. This operator normalizes how applications are configured with common interactions, from database to message queue and topics, to object storage. Clowder also helps define consistent mechanisms for driving integration tests with noted application dependencies and Job Invocations. Getting started with Clowder is quite simple using a single command to deploy the operator.
Bonfire is a CLI tool used to deploy ephemeral environments for testing cloud.redhat.com applications. bonfire interacts with a local configuration file to obtain applications' OpenShift templates, process them, and deploy them. There is a Pipfile in this repository that specifies bonfire as a dependency, and if you run the installation steps above, you will have it installed on your virtual environment.
Podman / Docker are used to build a container for edge-api that will run in Kubernetes / Red Hat OpenShift. Get started with Podman following this installation document. Get started with Docker following this installation document.
OpenShift CLI is used by Bonfire because of its templating capabitilies to generate the files that will be deployed to your local Kubernetes cluster. Follow the instructions on Installing the OpenShift CLI to install it in your machine.
For these steps, only git and Docker are necessary from the information above. Keep in mind that you might need to run inside of a Kubernetes cluster if you want to test more complex use-case scenarios.
- Clone the project.
git clone [email protected]:RedHatInsights/edge-api.git
- Change directories to the project.
cd edge-api
- Run the migrations to create the database schema (this will download dependencies).
go run cmd/migrate/migrate.go
- Run the project in debug mode. Debug mode allows unauthenticated calls and it's essential for local development.
DEBUG=true go run main.go
- Open another terminal and make sure you can reach the API.
curl -v http://localhost:3000/
If you find an error message saying that you don't have gcc installed, install it.
Keep in mind that if you change the models you might need to run the migrations again. By default, Edge API will run with a sqllite database that will be created in the first run. If you want to use your own postgresql container (which is particularly good for corner cases on migrations and queries), you can do this by:
- Run the dabatase (example with podman)
podman run -d --name postgresql_database -e POSTGRESQL_USER=user -e POSTGRESQL_PASSWORD=pass -e POSTGRESQL_DATABASE=db -p 5432:5432 rhel8/postgresql-10
- Add some environment variables
DATABASE=pgsql
PGSQL_USER=user
PGSQL_PASSWORD=pass
PGSQL_HOSTNAME=127.0.0.1
PGSQL_PORT=5432
PGSQL_DATABASE=db
Following the information above you should have Docker or Podman, a minikube cluster running with Clowder installed, and a Python environment with bonfire installed. Now move on to running the edge-api application.
- Clone the project.
git clone [email protected]:RedHatInsights/edge-api.git
- Change directories to the project.
cd edge-api
- Setup your Python virtual environment.
pipenv install --dev
- Enter the Python virtual environment to enable access to Bonfire.
pipenv shell
- Setup access to the Docker enviroment within minikube, so you can build images directly to the cluster's registry.
eval $(minikube -p minikube docker-env)
- Build the container image.
make build
- Create Bonfire configuration. To deploy from your local repository run the following:
make bonfire-config-local
The above command will create a file named default_config.yaml pointing to your local repository. At times you may need to update the branch which is referred to with the ref parameter (defaults to main).
Bonfire can also deploy from GitHub. Running the following command will setup the GitHub based configuration:
make bonfire-config-github
- Setup test namespace for deployment.
make create-ns NAMESPACE=test
- Deploy a Clowder environment (ClowdEnviroment) to the namespace with bonfire.
make deploy-env NAMESPACE=test
- Deploy the application to the namespace.
make deploy-app NAMESPACE=test
Now the application should be running. You can test this by port-forwarding the app in one terminal and running a curl command in another as follows:
Terminal 1
kubectl -n test port-forward service/edge-api-service 8000:8000
Terminal 2
curl -v http://localhost:8000/
You should get a 200 response back.
Now you can build and deploy the application.
Once a code change has been performed you can rebuild the container in the minikube registry:
make build
Then scale down and up the deployment to pick up the image change:
make restart-app NAMESPACE=test
This project makes use of Golang's built in lint and vet capabilites. You can run these with the following commands:
lint:
make lint
vet:
make vet
Golang also provides a unit test infrastructure test. Run unit tests with the following command:
make test
go-swagger is a tool that, amongst other things, parses comments in the code to generate a file in the Open API 2.0 spec.
To generate the API docs, you need to install go-swagger. Then you can use the following command:
make generate-docs
The go-swagger docs have several examples on how to annotate the code and this example is useful when it comes to understanding all the possiblities.
The API will serve the docs under a /docs endpoint.