-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: add more cloud tutorials and refactor a bit (#104)
- Loading branch information
Showing
12 changed files
with
369 additions
and
15 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
{ | ||
"label": "How-To Guides", | ||
"position": 20 | ||
"position": 30 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,132 @@ | ||
--- | ||
title: "Install Fluvio Cluster in Docker" | ||
menu: "Use Fluvio in Docker" | ||
weight: 11 | ||
description: "Install Fluvio Cluster with Docker Compose" | ||
--- | ||
|
||
# Docker Compose + Fluvio | ||
|
||
You can run Fluvio Clusters in a Docker Compose setup, for a local Proof of Concept or | ||
development. | ||
|
||
To run a Fluvio Cluster through Docker, you will need to run Fluvio | ||
components separately, we can use Docker Compose `service`s to achieve this. | ||
|
||
## Services | ||
|
||
- `sc`: Streaming Controller | ||
- `sc-setup`: Post-Initialization Commands | ||
- `spu`: Streaming Processing Unit | ||
|
||
> To learn more about Fluvio architecture, please refer to [Fluvio Documentation][1] | ||
## Running Locally | ||
|
||
Copy the docker-compose.yaml and Dockerfile to a directory. | ||
|
||
> Or clone the Fluvio repo `git clone https://github.com/infinyon/fluvio.git` and | ||
cd into `./fluvio/examples/docker-compose` | ||
|
||
`docker-compose.yaml`: | ||
|
||
```yaml | ||
version: "3" | ||
services: | ||
sc: | ||
image: infinyon/fluvio:stable | ||
container_name: sc | ||
hostname: sc | ||
ports: | ||
- "9103:9003" | ||
environment: | ||
- RUST_LOG=info | ||
command: "./fluvio-run sc --local /fluvio/metadata" | ||
volumes: | ||
- ./fluvio-metadata:/fluvio/metadata | ||
sc-setup: | ||
build: | ||
context: . | ||
dockerfile: Dockerfile | ||
container_name: sc-setup | ||
environment: | ||
- RUST_LOG=info | ||
entrypoint: > | ||
/bin/sh -c " | ||
fluvio profile add docker sc:9003 docker; | ||
fluvio cluster spu register --id 5001 -p 0.0.0.0:9110 --private-server spu:9011; | ||
exit 0; | ||
" | ||
depends_on: | ||
- sc | ||
spu: | ||
image: infinyon/fluvio:stable | ||
container_name: spu | ||
hostname: spu | ||
volumes: | ||
- ./fluvio-data:/fluvio/data | ||
environment: | ||
- RUST_LOG=info | ||
ports: | ||
- "9110:9010" | ||
- "9111:9011" | ||
command: "./fluvio-run spu -i 5001 -p spu:9010 -v spu:9011 --sc-addr sc:9004 --log-base-dir /fluvio/data" | ||
depends_on: | ||
- sc | ||
|
||
``` | ||
|
||
`Dockerfile`: | ||
|
||
``` | ||
FROM ubuntu:20.04 | ||
RUN apt-get update | ||
RUN apt-get install -y curl unzip | ||
RUN curl -fsS https://hub.infinyon.cloud/install/install.sh?ctx=dc | bash | ||
ENV PATH "$PATH:/root/.fluvio/bin" | ||
ENV PATH "$PATH:/root/.fvm/bin" | ||
``` | ||
|
||
Run `docker compose up`. | ||
|
||
> Optionally you can run on detached mode `docker compose up -d` so | ||
> Fluvio runs in the background. | ||
Then use the `fluvio` CLI to connect to the cluster running in Docker, to do | ||
that you must set the _Fluvio Profile_ to point to Docker's container SC: | ||
|
||
> If you dont have the Fluvio CLI installed on your local machine, run the following command | ||
> `curl -fsS https://hub.infinyon.cloud/install/install.sh | bash`. | ||
> Refer to [Fluvio CLI Reference][2] for more details. | ||
```bash copy="fl" | ||
fluvio profile add docker 127.0.0.1:9103 docker | ||
``` | ||
|
||
> Fluvio Streaming Controller (SC) usually runs on port `9003` but given that our | ||
> SC is running in a Docker Container, internal port `9003` is mapped to `9103` | ||
> in your system's network. | ||
With the profile set, you are now able to perform Fluvio Client operations | ||
like listing topics: | ||
|
||
```bash copy="fl" | ||
fluvio topics list | ||
``` | ||
|
||
## Teardown | ||
|
||
In order to shutdown the Fluvio Cluster running in Docker, you must issue the | ||
following `compose` command: | ||
|
||
```bash copy="fl" | ||
docker compose down | ||
``` | ||
|
||
> Remember to run this command in the same directory as the `docker-compose.yml` | ||
> file. | ||
[1]: https://www.fluvio.io/docs/architecture/overview/ | ||
[2]: https://www.fluvio.io/cli/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
{ | ||
"position": 30, | ||
"position": 40, | ||
"label": "Hub", | ||
"collapsible": true | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
--- | ||
sidebar_position: 1 | ||
title: "Overview" | ||
description: "Documentation for the InfinyOn Cloud" | ||
--- | ||
|
||
In this area of the documents, learn how to login and use the InfinyOn Cloud. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
{ | ||
"position": 20, | ||
"label": "Tutorials", | ||
"collapsible": true | ||
} |
4 changes: 2 additions & 2 deletions
4
.../cloud/how-to/use-connectors-in-cloud.mdx → docs/cloud/tutorials/connector-basics.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
--- | ||
title: "Hub Basics" | ||
description: "A short tutorial on InfinyOn Hub" | ||
sidebar_position: 40 | ||
--- | ||
|
||
The InfinyOn Hub serves as a centralized repository for InfinyOn SmartConnectors, SmartModules, and other extensions. It facilitates the discovery and deployment of extensions to enhance the data streaming capabilities of Fluvio. The hub provides a platform for developers and data engineers to easily access pre-built solutions or share their own extensions. | ||
|
||
## List | ||
|
||
Listing SmartModules will display the SmartModules available to your cluster. | ||
|
||
```shell copy="fl" | ||
$ fluvio hub sm list | ||
SMARTMODULE Visibility | ||
infinyon-labs/[email protected] public | ||
infinyon-labs/[email protected] public | ||
infinyon-labs/[email protected] public | ||
infinyon-labs/[email protected] public | ||
infinyon-labs/[email protected] public | ||
infinyon-labs/[email protected] public | ||
infinyon-labs/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
``` | ||
|
||
Listing Connectors will display the connector packages you can download locally. | ||
```shell copy="fl" | ||
$ fluvio hub conn list | ||
CONNECTOR Visibility | ||
infinyon-labs/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
infinyon/[email protected] public | ||
``` | ||
|
||
|
||
## Download | ||
|
||
Downloading a SmartModule is done with respect to your cluster. SmartModules need to be downloaded before they are used in transformations. | ||
|
||
```shell copy="fl" | ||
$ fluvio hub sm download infinyon/[email protected] | ||
downloading infinyon/[email protected] to infinyon-jolt-0.3.0.ipkg | ||
... downloading complete | ||
... checking package | ||
trying connection to fluvio router.infinyon.cloud:9003 | ||
... cluster smartmodule install complete | ||
``` | ||
|
||
Running `fluvio hub conn download` will download the package containing source code of the Connector. | ||
|
||
```shell copy="fl" | ||
$ fluvio hub conn download infinyon/[email protected] | ||
downloading infinyon/[email protected] to infinyon-http-source-0.2.5.ipkg | ||
... downloading complete | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.