Skip to content

Commit

Permalink
docs: add more cloud tutorials and refactor a bit (#104)
Browse files Browse the repository at this point in the history
  • Loading branch information
digikata authored Jul 18, 2024
1 parent 4b3f0b0 commit ff7c7df
Show file tree
Hide file tree
Showing 12 changed files with 369 additions and 15 deletions.
2 changes: 1 addition & 1 deletion docs/cloud/how-to/_category_.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"label": "How-To Guides",
"position": 20
"position": 30
}
132 changes: 132 additions & 0 deletions docs/cloud/how-to/docker-installation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
---
title: "Install Fluvio Cluster in Docker"
menu: "Use Fluvio in Docker"
weight: 11
description: "Install Fluvio Cluster with Docker Compose"
---

# Docker Compose + Fluvio

You can run Fluvio Clusters in a Docker Compose setup, for a local Proof of Concept or
development.

To run a Fluvio Cluster through Docker, you will need to run Fluvio
components separately, we can use Docker Compose `service`s to achieve this.

## Services

- `sc`: Streaming Controller
- `sc-setup`: Post-Initialization Commands
- `spu`: Streaming Processing Unit

> To learn more about Fluvio architecture, please refer to [Fluvio Documentation][1]
## Running Locally

Copy the docker-compose.yaml and Dockerfile to a directory.

> Or clone the Fluvio repo `git clone https://github.com/infinyon/fluvio.git` and
cd into `./fluvio/examples/docker-compose`

`docker-compose.yaml`:

```yaml
version: "3"
services:
sc:
image: infinyon/fluvio:stable
container_name: sc
hostname: sc
ports:
- "9103:9003"
environment:
- RUST_LOG=info
command: "./fluvio-run sc --local /fluvio/metadata"
volumes:
- ./fluvio-metadata:/fluvio/metadata
sc-setup:
build:
context: .
dockerfile: Dockerfile
container_name: sc-setup
environment:
- RUST_LOG=info
entrypoint: >
/bin/sh -c "
fluvio profile add docker sc:9003 docker;
fluvio cluster spu register --id 5001 -p 0.0.0.0:9110 --private-server spu:9011;
exit 0;
"
depends_on:
- sc
spu:
image: infinyon/fluvio:stable
container_name: spu
hostname: spu
volumes:
- ./fluvio-data:/fluvio/data
environment:
- RUST_LOG=info
ports:
- "9110:9010"
- "9111:9011"
command: "./fluvio-run spu -i 5001 -p spu:9010 -v spu:9011 --sc-addr sc:9004 --log-base-dir /fluvio/data"
depends_on:
- sc

```

`Dockerfile`:

```
FROM ubuntu:20.04
RUN apt-get update
RUN apt-get install -y curl unzip
RUN curl -fsS https://hub.infinyon.cloud/install/install.sh?ctx=dc | bash
ENV PATH "$PATH:/root/.fluvio/bin"
ENV PATH "$PATH:/root/.fvm/bin"
```

Run `docker compose up`.

> Optionally you can run on detached mode `docker compose up -d` so
> Fluvio runs in the background.
Then use the `fluvio` CLI to connect to the cluster running in Docker, to do
that you must set the _Fluvio Profile_ to point to Docker's container SC:

> If you dont have the Fluvio CLI installed on your local machine, run the following command
> `curl -fsS https://hub.infinyon.cloud/install/install.sh | bash`.
> Refer to [Fluvio CLI Reference][2] for more details.
```bash copy="fl"
fluvio profile add docker 127.0.0.1:9103 docker
```

> Fluvio Streaming Controller (SC) usually runs on port `9003` but given that our
> SC is running in a Docker Container, internal port `9003` is mapped to `9103`
> in your system's network.
With the profile set, you are now able to perform Fluvio Client operations
like listing topics:

```bash copy="fl"
fluvio topics list
```

## Teardown

In order to shutdown the Fluvio Cluster running in Docker, you must issue the
following `compose` command:

```bash copy="fl"
docker compose down
```

> Remember to run this command in the same directory as the `docker-compose.yml`
> file.
[1]: https://www.fluvio.io/docs/architecture/overview/
[2]: https://www.fluvio.io/cli/
2 changes: 1 addition & 1 deletion docs/cloud/hub/_category_.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"position": 30,
"position": 40,
"label": "Hub",
"collapsible": true
}
7 changes: 7 additions & 0 deletions docs/cloud/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
sidebar_position: 1
title: "Overview"
description: "Documentation for the InfinyOn Cloud"
---

In this area of the documents, learn how to login and use the InfinyOn Cloud.
5 changes: 5 additions & 0 deletions docs/cloud/tutorials/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"position": 20,
"label": "Tutorials",
"collapsible": true
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebar_position: 30
title: "Use Connectors on InfinyOn Cloud"
title: "Connector Basics"
description: "Connectors with InfinyOn Cloud is the best option for those who want to manage their data pipelines in one place."
sidebar_position: 30
---

Connectors with [InfinyOn Cloud] is the best option for those who want to manage their data pipelines in one place.
Expand Down
63 changes: 63 additions & 0 deletions docs/cloud/tutorials/hub-basics.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
title: "Hub Basics"
description: "A short tutorial on InfinyOn Hub"
sidebar_position: 40
---

The InfinyOn Hub serves as a centralized repository for InfinyOn SmartConnectors, SmartModules, and other extensions. It facilitates the discovery and deployment of extensions to enhance the data streaming capabilities of Fluvio. The hub provides a platform for developers and data engineers to easily access pre-built solutions or share their own extensions.

## List

Listing SmartModules will display the SmartModules available to your cluster.

```shell copy="fl"
$ fluvio hub sm list
SMARTMODULE Visibility
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
```

Listing Connectors will display the connector packages you can download locally.
```shell copy="fl"
$ fluvio hub conn list
CONNECTOR Visibility
infinyon-labs/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
```


## Download

Downloading a SmartModule is done with respect to your cluster. SmartModules need to be downloaded before they are used in transformations.

```shell copy="fl"
$ fluvio hub sm download infinyon/[email protected]
downloading infinyon/[email protected] to infinyon-jolt-0.3.0.ipkg
... downloading complete
... checking package
trying connection to fluvio router.infinyon.cloud:9003
... cluster smartmodule install complete
```

Running `fluvio hub conn download` will download the package containing source code of the Connector.

```shell copy="fl"
$ fluvio hub conn download infinyon/[email protected]
downloading infinyon/[email protected] to infinyon-http-source-0.2.5.ipkg
... downloading complete
```
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
---
sidebar_position: 10
title: "Use InfinyOn Cloud"
title: Use InfinyOn Cloud Cluster
menu: Use InfinyOn Cloud
description: "Step-by-step instructions on how to create an account and login to InfinyOn Cloud using the CLI."
sidebar_position: 10
---

# Use InfinyOn Cloud Cluster

InfinyOn Cloud is managed through the [`Fluvio CLI`] via the `fluvio-cloud` plugin distributed by default by the installation script.

Let's create an InfinyOn Cloud account and login using the Fluvio CLI:
Expand Down Expand Up @@ -68,4 +67,4 @@ Password: <hidden>
:tada: Congratulations! You are now connected to InfinyOn Cloud.


[`Fluvio CLI`]: https://www.fluvio.io/cli/
[`Fluvio CLI`]: https://www.fluvio.io/cli/
Loading

0 comments on commit ff7c7df

Please sign in to comment.