Skip to content

Commit

Permalink
feat: centralize knowledge base (#6)
Browse files Browse the repository at this point in the history
  • Loading branch information
EstebanBorai authored May 8, 2024
1 parent d716587 commit c37f527
Show file tree
Hide file tree
Showing 69 changed files with 5,548 additions and 492 deletions.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"label": "Tutorial - Extras",
"label": "How-To Guides",
"position": 3,
"link": {
"type": "generated-index"
Expand Down
180 changes: 180 additions & 0 deletions docs/how-to/install-cli-toolchain.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
---
sidebar_position: 8
title: "Install Fluvio CLI Toolchain"
description: "A short tutorial for using fvm"
---

This is a short introduction to `fvm`, the Fluvio Version Manager, which allows you to use multiple versions of the Fluvio CLI toolchain.

The `fvm` CLI is the official package manager for Fluvio, managing various `fluvio` binaries and development tools -- enabling the selection from multiple versions or release channels.

## Install `fvm`

The following command will install `fvm` and `fluvio` and other binaries in the development kit.

%copy first-line%
```shell
$ curl -fsS https://hub.infinyon.cloud/install/install.sh | bash
```

The installation script prints a command that adds the the `fluvio` binaries to your shell PATH environment variable. This is required to follow the following steps.

On macOS, run this command and then close and reopen your terminal.

%copy first-line%
```shell
$ echo 'export PATH="${HOME}/.fvm/bin:${HOME}/.fluvio/bin:${PATH}"' >> ~/.zshrc
```

<Idea>
If you've got existing installation of Fluvio (version `0.10.15` or older), please head on to the next section for additional follow-up steps.
</Idea>

### (Optional) For existing installation of Fluvio CLI

must run this command before continuing further. We will cover the usage later when we talk about release channels.

If you have an existing `fluvio` installation (version `0.10.15` or older), you can choose 1 of the following:

#### Option 1: Delete the fluvio directory
This will clear out your existing settings.

%copy first-line%
```shell
$ rm -ri $HOME/.fluvio
rm -ri $HOME/.fluvio
examine files in directory $HOME/.fluvio? y
remove $HOME/.fluvio? y
```
After deleting this directory, you can re-run the [installation instructions](#install-fvm).


#### Option 2: Install the stable channel

New installs run this step as part of the [installation script](#install-fvm), but you can migrate your existing settings

%copy%
```shell
$ fvm install
info: Downloading (1/5): [email protected]
info: Downloading (2/5): [email protected]
info: Downloading (3/5): [email protected]
info: Downloading (4/5): [email protected]
info: Downloading (5/5): [email protected]
done: Installed fluvio version stable
done: Now using fluvio version 0.10.16
```

### What version are you running?

Running `fvm current` should display the most recent version of `fluvio` toolchain installed. (The most recent version is `0.10.16` for the purposes of this tutorial)

%copy first-line%
```shell
$ fvm current
0.10.16 (stable)
```

## Release channels

Also in the output of `fvm current` we see `stable`, which is the name of the default release channel, and the active channel in use. There are 2 channels: `stable`, `latest`.

Installing channels with `fvm install` will also make that channel active.

### The Active channel

Only one channel is active at a time. You can use [`fvm switch`](#switching-between-channels) to select one of your installed channels.

FVM updates the `fluvio`/`smdk`/`cdk` etc. binaries used with the active channel's binaries.

### Stable release channel

The `stable` channel is installed by default. It is the most recent supported release.

The following commands are equivalent for installing or updating the `stable` release channel.

%copy first-line%
```shell
$ fvm install
$ fvm install stable
```

### Latest release channel

If you contact us for support in GitHub or Discord, we may request you to validate fixes from the `latest` channel.

This channel consists of most recent updates that have not yet been released, which may include experimental features or unexpected behavior.

The `latest` channel is not intended for typical usage.

%copy first-line%
```shell
$ fvm install latest
info: Downloading (1/5): [email protected]+bf4e86674ce546d6b853adbf36a97e8e3344bd17
info: Downloading (2/5): [email protected]+bf4e86674ce546d6b853adbf36a97e8e3344bd17
info: Downloading (3/5): [email protected]
info: Downloading (4/5): [email protected]+bf4e86674ce546d6b853adbf36a97e8e3344bd17
info: Downloading (5/5): [email protected]+bf4e86674ce546d6b853adbf36a97e8e3344bd17
done: Installed fluvio version latest
done: Now using fluvio version 0.11.0-dev-1+bf4e86674ce546d6b853adbf36a97e8e3344bd17
```

### Install a specific version

Specific releases can also be installed when you provide the version.

```shell
$ fvm install 0.10.16
info: Downloading (1/5): [email protected]
info: Downloading (2/5): [email protected]
info: Downloading (3/5): [email protected]
info: Downloading (4/5): [email protected]
info: Downloading (5/5): [email protected]
done: Installed fluvio version 0.10.16
done: Now using fluvio version 0.10.16
```

### Listing installed channels

The `fvm show` command will list out the installed channels and their corresponding version. The row with the checkmark (``) is the current active channel.

%copy first-line%
```shell
$ fvm show
CHANNEL VERSION
✓ 0.10.16 0.10.16
stable 0.10.16
latest 0.11.0-dev-1+bf4e86674ce546d6b853adbf36a97e8e3344bd17
```

## Switching between channels

If you are on another channel, you can change between them by running `fvm switch` with the name of the channel.

For typical usage of InfinyOn Cloud, we suggest using `stable`.

%copy first-line%
```shell
$ fvm switch stable

done: Now using Fluvio version stable
```

And we verify with `fvm show` that we are now back on the `stable` release channel.

%copy first-line%
```shell
$ fvm show
CHANNEL VERSION
✓ stable 0.10.16
latest 0.11.0-dev-1+bf4e86674ce546d6b853adbf36a97e8e3344bd17
0.10.16 0.10.16
```

## Conclusion

This wraps up the fundamental usage of `fvm`. While this guide covers the basics, `fvm` has more under the hood to explore and adapt to your workflows, ensuring a frictionless experience as you delve into Fluvio's ecosystem.

However, our expectation is that a majority of users will find the `stable` channel adequately meets their needs, making the transition to other channels unnecessary. For developers actively contributing through issues or code, and our design partners, `fvm` serves as a bridge for closer and smoother collaboration.

Hopefully this is just the beginning of your journey with `fvm`. We're excited to see how it enhances your interactions with our platform.
183 changes: 183 additions & 0 deletions docs/how-to/smartmodule-basics.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
---
sidebar_position: 3
title: "Use SmartModules"
description: "A short tutorial on using SmartModules"
---

<img src="/img/docs/how-to/smartmodule-basics/smartmodule-overview.svg" />

This tutorial assumes that `fluvio` is installed, and logged-in to InfinyOn Cloud. Follow the [Quick Start](/docs/quickstart) to get set up.

SmartModules are the basic building blocks for transformations in Fluvio, allowing users to define custom functions for processing or transforming streaming data. They provide a flexible way to tailor data handling to meet particular needs, enhancing Fluvio's capabilities.

## See list of available SmartModules

%copy first-line%
```shell
$ fluvio hub sm list
SMARTMODULE Visibility
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon-labs/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
infinyon/[email protected] public
```

## Download SmartModules

SmartModules must be downloaded before they can be used. Afterwards, downloaded SmartModules are available for your Producers and Consumers.


%copy first-line%
```shell
$ fluvio hub sm download infinyon/[email protected]
downloading infinyon/[email protected] to infinyon-regex-filter-0.1.0.ipkg
... downloading complete
... checking package
trying connection to fluvio router.infinyon.cloud:9003
... cluster smartmodule install complete
```

## Use SmartModule with Producer and Consumer

You can specify a SmartModule to use with `fluvio produce` and `fluvio consume`

This consumer is using the SmartModule we just downloaded with `--smartmodule`, and is also configuring it with `--params`/`-e`.

%copy first-line%
```shell
$ fluvio consume --smartmodule infinyon/[email protected] --params regex='[Cc]at' cat-facts
```

<Idea>
You can configure SmartModules with multiple parameters by passing multiple `--params`/`-e`.

e.g.

%copy first-line%
```shell
$ fluvio produce --smartmodule example/[email protected] -e name=example -e point_made=true
```
</Idea>

The `--transforms-file` flag are for more complex transformations defined in a YAML file. See the [Transformation Chaining][transformation-chaining] page for more detail.

Example `transforms.yaml` with multiple transformations. Order matters here, so `infinyon/[email protected]` is first and `infinyon/[email protected]` is second.

%copy%
```yaml
transforms:
- uses: infinyon/[email protected]
with:
spec:
- operation: shift
spec:
fact: "animal.fact"
length: "length"
- uses: infinyon/[email protected]
with:
regex: "[Cc]at"
```
%copy first-line%
```shell
$ fluvio consume --transforms-file ./my-transforms.yaml my-topic
```

## See list of Downloaded SmartModules

%copy first-line%
```shell
$ fluvio sm list
SMARTMODULE SIZE
infinyon/[email protected] 611.5 KB
infinyon/[email protected] 558.4 KB
infinyon/[email protected] 312.7 KB
infinyon/[email protected] 564.0 KB
infinyon/[email protected] 559.6 KB
```

## Delete SmartModules

%copy first-line%
```shell
$ fluvio sm delete infinyon/[email protected]
smartmodule "infinyon/[email protected]" deleted
```

## Use SmartModule with Connector

You can define transforms when you create connectors with `transforms`

%copy%
```yaml title="connector-example.yaml"
apiVersion: 0.1.0
meta:
version: 0.2.5
name: cat-facts
type: http-source
topic: cat-facts
create-topic: true
secrets:
- name: AUTHORIZATION_TOKEN
http:
endpoint: "https://catfact.ninja/fact"
interval: 10s
headers:
- "Authorization: token ${{ secrets.AUTHORIZATION_TOKEN }}"
- "Cache-Control: no-cache"
transforms:
- uses: infinyon/[email protected]
with:
spec:
- operation: shift
spec:
fact: "animal.fact"
length: "length"
- uses: infinyon/[email protected]
with:
regex: "[Cc]at"
```
Everything is in the config file. You create a connector as usual.
%copy first-line%
```shell
$ fluvio cloud connector create --config connector-example.yaml
```

## Use SmartModule with Webhook

%copy%
```yaml title="example-webhook.yaml"
meta:
name: my-webhook
topic: my-topic
transforms:
- uses: infinyon/[email protected]
with:
spec:
- operation: shift
spec:
fact: "animal.fact"
length: "length"
- uses: infinyon/[email protected]
with:
regex: "[Cc]at"
webhook:
outputParts: body
```
Just like Connectors, everything is in the config file. You create a webhook as usual.
%copy first-line%
```shell
fluvio cloud webhook create --config example-webhook.yaml
```

[transformation-chaining]:
Loading

0 comments on commit c37f527

Please sign in to comment.