Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipe: Upgraded pipe docs #169

Merged
merged 17 commits into from
Apr 8, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ grafana-cli plugins install apache-iotdb-datasource

* Click on Configuration ->Plugins ->Search IoTDB from local Grafana to install the plugin

### Method 3: Manually install the grafana-plugin plug-in (not recommended)
### Method 3: Manually install the grafana-plugin plugin (not recommended)


* Copy the front-end project target folder generated above to Grafana's plugin directory `${Grafana directory}\data\plugins\`。If there is no such directory, you can manually create it or start grafana and it will be created automatically. Of course, you can also modify the location of plugins. For details, please refer to the following instructions for modifying the location of Grafana's plugin directory.
Expand Down
499 changes: 499 additions & 0 deletions src/UserGuide/Master/User-Manual/Data-Sync.md

Large diffs are not rendered by default.

213 changes: 146 additions & 67 deletions src/UserGuide/Master/User-Manual/Data-Sync_timecho.md

Large diffs are not rendered by default.

412 changes: 223 additions & 189 deletions src/UserGuide/Master/User-Manual/Streaming.md

Large diffs are not rendered by default.

835 changes: 439 additions & 396 deletions src/UserGuide/Master/User-Manual/Streaming_timecho.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ grafana-cli plugins install apache-iotdb-datasource

* Click on Configuration ->Plugins ->Search IoTDB from local Grafana to install the plugin

### Method 3: Manually install the grafana-plugin plug-in (not recommended)
### Method 3: Manually install the grafana-plugin plugin (not recommended)


* Copy the front-end project target folder generated above to Grafana's plugin directory `${Grafana directory}\data\plugins\`。If there is no such directory, you can manually create it or start grafana and it will be created automatically. Of course, you can also modify the location of plugins. For details, please refer to the following instructions for modifying the location of Grafana's plugin directory.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ grafana-cli plugins install apache-iotdb-datasource

* Click on Configuration ->Plugins ->Search IoTDB from local Grafana to install the plugin

### Method 3: Manually install the grafana-plugin plug-in (not recommended)
### Method 3: Manually install the grafana-plugin plugin (not recommended)


* Copy the front-end project target folder generated above to Grafana's plugin directory `${Grafana directory}\data\plugins\`。If there is no such directory, you can manually create it or start grafana and it will be created automatically. Of course, you can also modify the location of plugins. For details, please refer to the following instructions for modifying the location of Grafana's plugin directory.
Expand Down
30 changes: 15 additions & 15 deletions src/UserGuide/V1.2.x/User-Manual/Data-Sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ By utilizing the data sync functionality, a complete data pipeline can be built

> ❗️**Note: The current IoTDB -> IoTDB implementation of data sync does not support DDL sync**
>
> That is: ttl, trigger, alias, template, view, create/delete sequence, create/delete storage group, etc. are not supported.
> That is: ttl, trigger, alias, template, view, create/delete sequence, create/delete database, etc. are not supported.
>
> **IoTDB -> IoTDB data sync requires the target IoTDB:**
>
Expand Down Expand Up @@ -119,11 +119,11 @@ WITH CONNECTOR (
| configuration item | description | Required or not | default implementation | Default implementation description | Whether to allow custom implementations |
| --------- | ------------------------------------------------- | --------------------------- | -------------------- | ------------------------------------------------------ | ------------------------- |
| pipeId | Globally uniquely identifies the name of a sync task | <font color=red>required</font> | - | - | - |
| extractor | pipe Extractor plug-in, for extracting synchronized data at the bottom of the database | Optional | iotdb-extractor | Integrate all historical data of the database and subsequent realtime data into the sync task | no |
| processor | Pipe Processor plug-in, for processing data | Optional | do-nothing-processor | no processing of incoming data | <font color=red>yes</font> |
| connector | Pipe Connector plug-in,for sending data | <font color=red>required</font> | - | - | <font color=red>yes</font> |
| extractor | pipe Extractor plugin, for extracting synchronized data at the bottom of the database | Optional | iotdb-extractor | Integrate all historical data of the database and subsequent realtime data into the sync task | no |
| processor | Pipe Processor plugin, for processing data | Optional | do-nothing-processor | no processing of incoming data | <font color=red>yes</font> |
| connector | Pipe Connector plugin,for sending data | <font color=red>required</font> | - | - | <font color=red>yes</font> |

In the example, the iotdb-extractor, do-nothing-processor, and iotdb-thrift-connector plug-ins are used to build the data sync task. iotdb has other built-in data sync plug-ins, **see the section "System Pre-built Data Sync Plugin"**.
In the example, the iotdb-extractor, do-nothing-processor, and iotdb-thrift-connector plugins are used to build the data sync task. iotdb has other built-in data sync plugins, **see the section "System Pre-built Data Sync Plugin"**.
**An example of a minimalist CREATE PIPE statement is as follows:**

```sql
Expand Down Expand Up @@ -250,7 +250,7 @@ The following diagram illustrates the different states and their transitions:

### View pre-built plugin

User can view the plug-ins in the system on demand. The statement for viewing plug-ins is shown below.
User can view the plugins in the system on demand. The statement for viewing plugins is shown below.
```sql
SHOW PIPEPLUGINS
```
Expand Down Expand Up @@ -386,19 +386,19 @@ Note: In theory, any version prior to v1.2.0 of IoTDB can serve as the data sync
Function: Does not do anything with the events passed in by the processor.


| key | value | value range | required or optional with default |
| --------- | -------------------- | ---------------------------- | --------------------------------- |
| key | value | value range | required or optional with default |
|-----------|----------------------|------------------------------|-----------------------------------|
| connector | do-nothing-connector | String: do-nothing-connector | required |

## Authority Management

| Authority Name | Description |
| ----------- | -------------------- |
| CREATE_PIPE | Register task,path-independent |
| START_PIPE | Start task,path-independent |
| STOP_PIPE | Stop task,path-independent |
| DROP_PIPE | Uninstall task,path-independent |
| SHOW_PIPES | Query task,path-independent |
| Authority Name | Description |
|----------------|---------------------------------|
| CREATE_PIPE | Register task,path-independent |
| START_PIPE | Start task,path-independent |
| STOP_PIPE | Stop task,path-independent |
| DROP_PIPE | Uninstall task,path-independent |
| SHOW_PIPES | Query task,path-independent |

## Configure Parameters

Expand Down
12 changes: 6 additions & 6 deletions src/UserGuide/V1.2.x/User-Manual/Data-Sync_timecho.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ By utilizing the data sync functionality, a complete data pipeline can be built

> ❗️**Note: The current IoTDB -> IoTDB implementation of data sync does not support DDL sync**
>
> That is: ttl, trigger, alias, template, view, create/delete sequence, create/delete storage group, etc. are not supported.
> That is: ttl, trigger, alias, template, view, create/delete sequence, create/delete database, etc. are not supported.
>
> **IoTDB -> IoTDB data sync requires the target IoTDB:**
>
Expand Down Expand Up @@ -119,11 +119,11 @@ WITH CONNECTOR (
| configuration item | description | Required or not | default implementation | Default implementation description | Whether to allow custom implementations |
| --------- | ------------------------------------------------- | --------------------------- | -------------------- | ------------------------------------------------------ | ------------------------- |
| pipeId | Globally uniquely identifies the name of a sync task | <font color=red>required</font> | - | - | - |
| extractor | pipe Extractor plug-in, for extracting synchronized data at the bottom of the database | Optional | iotdb-extractor | Integrate all historical data of the database and subsequent realtime data into the sync task | no |
| processor | Pipe Processor plug-in, for processing data | Optional | do-nothing-processor | no processing of incoming data | <font color=red>yes</font> |
| connector | Pipe Connector plug-in,for sending data | <font color=red>required</font> | - | - | <font color=red>yes</font> |
| extractor | pipe Extractor plugin, for extracting synchronized data at the bottom of the database | Optional | iotdb-extractor | Integrate all historical data of the database and subsequent realtime data into the sync task | no |
| processor | Pipe Processor plugin, for processing data | Optional | do-nothing-processor | no processing of incoming data | <font color=red>yes</font> |
| connector | Pipe Connector plugin,for sending data | <font color=red>required</font> | - | - | <font color=red>yes</font> |

In the example, the iotdb-extractor, do-nothing-processor, and iotdb-thrift-connector plug-ins are used to build the data sync task. iotdb has other built-in data sync plug-ins, **see the section "System Pre-built Data Sync Plugin"**.
In the example, the iotdb-extractor, do-nothing-processor, and iotdb-thrift-connector plugins are used to build the data sync task. iotdb has other built-in data sync plugins, **see the section "System Pre-built Data Sync Plugin"**.
**An example of a minimalist CREATE PIPE statement is as follows:**

```sql
Expand Down Expand Up @@ -250,7 +250,7 @@ The following diagram illustrates the different states and their transitions:

### View pre-built plugin

User can view the plug-ins in the system on demand. The statement for viewing plug-ins is shown below.
User can view the plugins in the system on demand. The statement for viewing plugins is shown below.
```sql
SHOW PIPEPLUGINS
```
Expand Down
32 changes: 16 additions & 16 deletions src/UserGuide/V1.2.x/User-Manual/Streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,7 @@ USING URI '<https://example.com:8080/iotdb/pipe-plugin.jar>'

### Delete Plugin Statement

When user no longer wants to use a plugin and needs to uninstall the plug-in from the system, you can use the Remove plugin statement as shown below.
When user no longer wants to use a plugin and needs to uninstall the plugin from the system, you can use the Remove plugin statement as shown below.
```sql
DROP PIPEPLUGIN <alias>
```
Expand Down Expand Up @@ -593,11 +593,11 @@ WITH CONNECTOR (
| configuration item | description | Required or not | default implementation | Default implementation description | Whether to allow custom implementations |
| --------- | ------------------------------------------------- | --------------------------- | -------------------- | ------------------------------------------------------ | ------------------------- |
| pipeId | Globally uniquely identifies the name of a sync task | <font color=red>required</font> | - | - | - |
| extractor | pipe Extractor plug-in, for extracting synchronized data at the bottom of the database | Optional | iotdb-extractor | Integrate all historical data of the database and subsequent realtime data into the sync task | no |
| processor | Pipe Processor plug-in, for processing data | Optional | do-nothing-processor | no processing of incoming data | <font color=red>yes</font> |
| connector | Pipe Connector plug-in,for sending data | <font color=red>required</font> | - | - | <font color=red>yes</font> |
| extractor | pipe Extractor plugin, for extracting synchronized data at the bottom of the database | Optional | iotdb-extractor | Integrate all historical data of the database and subsequent realtime data into the sync task | no |
| processor | Pipe Processor plugin, for processing data | Optional | do-nothing-processor | no processing of incoming data | <font color=red>yes</font> |
| connector | Pipe Connector plugin,for sending data | <font color=red>required</font> | - | - | <font color=red>yes</font> |

In the example, the iotdb-extractor, do-nothing-processor, and iotdb-thrift-connector plug-ins are used to build the data synchronisation task. iotdb has other built-in data synchronisation plug-ins, **see the section "System pre-built data synchronisation plug-ins" **. See the "System Pre-installed Stream Processing Plugin" section**.
In the example, the iotdb-extractor, do-nothing-processor, and iotdb-thrift-connector plugins are used to build the data synchronisation task. iotdb has other built-in data synchronisation plugins, **see the section "System pre-built data synchronisation plugins" **. See the "System Pre-installed Stream Processing Plugin" section**.

**An example of a minimalist CREATE PIPE statement is as follows:**

Expand Down Expand Up @@ -722,21 +722,21 @@ The following diagram illustrates the different states and their transitions:

### Stream Processing Task

| Authority Name | Description |
| ----------- | -------------------- |
| CREATE_PIPE | Register task,path-independent |
| START_PIPE | Start task,path-independent |
| STOP_PIPE | Stop task,path-independent |
| DROP_PIPE | Uninstall task,path-independent |
| SHOW_PIPES | Query task,path-independent |
| Authority Name | Description |
|----------------|---------------------------------|
| CREATE_PIPE | Register task,path-independent |
| START_PIPE | Start task,path-independent |
| STOP_PIPE | Stop task,path-independent |
| DROP_PIPE | Uninstall task,path-independent |
| SHOW_PIPES | Query task,path-independent |
### Stream Processing Task Plugin


| Authority Name | Description |
| ----------------- | ------------------------------ |
| Authority Name | Description |
|-------------------|---------------------------------------------------------|
| CREATE_PIPEPLUGIN | Register stream processing task plugin,path-independent |
| DROP_PIPEPLUGIN | Delete stream processing task plugin,path-independent |
| SHOW_PIPEPLUGINS | Query stream processing task plugin,path-independent |
| DROP_PIPEPLUGIN | Delete stream processing task plugin,path-independent |
| SHOW_PIPEPLUGINS | Query stream processing task plugin,path-independent |

## Configure Parameters

Expand Down
2 changes: 1 addition & 1 deletion src/UserGuide/V1.2.x/User-Manual/Streaming_timecho.md
Original file line number Diff line number Diff line change
Expand Up @@ -455,7 +455,7 @@ AS <full class name>
USING <URI of JAR package>
```

Example: If you implement a data processing plugin named edu.tsinghua.iotdb.pipe.ExampleProcessor, and the packaged jar package is pipe-plugin.jar, you want to use this plugin in the stream processing engine, and mark the plugin as example. There are two ways to use the plug-in package, one is to upload to the URI server, and the other is to upload to the local directory of the cluster.
Example: If you implement a data processing plugin named edu.tsinghua.iotdb.pipe.ExampleProcessor, and the packaged jar package is pipe-plugin.jar, you want to use this plugin in the stream processing engine, and mark the plugin as example. There are two ways to use the plugin package, one is to upload to the URI server, and the other is to upload to the local directory of the cluster.

Method 1: Upload to the URI server

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ grafana-cli plugins install apache-iotdb-datasource

* Click on Configuration ->Plugins ->Search IoTDB from local Grafana to install the plugin

### Method 3: Manually install the grafana-plugin plug-in (not recommended)
### Method 3: Manually install the grafana-plugin plugin (not recommended)


* Copy the front-end project target folder generated above to Grafana's plugin directory `${Grafana directory}\data\plugins\`。If there is no such directory, you can manually create it or start grafana and it will be created automatically. Of course, you can also modify the location of plugins. For details, please refer to the following instructions for modifying the location of Grafana's plugin directory.
Expand Down
Loading