Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the best encoding and compression methods #455

Merged
merged 2 commits into from
Dec 9, 2024
Merged

Add the best encoding and compression methods #455

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f18cbe9
Add the best encoding and compression methods
W1y1r Dec 6, 2024
8e22905
Supplement the best encoding for the old version
W1y1r Dec 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions src/UserGuide/Master/Tree/Technical-Insider/Encoding-and-Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,18 +78,18 @@ The five encodings described in the previous sections are applicable to differen

The correspondence between the data type and its supported encodings is summarized in the Table below.

| Data Type | Supported Encoding |
|:---------:|:-----------------------------------------------------------:|
| BOOLEAN | PLAIN, RLE |
| INT32 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| DATE | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| LONG | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN, DICTIONARY |
| STRING | PLAIN, DICTIONARY |
| BLOB | PLAIN, DICTIONARY |
| **Data Type** | **Best Encoding (default)** | **Supported Encoding** |
| ------------- | --------------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| DATE | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| TIMESTAMP | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |
| STRING | PLAIN | PLAIN, DICTIONARY |
| BLOB | PLAIN | PLAIN |

When the data type specified by the user does not correspond to the encoding method, the system will prompt an error.

Expand All @@ -111,7 +111,7 @@ IoTDB allows you to specify the compression method of the column when creating a

* SNAPPY

* LZ4
* LZ4 (Best compression method)

* GZIP

Expand Down
2 changes: 1 addition & 1 deletion src/UserGuide/V0.13.x/Data-Concept/Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ IoTDB allows you to specify the compression method of the column when creating a

* SNAPPY

* LZ4
* LZ4 (Best compression method)

* GZIP

Expand Down
16 changes: 8 additions & 8 deletions src/UserGuide/V0.13.x/Data-Concept/Encoding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,13 @@ The five encodings described in the previous sections are applicable to differen

**The correspondence between the data type and its supported encodings**

|Data Type |Supported Encoding|
|:---:|:---:|
|BOOLEAN| PLAIN, RLE|
|INT32 |PLAIN, RLE, TS_2DIFF, GORILLA|
|INT64 |PLAIN, RLE, TS_2DIFF, GORILLA|
|FLOAT |PLAIN, RLE, TS_2DIFF, GORILLA|
|DOUBLE |PLAIN, RLE, TS_2DIFF, GORILLA|
|TEXT |PLAIN, DICTIONARY|
| **Data Type** | **Best Encoding (default)** | **Supported Encoding** |
| ------------- | --------------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |

</div>
2 changes: 1 addition & 1 deletion src/UserGuide/V1.0.x/Data-Concept/Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ IoTDB allows you to specify the compression method of the column when creating a

* SNAPPY

* LZ4
* LZ4 (Best compression method)

* GZIP

Expand Down
16 changes: 8 additions & 8 deletions src/UserGuide/V1.0.x/Data-Concept/Encoding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,13 @@ The five encodings described in the previous sections are applicable to differen

**The correspondence between the data type and its supported encodings**

|Data Type |Supported Encoding|
|:---:|:---:|
|BOOLEAN| PLAIN, RLE|
|INT32 |PLAIN, RLE, TS_2DIFF, GORILLA, FREQ, ZIGZAG|
|INT64 |PLAIN, RLE, TS_2DIFF, GORILLA, FREQ, ZIGZAG|
|FLOAT |PLAIN, RLE, TS_2DIFF, GORILLA, FREQ|
|DOUBLE |PLAIN, RLE, TS_2DIFF, GORILLA, FREQ|
|TEXT |PLAIN, DICTIONARY|
| **Data Type** | **Best Encoding (default)** | **Supported Encoding** |
| ------------- | --------------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |

</div>
2 changes: 1 addition & 1 deletion src/UserGuide/V1.1.x/Data-Concept/Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ IoTDB allows you to specify the compression method of the column when creating a

* SNAPPY

* LZ4
* LZ4 (Best compression method)

* GZIP

Expand Down
16 changes: 8 additions & 8 deletions src/UserGuide/V1.1.x/Data-Concept/Encoding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,14 @@ The five encodings described in the previous sections are applicable to differen

The correspondence between the data type and its supported encodings is summarized in the Table below.

| Data Type | Supported Encoding |
|:---------:|:-------------------------------------------:|
| BOOLEAN | PLAIN, RLE |
| INT32 | PLAIN, RLE, TS_2DIFF, GORILLA, FREQ, ZIGZAG |
| INT64 | PLAIN, RLE, TS_2DIFF, GORILLA, FREQ, ZIGZAG |
| FLOAT | PLAIN, RLE, TS_2DIFF, GORILLA, FREQ |
| DOUBLE | PLAIN, RLE, TS_2DIFF, GORILLA, FREQ |
| TEXT | PLAIN, DICTIONARY |
| **Data Type** | **Best Encoding (default)** | **Supported Encoding** |
| ------------- | --------------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |

When the data type specified by the user does not correspond to the encoding method, the system will prompt an error.

Expand Down
18 changes: 9 additions & 9 deletions src/UserGuide/V1.2.x/Basic-Concept/Encoding-and-Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,14 +79,14 @@ The five encodings described in the previous sections are applicable to differen

The correspondence between the data type and its supported encodings is summarized in the Table below.

| Data Type | Supported Encoding |
|:---------:|:-----------------------------------------------------------:|
| BOOLEAN | PLAIN, RLE |
| INT32 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN, DICTIONARY |
| **Data Type** | **Best Encoding (default)** | **Supported Encoding** |
| ------------- | --------------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY

When the data type specified by the user does not correspond to the encoding method, the system will prompt an error.

Expand All @@ -108,7 +108,7 @@ IoTDB allows you to specify the compression method of the column when creating a

* SNAPPY

* LZ4
* LZ4 (Best compression method)

* GZIP

Expand Down
19 changes: 10 additions & 9 deletions src/UserGuide/V1.3.0-2/Basic-Concept/Encoding-and-Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,14 +78,15 @@ The five encodings described in the previous sections are applicable to differen

The correspondence between the data type and its supported encodings is summarized in the Table below.

| Data Type | Supported Encoding |
|:---------:|:-----------------------------------------------------------:|
| BOOLEAN | PLAIN, RLE |
| INT32 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN, DICTIONARY |

| **Data Type** | **Best Encoding (default)** | **Supported Encoding** |
| ------------- | --------------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |

When the data type specified by the user does not correspond to the encoding method, the system will prompt an error.

Expand All @@ -107,7 +108,7 @@ IoTDB allows you to specify the compression method of the column when creating a

* SNAPPY

* LZ4
* LZ4 (Best compression method)

* GZIP

Expand Down
26 changes: 13 additions & 13 deletions src/UserGuide/latest/Technical-Insider/Encoding-and-Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,18 +78,18 @@ The five encodings described in the previous sections are applicable to differen

The correspondence between the data type and its supported encodings is summarized in the Table below.

| Data Type | Supported Encoding |
|:---------:|:-----------------------------------------------------------:|
| BOOLEAN | PLAIN, RLE |
| INT32 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| DATE | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| LONG | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN, DICTIONARY |
| STRING | PLAIN, DICTIONARY |
| BLOB | PLAIN, DICTIONARY |
| **Data Type** | **Best Encoding (default)** | **Supported Encoding** |
| ------------- | --------------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| DATE | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| TIMESTAMP | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |
| STRING | PLAIN | PLAIN, DICTIONARY |
| BLOB | PLAIN | PLAIN |

When the data type specified by the user does not correspond to the encoding method, the system will prompt an error.

Expand All @@ -111,7 +111,7 @@ IoTDB allows you to specify the compression method of the column when creating a

* SNAPPY

* LZ4
* LZ4 (Best compression method)

* GZIP

Expand Down
26 changes: 13 additions & 13 deletions src/zh/UserGuide/Master/Tree/Technical-Insider/Encoding-and-Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,18 +77,18 @@

前文介绍的五种编码适用于不同的数据类型，若对应关系错误，则无法正确创建时间序列。数据类型与支持其编码的编码方式对应关系总结如下表所示。

| 数据类型 | 支持的编码 |
|:---------:| :---------------------------------------------------------: |
| BOOLEAN | PLAIN, RLE |
| INT32 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| DATE | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| TIMESTAMP | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN, DICTIONARY |
| STRING | PLAIN, DICTIONARY |
| BLOB | PLAIN, DICTIONARY |
| **数据类型** | **最佳的编码（默认）** | **支持的编码** |
| ------------ | ---------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| DATE | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| TIMESTAMP | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |
| STRING | PLAIN | PLAIN, DICTIONARY |
| BLOB | PLAIN | PLAIN |

当用户输入的数据类型与编码方式不对应时，系统会提示错误。如下所示，二阶差分编码不支持布尔类型：

Expand All @@ -107,7 +107,7 @@ IoTDB 允许在创建一个时间序列的时候指定该列的压缩方式。

* UNCOMPRESSED（不压缩）
* SNAPPY 压缩
* LZ4 压缩
* LZ4 压缩（最佳压缩方式）
* GZIP 压缩
* ZSTD 压缩
* LZMA2 压缩
Expand Down
2 changes: 1 addition & 1 deletion src/zh/UserGuide/V0.13.x/Data-Concept/Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ IoTDB 允许在创建一个时间序列的时候指定该列的压缩方式。

* UNCOMPRESSED（不压缩）
* SNAPPY 压缩
* LZ4 压缩
* LZ4 压缩 （最佳压缩方式）
* GZIP 压缩

压缩方式的指定语法详见本文 [SQL 参考文档](../Reference/SQL-Reference.md)。
Expand Down
16 changes: 8 additions & 8 deletions src/zh/UserGuide/V0.13.x/Data-Concept/Encoding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,13 @@ GORILLA 编码是一种无损编码，它比较适合编码前后值比较接近

**表格 2-3 数据类型与支持其编码的对应关系**

|数据类型 |支持的编码|
|:---:|:---:|
|BOOLEAN| PLAIN, RLE|
|INT32 |PLAIN, RLE, TS_2DIFF, GORILLA|
|INT64 |PLAIN, RLE, TS_2DIFF, GORILLA|
|FLOAT |PLAIN, RLE, TS_2DIFF, GORILLA|
|DOUBLE |PLAIN, RLE, TS_2DIFF, GORILLA|
|TEXT |PLAIN, DICTIONARY|
| **数据类型** | **最佳的编码（默认）** | **支持的编码** |
| ------------ | ---------------------- | ----------------------------------------------------------- |
| BOOLEAN | RLE | PLAIN, RLE |
| INT32 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| INT64 | TS_2DIFF | PLAIN, RLE, TS_2DIFF, GORILLA, ZIGZAG, CHIMP, SPRINTZ, RLBE |
| FLOAT | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| DOUBLE | GORILLA | PLAIN, RLE, TS_2DIFF, GORILLA, CHIMP, SPRINTZ, RLBE |
| TEXT | PLAIN | PLAIN, DICTIONARY |

</div>
2 changes: 1 addition & 1 deletion src/zh/UserGuide/V1.0.x/Data-Concept/Compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ IoTDB 允许在创建一个时间序列的时候指定该列的压缩方式。

* UNCOMPRESSED（不压缩）
* SNAPPY 压缩
* LZ4 压缩
* LZ4 压缩 （最佳压缩方式）
* GZIP 压缩

压缩方式的指定语法详见本文 [SQL 参考文档](../Reference/SQL-Reference.md)。
Expand Down
Loading
Loading