Skip to content

Commit

Permalink
fix wrong thing in Tiered Storage doc and delete the doc overlap in s…
Browse files Browse the repository at this point in the history
…ecuity management (#107)
  • Loading branch information
wanghui42 authored Oct 25, 2023
1 parent 930f1d1 commit c6dc88c
Show file tree
Hide file tree
Showing 9 changed files with 16 additions and 2,070 deletions.
1 change: 0 additions & 1 deletion src/.vuepress/sidebar/V1.2.x/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@ export const enSidebar = {
{ text: 'Streaming', link: 'Streaming' },
{ text: 'Data Sync', link: 'Data-Sync' },
{ text: 'Database Programming', link: 'Database-Programming' },
{ text: 'Security Management', link: 'Security-Management' },
{ text: 'Authority Management', link: 'Authority-Management' },
],
},
Expand Down
1 change: 0 additions & 1 deletion src/.vuepress/sidebar/V1.2.x/zh.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@ export const zhSidebar = {
{ text: '流处理', link: 'Streaming' },
{ text: '数据同步', link: 'Data-Sync' },
{ text: '数据库编程', link: 'Database-Programming' },
{ text: '安全控制', link: 'Security-Management' },
{ text: '权限管理', link: 'Authority-Management' },
],
},
Expand Down
524 changes: 0 additions & 524 deletions src/UserGuide/V1.2.x/User-Manual/Security-Management.md

This file was deleted.

501 changes: 0 additions & 501 deletions src/UserGuide/V1.2.x/User-Manual/Security-Management_timecho.md

Large diffs are not rendered by default.

16 changes: 8 additions & 8 deletions src/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
# Tiered Storage
## Overview

Tiered storage function provides users with the ability to manage tiered storage media. users can use the tiered storage function to configure different types of storage media for IoTDB and to classify the storage media. ioTDB can support tiered storage from memory, SSD, normal hard disc to network hard disc by parameter configuration only according to the degree of hot and cold data. Specifically, in IoTDB, the configuration of tiered storage is reflected in the management of multiple directories. Users can group tiered storage directories into the same category and configure them into IoTDB as a "tier", which is called storage tier; at the same time, users can categorize data according to hot or cold, and store different categories of data into designated storage tiers. Meanwhile, users can categorise data according to hot or cold and store different categories of data in the specified tier. Currently, IoTDB supports the classification of hot and cold data by TTL, when the data in one tier does not meet the TTL rules defined in the current tier, the data will be automatically migrated to the next tier.
The Tiered storage functionality allows users to define multiple layers of storage, spanning across multiple types of storage media (Memory mapped directory, SSD, rotational hard discs or cloud storage). While memory and cloud storage is usually singular, the local file system storages can consist of multiple directories joined together into one tier. Meanwhile, users can classify data based on its hot or cold nature and store data of different categories in specified "tier". Currently, IoTDB supports the classification of hot and cold data through TTL (Time to live / age) of data. When the data in one tier does not meet the TTL rules defined in the current tier, the data will be automatically migrated to the next tier.

## Parameter Definition

Expand All @@ -36,14 +36,14 @@ The specific parameter definitions and their descriptions are as follows.

| Configuration | Default | Description | Constraint |
| ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| dn_data_dirs | None | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE |
| default_ttl_in_ms | None | Define the scope of data for which each tier is responsible, expressed through a TTL | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| dn_data_dirs | None | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE |
| default_ttl_in_ms | None | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| dn_default_space_move_thresholds | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| object_storage_type | AWS_S3 | Cloud Storage Type | IoTDB currently only supports AWS S3 as a remote storage type, and this parameter can't be modified |
| object_storage_bucket | None | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used |
| object_storage_endpoiont | | endpoint of cloud storage | endpoint of AWS S3;If remote storage is not used, no configuration required |
| object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key;If remote storage is not used, no configuration required |
| object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret;If remote storage is not used, no configuration required |
| object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key;If remote storage is not used, no configuration required |
| object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret;If remote storage is not used, no configuration required |
| remote_tsfile_cache_dirs | data/datanode/data/cache | Cache directory stored locally in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_page_size_in_kb | 20480 |Block size of locally cached files stored in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_max_disk_usage_in_mb | 51200 | Maximum Disk Occupancy Size for Cloud Storage Local Cache | If remote storage is not used, no configuration required |
Expand Down Expand Up @@ -91,6 +91,6 @@ In this example, a total of three levels of storage are configured, specifically

| **tier** | **data path** | **data range** | **threshold for minimum remaining disk space** |
| -------- | -------------------------------------- | ---------------------------- | ------------------------ |
| tier一 | path 1:/data1/data | data for last 1 day | 20% |
| tier二 | path 1:/data2/data path 2:/data3/data | data from past 1 day to past 10 days | 15% |
| tier三 | Remote AWS S3 Storage | data from 1 day ago | 10% |
| tier1 | path 1:/data1/data | data for last 1 day | 20% |
| tier2 | path 1:/data2/data path 2:/data3/data | data from past 1 day to past 10 days | 15% |
| tier3 | Remote AWS S3 Storage | data from 1 day ago | 10% |
6 changes: 4 additions & 2 deletions src/zh/UserGuide/V1.1.x/IoTDB-Introduction/Features.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,11 @@ IoTDB 具有以下特点:
* 终端解压即用
* 终端-云端无缝连接(数据云端同步工具)
* 低硬件成本的存储解决方案
* 高压缩比的磁盘存储(10 亿数据点硬盘成本低于 1.4 元)
* 高压缩比的磁盘存储
* 无损压缩比可达 20:1以上
* 10 亿数据点硬盘成本低于 1.4 元
* 目录结构的时间序列组织管理方式
* 支持复杂结构的智能网联设备的时间序列组织
* 支持复杂结构的智能网联设备的时间序列组织(多层树形结构,层级数量无限制)
* 支持大量同类物联网设备的时间序列组织
* 可用模糊方式对海量复杂的时间序列目录结构进行检索
* 高通量的时间序列数据读写
Expand Down
6 changes: 4 additions & 2 deletions src/zh/UserGuide/V1.2.x/IoTDB-Introduction/Features.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,11 @@ IoTDB 具有以下特点:
* 终端解压即用
* 终端-云端无缝连接(数据云端同步工具)
* 低硬件成本的存储解决方案
* 高压缩比的磁盘存储(10 亿数据点硬盘成本低于 1.4 元)
* 高压缩比的磁盘存储
* 无损压缩比可达 20:1以上
* 10 亿数据点硬盘成本低于 1.4 元
* 目录结构的时间序列组织管理方式
* 支持复杂结构的智能网联设备的时间序列组织
* 支持复杂结构的智能网联设备的时间序列组织(多层树形结构,层级数量无限制)
* 支持大量同类物联网设备的时间序列组织
* 可用模糊方式对海量复杂的时间序列目录结构进行检索
* 高通量的时间序列数据读写
Expand Down
Loading

0 comments on commit c6dc88c

Please sign in to comment.