Skip to content

Commit

Permalink
operator deploy on cloud
Browse files Browse the repository at this point in the history
  • Loading branch information
catpineapple committed Jan 23, 2025
1 parent 0ecf79c commit a763109
Show file tree
Hide file tree
Showing 3 changed files with 53 additions and 52 deletions.
50 changes: 25 additions & 25 deletions docs/install/cluster-deployment/k8s-deploy/install-on-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ specific language governing permissions and limitations
under the License.
-->

Doris-operator is developed in accordance with standard K8s specifications and is compatible with all standard K8s platforms, including those provided by mainstream cloud vendors and self-built based on standards. This article mainly provides precautions and some usage suggestions for Doris-operator on the containerized service platforms of mainstream cloud vendors. More documents for cloud vendors and their products will be updated later.
Doris Operator is developed in accordance with standard K8s specifications and is compatible with all standard K8s platforms, including those provided by mainstream cloud vendors and self-built based on standards. This article mainly provides precautions and some usage suggestions for Doris Operator on the containerized service platforms of mainstream cloud vendors. More documents for cloud vendors and their products will be updated later.

## Alibaba ACK

Alibaba Cloud Container Service ACK is a managed containerized service after purchasing an ECS instance, so you can obtain full access control permissions to adjust related system parameters. Use the instance image: Alibaba Cloud Linux 3. The current system parameters fully meet the requirements for running Doris. Those that do not meet the requirements can also be corrected in the container through the K8s privileged mode to ensure stable operation.
**Alibaba Cloud ACK cluster, deployed using doris-operator, most environmental requirements can be met by the ECS default configuration. If not met, doris-operator can correct it by itself**. Users can also manually correct it, as follows:
**Alibaba Cloud ACK cluster, deployed using Doris Operator, most environmental requirements can be met by the ECS default configuration. If not met, Doris Operator can correct it by itself**. Users can also manually correct it, as follows:

### Already exists cluster

Expand Down Expand Up @@ -82,27 +82,27 @@ After configuring the ACR and image transfer environment, you need to migrate th

If you use a private ACR to enable authentication, you can refer to the following steps:

1. You need to set a `secret` of type `docker-registry` in advance to configure the authentication information for accessing the image warehouse.
```
kubectl create secret docker-registry image-hub-secret --docker-server={your-server} --docker-username={your-username} --docker-password={your-pwd}
```
1. You need to set a `secret` of type `docker-registry` in advance to configure the authentication information for accessing the image warehouse.
```shell
kubectl create secret docker-registry image-hub-secret --docker-server={your-server} --docker-username={your-username} --docker-password={your-pwd}
```
2. Configure the secret using the above steps on DCR:

```yaml
spec:
feSpec:
replicas: 1
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.fe-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
beSpec:
replicas: 3
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.be-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
systemInitialization:
initImage: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/alpine:latest
```
```yaml
spec:
feSpec:
replicas: 1
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.fe-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
beSpec:
replicas: 3
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.be-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
systemInitialization:
initImage: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/alpine:latest
```
### Be systemInitialization
Expand Down Expand Up @@ -143,7 +143,7 @@ Load balancing mode can be configured as follows:
The steps are as follows:
1. serviceType is ClusterIP (default policy)
2. You can create a load balancing service through the Alibaba Cloud console interface: Container Compute Service ACS -> Cluster List -> Cluster -> Service, and use the `Create` button.
3. Select the newly created LB in the interface for creating `service`, which will be bound to `service` and will also be deregistered when the `service` is deregistered. However, this `service` is not controlled by Doris-operator.
3. Select the newly created LB in the interface for creating `service`, which will be bound to `service` and will also be deregistered when the `service` is deregistered. However, this `service` is not controlled by Doris Operator.

## AWS EKS

Expand All @@ -155,7 +155,7 @@ It is recommended not to use the autonomous mode here, because the computing res

### Already exists cluster

On an existing cluster (non-auto mode), you can run the Doris cluster through Doris-operator, unless the cluster is restricted to use the privileged mode of K8s.
On an existing cluster (non-auto mode), you can run the Doris cluster through Doris Operator, unless the cluster is restricted to use the privileged mode of K8s.
It is recommended that the existing cluster configure a new node group to deploy and maintain Doris cluster resources separately, which involves the system settings for Doris BE operation and may adjust the system parameters of the host machine.

### assess DockerHub
Expand All @@ -166,15 +166,15 @@ If you need to access the DockerHub public image repository on EKS, you need to

Under EKS, EC2 instances completely belong to the current EKS user, and there is no situation where different user clusters affect each other in the resource pool and disable the K8s privileged mode.

- If your EKS allows privileged mode (allowed by default), you don't need to care about system parameters. Doris-operator will adjust system parameters for Doris operation by default.
- If your EKS allows privileged mode (allowed by default), you don't need to care about system parameters. Doris Operator will adjust system parameters for Doris operation by default.
- If privileged mode is not allowed, you need to adjust the following system parameters on the host:
- Modify the number of virtual memory areas: `sysctl -w vm.max_map_count=2000000` to adjust the maximum number of virtual memory mappings. View it through `sysctl vm.max_map_count`.
- Turn off transparent huge pages: Transparent huge pages may have an adverse effect on performance, so you need to turn it off. Judge by whether cat /sys/kernel/mm/transparent_hugepage/enabled contains never.
- Set the maximum number of open file handles: adjust the maximum number of file handles by modifying `/etc/security/limits.conf`. View it through `ulimit -n`.
- Disable swap: `swapoff -a` is used to disable all swap partitions and files. Verify with `swapon --show`, no output if not enabled.

### Storage
Doris-operator must use persistent configuration in the production environment to save the node storage. [EBS](https://aws.amazon.com/ebs) is recommended.
Doris Operator must use persistent configuration in the production environment to save the node storage. [EBS](https://aws.amazon.com/ebs) is recommended.

There are the following points to note:
- In the cluster configuration installation or management interface, add the EBS storage plug-in. If you use the EKS autonomous mode (not recommended), it is recommended to install EFS, and the storage plug-in needs to have the corresponding [role permissions](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ specific language governing permissions and limitations
under the License.
-->

Doris-operator 开发按照标准的 K8s 规范进行,兼容所有标准 K8s 平台,包含主流云厂商提供的和基于标准自建的 K8s 平台。本文主要提供在主流云厂商的容器化服务平台上,Doris-operator 使用的注意事项和一些使用建议。后续会更新更多云厂商及其产品的文档。
Doris Operator 开发按照标准的 K8s 规范进行,兼容所有标准 K8s 平台,包含主流云厂商提供的和基于标准自建的 K8s 平台。本文主要提供在主流云厂商的容器化服务平台上,Doris Operator 使用的注意事项和一些使用建议。后续会更新更多云厂商及其产品的文档。

## 阿里云容器服务 ACK

阿里云容器服务 ACK 属于 购买 ECS 实例后,托管容器化服务的,因此可以获得完全访问控制权限来进行相关系统参数调整,使用实例镜像:Alibaba Cloud Linux 3 当前系统参数完全满足运行 Doris 需求。不符合要求的也能够通过 K8s 特权模式在容器内进行修正,以保证稳定运行。
**阿里云ACK集群,使用 doris-operator 部署,大部分环境要求,ECS默认配置即可满足,未满足的,doris-operator 可自行修正**。 用户亦可手动修正, 如下:
**阿里云ACK集群,使用 Doris Operator 部署,大部分环境要求,ECS默认配置即可满足,未满足的,Doris Operator 可自行修正**。 用户亦可手动修正, 如下:

### 已存在集群

Expand Down Expand Up @@ -81,26 +81,27 @@ ACS 服务是以 K8s 为使用界面供给容器算力资源的云计算服务
在配置好镜像仓库和镜像中转的环境后,需要把 Doris 提供的官方镜像迁移到对应的阿里云镜像仓库中。

若使用私有镜像仓库开启了鉴权,可以参考以下步骤:
1. 需要提前设置类型为 `docker-registry``secret` 用以配置访问镜像仓库的身份认证信息。
```
kubectl create secret docker-registry image-hub-secret --docker-server={your-server} --docker-username={your-username} --docker-password={your-pwd}
```
2. 在 DCR 上配置使用上述步骤的 secret:
```yaml
spec:
feSpec:
replicas: 1
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.fe-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
beSpec:
replicas: 3
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.be-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
systemInitialization:
initImage: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/alpine:latest
```
1. 需要提前设置类型为 `docker-registry``secret` 用以配置访问镜像仓库的身份认证信息。
```shell
kubectl create secret docker-registry image-hub-secret --docker-server={your-server} --docker-username={your-username} --docker-password={your-pwd}
```
2. 在 DCR 上配置使用上述步骤的 secret:

```yaml
spec:
feSpec:
replicas: 1
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.fe-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
beSpec:
replicas: 3
image: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/doris.be-ubuntu:3.0.3
imagePullSecrets:
- name: image-hub-secret
systemInitialization:
initImage: crpi-4q6quaxa0ta96k7h-vpc.cn-hongkong.personal.cr.aliyuncs.com/selectdb-test/alpine:latest
```
### Be systemInitialization
Expand Down Expand Up @@ -140,7 +141,7 @@ ClusterIP模式,是 Operator 默认的网络模式,具体使用和访问方
步骤如下:
1. serviceType 为 ClusterIP(默认策略)
2. 可以通过阿里云控制台界面:容器计算服务 ACS -> 集群列表 -> 集群 -> 服务,通过 `创建` 按钮创建负载均衡服务。
3. 在创建 `服务` 的界面 选择新建的 LB ,会和 `service` 绑定,也会随着 该 `service` 的注销而注销。但是此 `service` 不受 Doris-operator 管控。
3. 在创建 `服务` 的界面 选择新建的 LB ,会和 `service` 绑定,也会随着 该 `service` 的注销而注销。但是此 `service` 不受 Doris Operator 管控。

## AWS 容器服务EKS

Expand All @@ -152,7 +153,7 @@ EKS 集群中运行的容器是托管在 EC2 实例上的, 需要根据 Doris

### 已有集群

在已有集群上(非自治模式),可以通过 Doris-operator 运行 Doris 集群,除非该集群被限制使用 K8s 的特权模式。
在已有集群上(非自治模式),可以通过 Doris Operator 运行 Doris 集群,除非该集群被限制使用 K8s 的特权模式。
建议已有集群配置新的节点组来单独进行 Doris 集群资源的部署和维护,涉及到 Doris BE 运行的系统设置,可能会对宿主机的系统参数进行调整。

### 镜像仓库访问
Expand All @@ -163,15 +164,15 @@ EKS 集群中运行的容器是托管在 EC2 实例上的, 需要根据 Doris

EKS 下,EC2 实例是完全属于当前 EKS 用户的,不存在不同用户集群在资源池中相互影响而禁掉 K8s 特权模式的情况。

- 若您的 EKS 允许特权模式(默认允许),则无需关心系统参数, Doris-operator 默认会为 Doris 运行调整系统参数。
- 若您的 EKS 允许特权模式(默认允许),则无需关心系统参数, Doris Operator 默认会为 Doris 运行调整系统参数。
- 若不允许特权模式,则需要在宿主机上进行如下系统参数调整:
- 修改虚拟内存区域数量:`sysctl -w vm.max_map_count=2000000` 调整虚拟内存的最大映射数量。通过 `sysctl vm.max_map_count` 查看。
- 关闭透明大页:透明大页对性能可能有不利影响,因此需要关闭它。通过 cat /sys/kernel/mm/transparent_hugepage/enabled 是否包含 never 来判断。
- 设置最大打开文件句柄数:通过修改 `/etc/security/limits.conf` 来调整最大文件句柄数。通过 `ulimit -n` 来查看。
- 禁用 swap:`swapoff -a` 用于禁用所有 swap 分区和文件。通过 `swapon --show` 验证, 未开启则无输出。

### 存储配置
Doris-operator 在生产环境一定需要用到持久化配置,用来保存节点状态,推荐 [EBS](https://aws.amazon.com/ebs) 存储。
Doris Operator 在生产环境一定需要用到持久化配置,用来保存节点状态,推荐 [EBS](https://aws.amazon.com/ebs) 存储。

需要有以下注意事项:
- 在集群配置安装或者管理界面,为其添加 EBS 存储插件,若使用 EKS 自治模式(不推荐),则推荐安装 EFS,并且存储插件需要拥有相应的 [角色权限](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html)
Expand Down
2 changes: 1 addition & 1 deletion sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
"type": "category",
"label": "Deploying on Kubernetes",
"items": [
{
"install/cluster-deployment/k8s-deploy/install-on-cloud", {
"type": "category",
"label": "Compute storage coupled",
"items": [
Expand Down

0 comments on commit a763109

Please sign in to comment.