Skip to content

Commit

Permalink
Move Advanced OS preop into requirements (#199)
Browse files Browse the repository at this point in the history
* Move Advanced OS preop into requirements

Signed-off-by: Derek Nola <[email protected]>
  • Loading branch information
dereknola authored Nov 20, 2023
1 parent 428ec4c commit 18f3ba9
Show file tree
Hide file tree
Showing 6 changed files with 79 additions and 78 deletions.
69 changes: 0 additions & 69 deletions docs/advanced/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,75 +280,6 @@ An example of disabling auto-starting and service enablement with the install sc
curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START=true INSTALL_K3S_SKIP_ENABLE=true sh -
```

## Additional OS Preparations

### Old iptables versions

Several popular Linux distributions ship a version of iptables that contain a bug which causes the accumulation of duplicate rules, which negatively affects the performance and stability of the node. See [Issue #3117](https://github.com/k3s-io/k3s/issues/3117) for information on how to determine if you are affected by this problem.

K3s includes a working version of iptables (v1.8.8) which functions properly. You can tell K3s to use its bundled version of iptables by starting K3s with the `--prefer-bundled-bin` option, or by uninstalling the iptables/nftables packages from your operating system.

:::info Version Gate

The `--prefer-bundled-bin` flag is available starting with the 2022-12 releases (v1.26.0+k3s1, v1.25.5+k3s1, v1.24.9+k3s1, v1.23.15+k3s1).

:::

### Red Hat Enterprise Linux / CentOS / Fedora

It is recommended to turn off firewalld:
```bash
systemctl disable firewalld --now
```

If you wish to keep firewalld enabled, by default, the following rules are required:
```bash
firewall-cmd --permanent --add-port=6443/tcp #apiserver
firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 #pods
firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 #services
firewall-cmd --reload
```

Additional ports may need to be opened depending on your setup. See [Inbound Rules](../installation/requirements.md#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly.

If enabled, it is required to disable nm-cloud-setup and reboot the node:
```bash
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
reboot
```

### Ubuntu / Debian

It is recommended to turn off ufw (uncomplicated firewall):
```bash
ufw disable
```

If you wish to keep ufw enabled, by default, the following rules are required:
```bash
ufw allow 6443/tcp #apiserver
ufw allow from 10.42.0.0/16 to any #pods
ufw allow from 10.43.0.0/16 to any #services
```

Additional ports may need to be opened depending on your setup. See [Inbound Rules](../installation/requirements.md#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly.

### Raspberry Pi

Raspberry Pi OS is Debian based, and may suffer from an old iptables version. See [workarounds](#old-iptables-versions).

Standard Raspberry Pi OS installations do not start with `cgroups` enabled. **K3S** needs `cgroups` to start the systemd service. `cgroups`can be enabled by appending `cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`.

Example cmdline.txt:
```
console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory
```

Starting with Ubuntu 21.10, vxlan support on Raspberry Pi has been moved into a separate kernel module.
```bash
sudo apt install linux-modules-extra-raspi
```

## Running K3s in Docker

There are several ways to run K3s in Docker:
Expand Down
2 changes: 1 addition & 1 deletion docs/faq/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ If you are having an issue with deploying K3s, you should:

1) Check the [Known Issues](../known-issues/known-issues.md) page.

2) Check that you have resolved any [Additional OS Preparation](../advanced/advanced.md#additional-os-preparations). Run `k3s check-config` and ensure that it passes.
2) Check that you have resolved any [Additional OS Preparation](../installation/requirements.md#operating-systems). Run `k3s check-config` and ensure that it passes.

3) Search the K3s [Issues](https://github.com/k3s-io/k3s/issues) and [Discussions](https://github.com/k3s-io/k3s/discussions) for one that matches your problem.

Expand Down
2 changes: 1 addition & 1 deletion docs/installation/network-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This page describes K3s network configuration options, including configuration o

* Flannel options can only be set on server nodes, and must be identical on all servers in the cluster.
* The default backend for Flannel is `vxlan`. To enable encryption, use the `wireguard-native` backend.
* Using `vxlan` on Rasperry Pi with recent versions of Ubuntu requires [additional preparation](../advanced/advanced.md#raspberry-pi).
* Using `vxlan` on Rasperry Pi with recent versions of Ubuntu requires [additional preparation](./requirements.md?os=pi#operating-systems).
* Using `wireguard-native` as the Flannel backend may require additional modules on some Linux distributions. Please see the [WireGuard Install Guide](https://www.wireguard.com/install/) for details.
The WireGuard install steps will ensure the appropriate kernel modules are installed for your operating system.
You must ensure that WireGuard kernel modules are available on every node, both servers and agents, before attempting to use the WireGuard Flannel backend.
Expand Down
65 changes: 60 additions & 5 deletions docs/installation/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,65 @@ Prior to May 2023 releases (v1.24.14+k3s1, v1.25.10+k3s1, v1.26.5+k3s1, v1.27.2+
K3s is expected to work on most modern Linux systems.

Some OSs have additional setup requirements:

- If you are using **RHEL/CentOS/Fedora**, follow [these steps](../advanced/advanced.md#red-hat-enterprise-linux--centos--fedora).
- If you are using **Ubuntu/Debian**, follow [these steps](../advanced/advanced.md#ubuntu--debian).
- If you are using **Raspberry Pi OS**, follow [these steps](../advanced/advanced.md#raspberry-pi).
<Tabs queryString="os">
<TabItem value="rhel" label="Red Hat Enterprise Linux / CentOS / Fedora">

It is recommended to turn off firewalld:
```bash
systemctl disable firewalld --now
```

If you wish to keep firewalld enabled, by default, the following rules are required:
```bash
firewall-cmd --permanent --add-port=6443/tcp #apiserver
firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 #pods
firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 #services
firewall-cmd --reload
```

Additional ports may need to be opened depending on your setup. See [Inbound Rules](#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly.

If enabled, it is required to disable nm-cloud-setup and reboot the node:
```bash
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
reboot
```
</TabItem>
<TabItem value="debian" label="Ubuntu / Debian">

Older Debian release may suffer from a known iptables bug. See [Known Issues](../known-issues/known-issues.md#iptables).

It is recommended to turn off ufw (uncomplicated firewall):
```bash
ufw disable
```

If you wish to keep ufw enabled, by default, the following rules are required:
```bash
ufw allow 6443/tcp #apiserver
ufw allow from 10.42.0.0/16 to any #pods
ufw allow from 10.43.0.0/16 to any #services
```

Additional ports may need to be opened depending on your setup. See [Inbound Rules](#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly.
</TabItem>
<TabItem value="pi" label="Raspberry Pi">

Raspberry Pi OS is Debian based, and may suffer from a known iptables bug. See [Known Issues](../known-issues/known-issues.md#iptables).

Standard Raspberry Pi OS installations do not start with `cgroups` enabled. **K3S** needs `cgroups` to start the systemd service. `cgroups`can be enabled by appending `cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`.

Example cmdline.txt:
```
console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory
```

Starting with Ubuntu 21.10, vxlan support on Raspberry Pi has been moved into a separate kernel module.
```bash
sudo apt install linux-modules-extra-raspi
```
</TabItem>
</Tabs>

For more information on which OSs were tested with Rancher managed K3s clusters, refer to the [Rancher support and maintenance terms.](https://rancher.com/support-maintenance-terms/)

Expand Down Expand Up @@ -89,7 +144,7 @@ Flannel relies on the [Bridge CNI plugin](https://www.cni.dev/plugins/current/ma

Typically, all outbound traffic is allowed.

Additional changes to the firewall may be required depending on the OS used. See [Additional OS Preparations](../advanced/advanced.md#additional-os-preparations).
Additional changes to the firewall may be required depending on the OS used.

## Large Clusters

Expand Down
17 changes: 15 additions & 2 deletions docs/known-issues/known-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,22 @@ If you plan to use K3s with docker, Docker installed via a snap package is not r

### Iptables

If you are running iptables in nftables mode instead of legacy you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues.
If you are running iptables v1.6.1 and older in nftables mode you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues or running iptables legacy mode.

Additionally, versions 1.8.0-1.8.4 have known issues that can cause K3s to fail. See [Additional OS Preparations](../advanced/advanced.md#old-iptables-versions) for workarounds.
```
update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
```

Iptables versions 1.8.0-1.8.4 have known issues that can cause K3s to fail. Several popular Linux distributions ship with these versions by default. One bug causes the accumulation of duplicate rules, which negatively affects the performance and stability of the node. See [Issue #3117](https://github.com/k3s-io/k3s/issues/3117) for information on how to determine if you are affected by this problem.

K3s includes a working version of iptables (v1.8.8) which functions properly. You can tell K3s to use its bundled version of iptables by starting K3s with the `--prefer-bundled-bin` option, or by uninstalling the iptables/nftables packages from your operating system.

:::info Version Gate

The `--prefer-bundled-bin` flag is available starting with the 2022-12 releases (v1.26.0+k3s1, v1.25.5+k3s1, v1.24.9+k3s1, v1.23.15+k3s1).

:::

### Rootless Mode

Expand Down
2 changes: 2 additions & 0 deletions docs/quick-start/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 10

This guide will help you quickly launch a cluster with default options. The [installation section](../installation/installation.md) covers in greater detail how K3s can be set up.

Make sure your nodes meet the the [requirements](../installation/requirements.md) before proceeding.

For information on how K3s components work together, refer to the [architecture section.](../architecture/architecture.md)

:::info
Expand Down

0 comments on commit 18f3ba9

Please sign in to comment.