Skip to content

Commit

Permalink
[Docs] add dqlite configuration to troubleshooting page (#1022)
Browse files Browse the repository at this point in the history
* add dqlite configuration to troubleshooting page

Co-authored-by: Louise K. Schmidtgen <[email protected]>
  • Loading branch information
nhennigan and louiseschmidtgen authored Jan 31, 2025
1 parent c24d820 commit c97b38a
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -376,3 +376,4 @@ DTMs
cybersecurity
uid
runc
unoptimized
42 changes: 42 additions & 0 deletions docs/src/snap/reference/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,47 @@ containerd-base-dir: $containerdBaseDir
EOF
```

## Increased memory usage in Dqlite

### Problem

The datastore used for {{product}} Dqlite, reported an [issue #196] of increased
memory usage over time. This was particularly evident in smaller clusters.

### Explanation

This issue was caused due to an inefficient resource configuration of
Dqlite for smaller clusters. The threshold and trailing parameters are
related to Dqlite transactions and must be adjusted. The threshold is
the number of transactions we allow before a snapshot is taken of the
leader. The trailing is the number of transactions we allow the follower
node to lag behind the leader before it consumes the updated snapshot of the
leader. Currently, the default snapshot configuration is 1024 for the
threshold and 8192 for trailing which is too large for small clusters. Only
setting the trailing parameter in a configuration yaml automatically sets the
threshold to 0. This leads to a snapshot being taken every transaction and
increased CPU usage.

### Solution

Apply a tuning.yaml custom configuration to the Dqlite datastore in order to
adjust the trailing and threshold snapshot values. The trailing parameter
should be twice the threshold value. Create the tuning.yaml
file and place it in the Dqlite directory
`/var/snap/k8s/common/var/lib/k8s-dqlite/tuning.yaml`:

```
snapshot:
trailing: 1024
threshold: 512
```

Restart Dqlite:

```
sudo snap restart snap.k8s.k8s-dqlite
```

<!-- LINKS -->

[lxd-install]: ../howto/install/lxd.md
Expand All @@ -120,3 +161,4 @@ EOF
[kubernetes-125923]: https://github.com/kubernetes/kubernetes/pull/125923
[kubernetes-122955-2020403422]: https://github.com/kubernetes/kubernetes/issues/122955#issuecomment-2020403422
[@haircommander]: https://github.com/haircommander
[issue #196]: https://github.com/canonical/k8s-dqlite/issues/196#issuecomment-2621527026

0 comments on commit c97b38a

Please sign in to comment.