Goals:
- (#4) Gentoo installed
- (#5) LUKS full disk encryption with performance tuning
- (#6) BTRFS raid0 for rootfs
- (#7) Striped LVM for VMs and other filesystems
- (#8) Ability to reallocate space between BTRFS and LVM
- (#9) BTRFS snapshots for rollback integrated into emerge
- (#10) System optimized with LTO, Graphite, PGO, and CFLAGS
- (#11) systemd-boot configured with secure boot
- (#12) Nvidia GPU working with CUDA and vGPU support
- (#13) KVM configured with virt-manager and libvirt support
- (#14) Podman setup for rootless containers
- (#15) TMP2 support for unlocking LUKS
- (#16) Yubikey support for unlocking LUKS
- (#17) i3-gaps for window manager
- (#18) spacemacs installed
- (#19) vscode installed
- (#20) Gentoo Containers for development via catalyst
The BIOS was reset to BIOS Defaults and the following changes were made:
- System Configuration -> Integrated NIC
- Enable UEFI Network Stack: Unchecked
- Radio Option: Enabled
- System Configuration -> SATA Operation
- Radio Option: AHCI
- System Configuration -> SMART Reporting
- Enable SMART Reporting: Checked
- System Configuration -> USB Configuration
- Enable USB Boot Support: Checked
- Enable External USB Port: Checked
- System Configuration -> Thunderbolt Adapter Configuration
- Thunderbolt: Checked
- Thunderbolt Boot Support: Checked
- Enable Thunderbolt (and PCIe behind TBT) Pre-boot Modules: Checked
- Radio Option: No Security
- System Configuration -> USB PowerShare
- Enable USB PowerShare: Checked
- Video -> Switchable Graphics
- Enable Switchable Graphics: Checked
- Discrete Graphics Controller: Unchecked
- Security -> UEFI Capsule Firmware Updates
- Enable UEFI Capsule Firmware Updates: Checked
- Security -> TPM 2.0 Security
- TPM On: Checked
- Clear: Checked
- Attestation Enable: Checked
- Key Storage Enable: Checked
- SHA-256: Checked
- Radio Option: Enabled
- Security -> Absolute
- Radio Option: Disabled
- Security -> SMM Security Mitigation
- SMM Security Mitigation: Checked
- Secure Boot -> Secure Boot Enable
- Secure Boot Enable: Checked
- Secure Boot -> Secure Boot Mode
- Radio Option: Audit Mode
- Intel Software Guard Extensions -> Intel SGX Enable
- Radio Option: Enabled
- Intel Software Guard Extensions -> Enclave Memory Size
- Radio Option: 32 MB
- Maintenance -> BIOS Recovery
- BIOS Recovery from Hard Drive: Unchecked
- SupportAssist System Resolution -> Auto OS Recovery Threshold
- Radio Option: OFF
- SupportAssist System Resolution -> SupportAssist OS Recovery
- SupportAssist OS Recovery: Unchecked
- SupportAssist System Resolution -> BIOSConnect
- BIOSConnect: Unchecked
The system was then rebooted twice to allow the TPM to fully reset.
The AMD64 LiveGUI USB Image was downloaded from here: https://www.gentoo.org/downloads/
Rufus was used to create a bootable USB drive from the image.
The USB drive was inserted into the laptop and the laptop was powered on.
F12 was pressed to enter the boot menu and the USB drive was selected.
The Boot LiveCD (kernel: gentoo)
option was selected.
Konsole was opened.
A root shell was opened using sudo su
and the Wi-Fi was connected to using nmtui
.
The gentoo repository was updated using the following commands:
emerge-webrsync
emerge --sync
Then the following commands were run to setup SSH access to allow for easier setup:
passwd gentoo
rc-update add sshd default
/etc/init.d/sshd start
LUKS vs. Direct disk access was tested using FIO. 4MiB blocks were used to test the sequential read/write performance and 4KiB blocks were used to test the random read/write performance. A single NVME disk was used for the tests. The results are summarized below:
Test Type | Queues Enabled | Submit from Crypt CPUs | Jobs | Random IO | Read (MiB/s) | Write (MiB/s) |
---|---|---|---|---|---|---|
Direct | #N/A | #N/A | 1 | FALSE | 1414.305713 | 1412.039197 |
Direct | #N/A | #N/A | 16 | FALSE | 2712.619886 | 2721.282936 |
Direct | #N/A | #N/A | 1 | TRUE | 133.6726561 | 133.4808588 |
Direct | #N/A | #N/A | 16 | TRUE | 754.250453 | 754.6367817 |
Encrypted | FALSE | TRUE | 1 | FALSE | 758.4574871 | 762.5904074 |
Encrypted | FALSE | FALSE | 1 | FALSE | 665.4001265 | 674.0659266 |
Encrypted | TRUE | TRUE | 1 | FALSE | 601.8466043 | 615.5794802 |
Encrypted | TRUE | FALSE | 1 | FALSE | 498.8501911 | 515.51408 |
Encrypted | FALSE | TRUE | 16 | FALSE | 2678.650202 | 2683.852674 |
Encrypted | FALSE | FALSE | 16 | FALSE | 2657.468244 | 2670.798386 |
Encrypted | TRUE | TRUE | 16 | FALSE | 2666.266011 | 2677.458804 |
Encrypted | TRUE | FALSE | 16 | FALSE | 2646.007566 | 2655.478645 |
Encrypted | FALSE | TRUE | 1 | TRUE | 112.1937857 | 112.0448322 |
Encrypted | FALSE | FALSE | 1 | TRUE | 109.5733004 | 109.4292946 |
Encrypted | TRUE | TRUE | 1 | TRUE | 100.8804913 | 100.6935177 |
Encrypted | TRUE | FALSE | 1 | TRUE | 96.65823555 | 96.48545551 |
Encrypted | FALSE | TRUE | 16 | TRUE | 750.838645 | 751.2814436 |
Encrypted | FALSE | FALSE | 16 | TRUE | 753.6204519 | 754.012641 |
Encrypted | TRUE | TRUE | 16 | TRUE | 850.8773651 | 851.1699362 |
Encrypted | TRUE | FALSE | 16 | TRUE | 882.7599688 | 883.0642529 |
Based on these results, the following flags were selected for the LUKS setup:
--perf-no_read_workqueue
--perf-no_write_workqueue
--perf-submit_from_crypt_cpus
The NVME disks were erased using the following commands:
wipefs -fa /dev/nvme0n1
wipefs -fa /dev/nvme1n1
wipefs -fa /dev/nvme2n1
The desired disk layout is one that is partitioned identically on all three disks. This allows for the disks to be used in RAID configurations with less complexity around certain calculations.
Data replication is desired for the ESP partition. This will be unencrypted and implemented via a systemd timer later in the setup process.
A separate RAID1 volume is desired for the /boot partition to make it easier to install other distros in the future. The /boot partition will be encrypted using LUKS1. GRUB does not support LUKS2 well and this will ensure better compatibility with other distros, if needed in the future.
There will be separate paritions for encrypted swap and the LVM volume group for the rest of the data. A LVM on LUKS approach will be used to allow for the ability to resize the LVM volume group in the future. BTRFS will be used for the root filesystem. RAID will be handled here at the BTRFS level so the filesystem has a better understanding of the physical layout of the disks.
They disks were partitioned using the following commands:
for _disk in /dev/nvme{0,1,2}n1; do
parted -a optimal "$_disk" --script mklabel gpt \
mkpart primary 1MiB 2049MiB \
mkpart primary 2049MiB 6145MiB \
mkpart primary 6145MiB 14377MiB \
mkpart primary 14377MiB 100% \
set 1 esp on
done
The ESP partitions were setup using the following commands:
mkfs.vfat -F32 -n esp0 /dev/nvme0n1p1
mkfs.vfat -F32 -n esp1 /dev/nvme1n1p1
mkfs.vfat -F32 -n esp2 /dev/nvme2n1p1
The /boot RAID1 was setup using the following commands:
mdadm --create /dev/md127 --name=cryptboot:luks --level 1 --raid-disks 3 --metadata 1.0 /dev/nvme{0,1,2}n1p2
The /boot encrypted LUKS1 partition was setup and formatted using the following commands:
cryptsetup luksFormat --type luks1 /dev/md127
cryptsetup luksOpen /dev/md127 cryptboot
mkfs.ext4 -L boot /dev/mapper/cryptboot
The encrypted swap partitions were setup, formatted, and activated using the following commands:
cryptsetup luksFormat --type luks1 /dev/nvme0n1p3
cryptsetup luksFormat --type luks1 /dev/nvme1n1p3
cryptsetup luksFormat --type luks1 /dev/nvme2n1p3
cryptsetup luksOpen /dev/nvme0n1p3 cryptswap0
cryptsetup luksOpen /dev/nvme1n1p3 cryptswap1
cryptsetup luksOpen /dev/nvme2n1p3 cryptswap2
mkswap -L swap0 /dev/mapper/cryptswap0
mkswap -L swap1 /dev/mapper/cryptswap1
mkswap -L swap2 /dev/mapper/cryptswap2
swapon -L swap0
swapon -L swap1
swapon -L swap2
The encrypted LVM volume group was setup and formatted using the following commands:
cryptsetup luksFormat --type luks2 /dev/nvme0n1p4
cryptsetup luksFormat --type luks2 /dev/nvme1n1p4
cryptsetup luksFormat --type luks2 /dev/nvme2n1p4
cryptsetup luksOpen \
--allow-discards \
--perf-no_read_workqueue \
--perf-no_write_workqueue \
--perf-submit_from_crypt_cpus \
/dev/nvme0n1p4 cryptdata0
cryptsetup luksOpen \
--allow-discards \
--perf-no_read_workqueue \
--perf-no_write_workqueue \
--perf-submit_from_crypt_cpus \
/dev/nvme1n1p4 cryptdata1
cryptsetup luksOpen \
--allow-discards \
--perf-no_read_workqueue \
--perf-no_write_workqueue \
--perf-submit_from_crypt_cpus \
/dev/nvme2n1p4 cryptdata2
pvcreate /dev/mapper/cryptdata0
pvcreate /dev/mapper/cryptdata1
pvcreate /dev/mapper/cryptdata2
vgcreate vg0 /dev/mapper/cryptdata{0,1,2}
The rootfs BTRFS RAID1 volume was setup and formatted using the following commands:
lvcreate -L 32G -n root0 vg0 /dev/mapper/cryptdata0
lvcreate -L 32G -n root1 vg0 /dev/mapper/cryptdata1
lvcreate -L 32G -n root2 vg0 /dev/mapper/cryptdata2
mkfs.btrfs -d raid1 -m raid1c3 --csum xxhash -n 4096 -L root /dev/mapper/vg0-root{0,1,2}
A thin pool was created for VM data using the following commands:
echo 'sys-fs/lvm2 thin' > /etc/portage/package.use/lvm2
emerge -av sys-fs/lvm2 sys-block/thin-provisioning-tools
lvcreate -L 768G -i3 -I4m -c 4m -Zn --thinpool thinpool0 vg0 /dev/mapper/cryptdata{0,1,2}
A BTRFS RAID0 volume was created for non-critical data using the following commands:
lvcreate -L 128G -n data0 vg0 /dev/mapper/cryptdata0
lvcreate -L 128G -n data1 vg0 /dev/mapper/cryptdata1
lvcreate -L 128G -n data2 vg0 /dev/mapper/cryptdata2
mkfs.btrfs -d raid0 -m raid1 --csum xxhash -n 4096 -L data /dev/mapper/vg0-data{0,1,2}
BTRFS subvolumes were created for the rootfs, home, and select var directories using the following commands:
mkdir -p /mnt/btrfs/{root,data}
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed \
/dev/mapper/vg0-root0 /mnt/btrfs/root
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed \
/dev/mapper/vg0-data0 /mnt/btrfs/data
btrfs subvolume create /mnt/btrfs/root/@gentoo
btrfs subvolume create /mnt/btrfs/data/@gentoo-home
btrfs subvolume create /mnt/btrfs/data/@gentoo-var-cache
btrfs subvolume create /mnt/btrfs/data/@gentoo-var-db
btrfs subvolume create /mnt/btrfs/data/@gentoo-var-log
btrfs subvolume create /mnt/btrfs/data/@gentoo-var-tmp
The chroot was mounted using the following commands:
mkdir /mnt/chroot
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo \
/dev/mapper/vg0-root0 /mnt/chroot
mkdir /mnt/chroot/boot
mount /dev/mapper/cryptboot /mnt/chroot/boot
mkdir /mnt/chroot/boot/efi{,.nvme1n1,.nvme2n1}
mount /dev/nvme0n1p1 /mnt/chroot/boot/efi
mount /dev/nvme1n1p1 /mnt/chroot/boot/efi.nvme1n1
mount /dev/nvme2n1p1 /mnt/chroot/boot/efi.nvme2n1
mkdir /mnt/chroot/home
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-home \
/dev/mapper/vg0-data0 /mnt/chroot/home
mkdir -p /mnt/chroot/var/{cache,db,log,tmp}
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-cache \
/dev/mapper/vg0-data0 /mnt/chroot/var/cache
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-db \
/dev/mapper/vg0-data0 /mnt/chroot/var/db
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-log \
/dev/mapper/vg0-data0 /mnt/chroot/var/log
mount -t btrfs \
-o compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-tmp \
/dev/mapper/vg0-data0 /mnt/chroot/var/tmp
The systemd | merged-usr
stage3 tarball link was copied from here:
https://www.gentoo.org/downloads/
The stage3 tarball was downloaded and verified using the following commands:
cd /mnt/chroot
wget ${STAGE3_URL}
wget ${STAGE3_URL}.CONTENTS.gz
wget ${STAGE3_URL}.DIGESTS
wget ${STAGE3_URL}.sha256
wget ${STAGE3_URL}.asc
sha256sum --check stage3-amd64-systemd-mergedusr-*.tar.xz.sha256
openssl dgst -r -sha512 stage3-amd64-systemd-mergedusr-*.tar.xz
openssl dgst -r -sha512 stage3-amd64-systemd-mergedusr-*.tar.xz.CONTENTS.gz
openssl dgst -r -blake2b512 stage3-amd64-systemd-mergedusr-*.tar.xz
openssl dgst -r -blake2b512 stage3-amd64-systemd-mergedusr-*.tar.xz.CONTENTS.gz
# compare above output to hashes in this file
cat stage3-amd64-systemd-mergedusr-*.tar.xz.DIGESTS
gpg --import /usr/share/openpgp-keys/gentoo-release.asc
gpg --verify stage3-amd64-systemd-mergedusr-*.tar.xz.asc
gpg --verify stage3-amd64-systemd-mergedusr-*.tar.xz.DIGESTS
gpg --verify stage3-amd64-systemd-mergedusr-*.tar.xz.sha256
The stage3 tarball was extracted using the following command:
tar xpvf stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner
The mounts needed to open a chroot were setup using the following commands:
mount --types proc /proc /mnt/chroot/proc
mount --rbind /sys /mnt/chroot/sys
mount --make-rslave /mnt/chroot/sys
mount --rbind /dev /mnt/chroot/dev
mount --make-rslave /mnt/chroot/dev
mount --bind /run /mnt/chroot/run
mount --make-slave /mnt/chroot/run
The chroot DNS was setup using the following commands:
cp --dereference /etc/resolv.conf /mnt/chroot/etc/
Bootstrapping the system consists of installing the portage tree, configuring portage, setting the profile, and updating the system, and the rebuilding everything twice to ensure all packages are built with the latest compiler.
After the initial update is completed, the GentooLTO overlay will be used to enable LTO and other optimizations. Doing this before the bootstrap will minimize the number of world rebuilds needed. Package testing needs enabled to ensure there are no issues with the packages being built.
ccache will also be setup to speed up rebuilds.
The following lines were uncommented in /mnt/chroot/etc/locale.gen
:
en_US ISO-8859-1
en_US.UTF-8 UTF-8
Then locale-gen
was run within the chroot to generate the locales.
Mirrors were selected using the following command:
mirrorselect -i -o >> /mnt/chroot/etc/portage/make.conf
The portage tree was configured using the following commands:
mkdir --parents /mnt/chroot/etc/portage/repos.conf
cp /mnt/chroot/usr/share/portage/config/repos.conf /mnt/chroot/etc/portage/repos.conf/gentoo.conf
The portage tree was synced using the following commands:
chroot /mnt/chroot
emerge-webrsync
emerge --sync
The /etc/portage/make.conf
file was updated in the chroot to the following:
## LTO Settings
## Use Flags
USE=""
## Compilter Settings
COMMON_FLAGS="-march=skylake -O2 -pipe"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
CHOST="x86_64-pc-linux-gnu"
CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3"
CMAKE_MAKEFILE_GENERATOR=ninja
MAKEOPTS="-j20"
## Directory Settings
PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
## Mirror List
GENTOO_MIRRORS="https://mirrors.rit.edu/gentoo/"
## Locale Settings
LC_MESSAGES=C.utf8
L10N="en"
## ACCEPT Settings
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="-* @FREE"
## Logging
PORTAGE_ELOG_CLASSES="info warn error log qa"
PORTAGE_ELOG_SYSTEM="echo save"
## Portage Features
FEATURES="binpkg-logs binpkg-multi-instance buildpkg candy ccache parallel-fetch parallel-install preserve-libs split-elog split-log test unmerge-logs"
PORTAGE_NICENESS=15
CCACHE_DIR=/var/cache/ccache
The default/linux/amd64/17.1/systemd/merged-usr
profile was selected using the following command:
eselect profile set default/linux/amd64/17.1/systemd/merged-usr
The system was updated using the following commands:
emerge -avuDU --with-bdeps=y --jobs=32 --load-average=20 @world
emerge -av --depclean
ccache was installed using the following command:
emerge -av dev-util/ccache
The ccache
feature was added to FEATURES
in /etc/portage/make.conf
. The CCACHE_DIR
variable
was set to /var/cache/ccache
in /etc/portage/make.conf
as well.
The ccache cache directory was configured using the following commands:
mkdir -p /mnt/chroot/var/cache/ccache
chown -R portage:portage /mnt/chroot/var/cache/ccache
cat <<EOF > /mnt/chroot/var/cache/ccache/ccache.conf
max_size = 64.0G
umask = 002
hash_dir = false
compiler_check = %compiler% -dumpversion
cache_dir_levels = 4
EOF
Git was installed so the /etc/portage directory could cloned from jhatler/jhatler-etc-portage.
emerge -av dev-vcs/git
The /etc/portage directory was cloned using the following commands:
rm -rf /etc/portage /var/db/repos/*
git clone https://github.com/jhatler/jhatler-etc-portage /etc/portage
emerge --sync # will take a while to clone everything
cd /var/db/repos/lto-overlay
git checkout workaroundCleanup # this branch is needed until the cleanup is complete
The cloned portage configuration enabled optimizations for GCC which need built before other packages can be optimized.
This was done by running the below command:
GCC_TESTS_NO_IGNORE_BASELINE=1 emerge -av1 sys-devel/gcc
That also creates a baseline copy of the GCC tests to be compared against later.
The test
feature is enabled in /etc/portage which means there will be many circular dependencies that need
to be addressed as part of bootstrapping.
The following command was run iteratively.
emerge -avuDU --with-bdeps=y --jobs=32 --load-average=20 @world
Each time a circular dependency was encountered, the packages with the interdepdencies were built with the following command:
FEATURES=-test emerge -av1 [PACKAGES] ...
Once all the circular test dependencies were built, the following command was run to generate a machine ID for systemd. This is needed by some packages to build.
systemd-machine-id-setup
dev-libs/glib
requires dev-util/desktop-file-utils
to be installed to bypass an ebuild bug which doesn't properly disable the dependent tests.
app-accessibility/at-spi2-core
requires app-editors/gedit
to be installed for the tests to pass (Gentoo Bug 678372).
dev-util/umockdev
needs x11-apps/xinput
and x11-drivers/xf86-input-synaptics
for the tests to pass.
The gnome-base/dconf
packages is needed for the net-libs/uhttpmock
tests to work.
All of these were installed using the below command:
FEATURES="-test" emerge -av dev-util/desktop-file-utils \
app-editors/gedit \
x11-apps/xinput \
x11-drivers/xf86-input-synaptics \
gnome-base/dconf
Then the system set was build once with tests enabled.
emerge -ave --with-bdeps=y --jobs=32 --load-average=20 @system
This gets the system to a point where the world set can be updated to point to the copy in /etc/portage from GitHub.
rm -f /var/lib/portage/world*
ln -s /etc/portage/world /var/lib/portage/world
ln -s /etc/portage/world_sets /var/lib/portage/world_sets
The system is in a state here where can be fully bootstrapped. Before doing that, we create a timestamp file. This will be used later to verify that all executables and libraries have been rebuilt.
touch /tmp/prebuild_checkpoint
The first world rebuild can now be done with this command.
emerge -av --emptytree --with-bdeps=y --jobs=32 --load-average=20 @world
Once it is complete, any configuration changes installed from the emerge should be applied with this command.
dispatch-conf
Then dependencies should be cleaned up and another rebuild performed.
emerge -av --depclean
emerge -av --emptytree --with-bdeps=y --jobs=32 --load-average=20 @world
Using the helpful instructions from Sakaki's unmaintained EFI Install Guide, the following commands were run to verify that the bootstrap is complete. See here for more details.
emerge -av --depclean
# below command checks for executables that are older than the checkpoint file
# It should produce no output
find / -type d -path /boot/efi -prune -o -path /proc -prune -o -type f -executable -not -newer /tmp/prebuild_checkpoint -print0 2>/dev/null | xargs -0 file --no-pad --separator="@@@" | grep -iv '@@@.* text'
# below command checks the libraries which may have been missed above
# It should also produce no output
find / -type d -path /boot/efi -prune -o -path /proc -prune -o -type f -not -executable -not -newer /tmp/prebuild_checkpoint -print0 2>/dev/null | xargs -0 file --no-pad --separator="@@@" | grep '@@@.*\( ELF\| ar archive\)'
The timezone was set to UTC:
ln -sf ../usr/share/zoneinfo/UTC /etc/localtime
A hostname was configured using the instructions here.
The journal was made peristent by updating /etc/systemd/journald.conf
on the following lines:
...
Storage=auto
Compress=yes
...
A root password was set using passwd
.
sshd was enabled using the command systemctl enable sshd
. Root logins were allowed by updating
/etc/ssh/sshd_config
on the below line.
...
PermitRootLogin yes
...
The systemd-networkd-wait-online
was masked using the below command:
systemctl mask systemd-networkd-wait-online.service
/etc/fstab
was updated to the following:
/dev/mapper/cryptboot /boot ext4 defaults 0 0
/dev/disk/by-label/esp0 /boot/efi vfat defaults 0 0
/dev/disk/by-label/esp1 /boot/efi.nvme1n1 vfat defaults 0 0
/dev/disk/by-label/esp2 /boot/efi.nvme2n1 vfat defaults 0 0
/dev/mapper/cryptswap0 none swap sw 0 0
/dev/mapper/cryptswap1 none swap sw 0 0
/dev/mapper/cryptswap2 none swap sw 0 0
/dev/mapper/vg0-root0 / btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo 0 0
/dev/mapper/vg0-data0 /home btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-home 0 0
/dev/mapper/vg0-data0 /var/cache btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-cache 0 0
/dev/mapper/vg0-data0 /var/db btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-db 0 0
/dev/mapper/vg0-data0 /var/log btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-log 0 0
/dev/mapper/vg0-data0 /var/tmp btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo-var-tmp 0 0
/dev/mapper/vg0-root0 /mnt/btrfs/root btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed 0 0
/dev/mapper/vg0-data0 /mnt/btrfs/data btrfs compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed 0 0
LVM was configured to allow discards by updating /etc/lvm/lvm.conf
to uncomment the below line:
...
issue_discards = 1
...
systemd-boot was installed using the below command.
bootctl install
Dracut needs configured to support the secure booting of the system.
The /etc/dracut.conf
file was cleared and the below two lines were added initially.
uefi="yes"
add_dracutmodules+=" bash btrfs crypt dm lvm mdraid systemd rescue rootfs-block "
The file needs the kernel_cmdline
parameter added still.
This requires the UUIDs of each of the LUKS partitions. These can be found using the below commands:
blkid /dev/md127 /dev/nvme{0,1,2}n1p{3,4}
The cmdline should begin with: rd.auto rd.lvm=1 rd.dm=1 rd.md=1 rd.luks=1 rd.luks.allow-discards
Then each LUKS volume needs added to the cmdline separated by spaces. The format is below and separated by newlines to make reading it easier.
The portions in [] should be replaced with the specified value from the above blkid
command.
rd.luks.uuid=[UUID of /dev/nvme0n1p3]
rd.luks.name=[UUID of /dev/nvme0n1p3]=cryptswap0
rd.luks.uuid=[UUID of /dev/nvme1n1p3]
rd.luks.name=[UUID of /dev/nvme1n1p3]=cryptswap1
rd.luks.uuid=[UUID of /dev/nvme2n1p3]
rd.luks.name=[UUID of /dev/nvme2n1p3]=cryptswap2
rd.luks.uuid=[UUID of /dev/md127]
rd.luks.name=[UUID of /dev/md127]=cryptboot
rd.luks.uuid=[UUID of /dev/nvme0n1p4]
rd.luks.name=[UUID of /dev/nvme0n1p4]=cryptdata0
rd.luks.uuid=[UUID of /dev/nvme1n1p4]
rd.luks.name=[UUID of /dev/nvme1n1p4]=cryptdata1
rd.luks.uuid=[UUID of /dev/nvme2n1p4]
rd.luks.name=[UUID of /dev/nvme2n1p4]=cryptdata2
Finally, the cmdline should be ended with the following settings to specify the root filesystem location and mount flags.
root=/dev/mapper/vg0-root0 rootflags=compress=zstd,discard=async,max_inline=0,space_cache=v2,ssd,commit=120,user_subvol_rm_allowed,subvol=@gentoo
The complete string comprising of the above three sections needs to be on a single line in quotes and added to the end of the
/etc/dracut.conf
file. That file should look like this when complete:
uefi="yes"
add_dracutmodules+=" bash btrfs crypt dm lvm mdraid systemd rescue rootfs-block "
kernel_cmdline="[COMBINDED CMDLINE]"
Now that dracut is configured, a UKI image can be created using the command below:
emerge --config sys-kernel/gentoo-kernel-bin
A shell was opened outside of the chroot and the following commands were run to unmount and close everything:
umount -R /mnt/chroot
umount /mnt/btrfs/root
umount /mnt/btrfs/data
swapoff -a
cryptsetup close /dev/mapper/cryptboot
mdadm --stop /dev/md127
cryptsetup close /dev/mapper/cryptswap0
cryptsetup close /dev/mapper/cryptswap1
cryptsetup close /dev/mapper/cryptswap2
vgchange -an vg0
cryptsetup close /dev/mapper/cryptdata0
cryptsetup close /dev/mapper/cryptdata1
cryptsetup close /dev/mapper/cryptdata2
The system was rebooted using the following command:
reboot
Once the UKI image boots, you will be prompted for the LUKS passphrase. Keeping the same passphrase for each LUKS volume will allow for it to be entered once and used for all volumes.