diff --git a/.jekyll-metadata b/.jekyll-metadata new file mode 100644 index 0000000..e6bd149 Binary files /dev/null and b/.jekyll-metadata differ diff --git a/_posts/2024-01-06-proxmox.md b/_posts/2024-01-06-proxmox.md index 980040b..6f6eb94 100644 --- a/_posts/2024-01-06-proxmox.md +++ b/_posts/2024-01-06-proxmox.md @@ -14,14 +14,9 @@ tags: - tutorial --- - Proxmox VE 8.1 was release [late in November of 2023](https://www.proxmox.com/en/about/press-releases/proxmox-virtual-environment-8-1). This iteration is based on [Debian Bookworm](https://www.debian.org/News/2023/20230610) bringing a slew a new features like defaulting to Linux Kernel 6.5. -This post is a complete step by step installation of my Proxmox Cluster configuration. - -## What's covered in this guide? - -This guide walks you through a bare metal installation of Proxmox, followed by deploying a Home Assistant OS (HAOS) VM. To be more specific I cover: +This guide walks you through a complete step by step bare metal installation of Proxmox and configuration as in my homelab. ## Why Proxmox VE for Homelab? @@ -33,41 +28,71 @@ This guide walks you through a bare metal installation of Proxmox, followed by d ### Installing Proxmox VE 8.1 +1. Accept EULA +2. Select disk (ext4) for single drive +3. Set local + * Country : Canada + * Time zone : America/Toronto + * Keyboard : US English +4. Set credentials + * Country : str82THE.str82THE. + * Email : +5. Setup network + * FQDN : pve-x.lan + * IP : 192.168.10.2x/24 + * Netmask: 255.255.255.0 + * Gateway : 192.168.10.1 + * DNS : 192.168.10.1 + ## Proxmox Post-Install Configuration +```bash +bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/post-pve-install.sh)" +``` + ### Update (Optional) ### Proxmox Let’s Encrypt SSL Cert (Optional) ## Proxmox Two Factor Setup (Optional) -### Intel/AMD Microcode Update (Optional) +### AMD/Intel Microcode Update (Optional) -### Optimize CPU Power (Optional) +AMD and Intel releases new microcode for their processors from time to time. This is different from BIOS firmware, as the microcode runs inside the processor. It can fix CPU bugs or make other changes, as needed. You can use the following tteck script to download the latest AMD/Intel microcode and install it. A full system reboot will be needed for the microcode to take effect. -### VLAN Enable Proxmox (Optional) +```bash +bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/microcode.sh)" +``` -## Device Passthrough (Optional) +After the Proxmox host reboots you can run the following command to see if any microcode update is active. Not all CPUs need microcode updates, so you may well not see anything listed. -### Intel iGPU (Optional) +```bash +journalctl -k | grep -E "microcode" | head -n 1 +``` -### nvidia GPU (Optional) +### Optimize CPU Power (Optional) -### AMD GPU (Optional) +## Network -### Coral TPU (Optional) +### Realtek R8125B (Optional) -### PCI-E Device (Optional) +```bash +cd /tmp +wget +sudo dpkg -i realtek-r8125-dkms*.deb +sudo apt install --fix-broken -## Monitoring +lsmod | grep -i r8169 +lspci -k -## Proxmox 8.1 Notifications (Optional) +sudo apt update && sudo apt upgrade -y && sudo apt autoremove -y && sudo apt autoclean -y +``` -### Check SMART Monitoring (Optional) +### VLAN Enable Proxmox (Optional) ---- +## Storage -## Increase the Storage on "local" storage +### Local vs Local-LVM 1. Navigate to "datacenter -> storage" 2. Delete the "local-lvm" storage @@ -79,25 +104,19 @@ lvresize -l +100%FREE /dev/pve/root resize2fs /dev/mapper/pve-root ``` -## AMD/Intel Microcode +### NFS Shares -## Realtek R8125B driver +### CEPH Storage -cd /tmp -wget -sudo dpkg -i realtek-r8125-dkms*.deb -sudo apt install --fix-broken - -lsmod | grep -i r8169 -lspci -k +1. Wipe disk + ceph-volume lvm zap /dev/nvme0n1 --destroy -sudo apt update && sudo apt upgrade -y && sudo apt autoremove -y && sudo apt autoclean -y +## Device Passthrough (Optional) -## Intel iGPU Passthrough +### Intel iGPU (Optional) -```shell +```bash nano /etc/default/grub -#GRUB_CMDLINE_LINUX_DEFAULT="quiet" GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on" update-grub @@ -112,68 +131,61 @@ update-initramfs -u -k all reboot ``` -## nVidia GPU Passthrough +### nvidia GPU (Optional) -## Google Coral TPU Passthrough +### AMD GPU (Optional) -```shell +### Google Coral TPU (Optional) + +A very popular NVR solution that integrates well with Home Assistant is Frigate. It provides fantastic object and person detection from your camera streams. The Frigate project is a container, so it’s easy to deploy. + +Special hardware can be used in order to improve the image detection and general performance. A Google Coral Tensor Processing Unit (TPU) is a very common fairly affordable option. This device is availabe for purchase in a few configuration sch as USB, PCI-E and m.2 (A+E Key) PCI-E. Two of my Proxmox host have m.2 (A+E Key) Google Coral TPU installed so here how to enable this device passthrough. + +First on the Proxmox host, we need to modify the configuration of the Proxmox host where the Coral TPU is installed. Login to Proxmox and open a shell. + +1. Modify the GRUB configuration by running the following command: + +```bash nano /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_aspm=off initcall_blacklist=sysfb_init" +``` +```bash +nano /etc/default/grub update-grub +``` +```bash nano /etc/modules vfio vfio_iommu_type1 vfio_pci + kvmgt xengt vfio-mdev +``` +```bash update-initramfs -u -k all -reboot +``` +```bash nano /etc/modprobe.d/blacklist-apex.conf blacklist gasket blacklist apex options vfio-pci ids=1ac1:089a +``` +```bash update-initramfs -u -k all lsmod | grep apex +reboot ``` -# Proxmox Clustering Guide - -Here is the installation guide for Traefik and the complete configuration. - -## Installation - -1. Accept EULA -2. Select disk (ext4) for single drive -3. Set local - * Country : Canada - * Time zone : America/Toronto - * Keyboard : US English -4. Set credentials - * Country : str82THE.str82THE. - * Email : -5. Setup network - * FQDN : pve-x.lan - * IP : 192.168.10.2x/24 - * Netmask: 255.255.255.0 - * Gateway : 192.168.10.1 - * DNS : 192.168.10.1 - -## Configuration - -1. Run ansible playbook -1. Add NFS shares -2. Free local-lvm -3. Create backup schedule -4. Create Pools -5. Wake On Lan +### PCI-E Device (Optional) -## Clustering +## Clustering Guide 1. Log in to the first Proxmox server, select Datacenter, then Cluster, and select Create Cluster. 2. Give the cluster a name, then select create. The cluster will then be created and you’ll be able to join it from other Proxmox instances. @@ -187,58 +199,40 @@ pvecm create pve-cluster pvecm status pvecm join -## CEPH - -1. Wipe disk - ceph-volume lvm zap /dev/nvme0n1 --destroy - -## High Availability - -1. Create VM HA -2. Create LXC HA -3. Create Disable HA +### High Availability -## Terraform +1. Create HA Groups +2. Create Disable HA -1. Create Terraform user -2. Create API Tokens -3. Create template VM - -## Metric Server - -## Coral AI - - - - -## Extra nic +### Create Pools -Start by running lspci -v to confirm you’re running the r8169 driver for an r8168 NIC +1. Create HA Groups +2. Create Disable HA -Add the non-free repo for Bookworm: +### Wake On Lan -nano /etc/apt/sources.list +1. Create HA Groups +2. Create Disable HA -add the line: +## Backup -deb bookworm main non-free +### Promox Backup Server -then: +1. Create HA Groups +2. Create Disable HA -apt update && apt install r8168-dkms +## Workload -Purge the r8169 driver: +### Terraform -modprobe -r -v r8169 - -Reboot - -Confirm correct driver is in use: +1. Create Terraform user +2. Create API Tokens +3. Create template VM -lspci -v +## Monitoring -If that doesn't work then uncomment the blacklist line in the config and redo the steps: +### Metric Server -cd /etc/modprobe.d/ +### Notifications (Optional) -nano r8168-dkms.conf +### Check SMART Monitoring (Optional)