opir1 (armv7) support working with new BTF kernel build

begin streamlining install scripts and docs

README.md

@@ -12,14 +12,16 @@ etc.) while avoiding the downsides of excessive use.
 
 ## Getting Started
 
-Nethadone is in active development and only been tested on an 
-[Orange Pi R1 Plus](http://www.orangepi.org/orangepiwiki/index.php/Orange_Pi_R1_Plus).
-If you are interested in giving it a try on a similar device,
-please check out `doc/install.md` for details, feedback and
-contributions are more than welcome!
+Nethadone is in active development and has been tested on the following devices:
+
+* [Orange Pi R1 Plus](http://www.orangepi.org/orangepiwiki/index.php/Orange_Pi_R1_Plus)
+* [Orange Pi R1](http://www.orangepi.org/orangepiwiki/index.php/Orange_Pi_R1)
+
+Any similar device using an armv7 or arm64 chipset should work.
+Please check out [doc/install.md](doc/install.md) for details.
 
 For development or testing on a local VM, please see 
-`doc/development.md` 
+[doc/development.md](doc/development.md)
 
 ## How does it work? 
 
@@ -38,9 +40,9 @@ As a user continues to compulsively use a given site, traffic
 to flagged IPs gradually works its way down the list above.
 
 Nethadone currently only supports a single policy, for more details
-see `policy/README.md`.
+see [policy/README.md](policy/README.md).
 
-For more details on bandwidth classes (i.e. qdisc configuration), see `doc/qdisc.md`
+For more details on bandwidth classes (i.e. qdisc configuration), see [doc/qdisc.md](doc/qdisc.md).
 
 ### Traffic flow
 

doc/install.md

@@ -18,12 +18,10 @@ Nethadone has been tested on:
     * 1GB RAM
     * 2x GBit ethernet
     * Wifi (not functional w/ Armbian yet)
-
-In progress:
-
 * Orange Pi R1
     * 32-bit Allwinner H3
     * 2 x 100Mbit ethernet
+    * Wifi (working)
     * 256MB / 512MB RAM 
 
 In theory, any device with two network interfaces should work, but I have not had a chance to test others.
@@ -32,13 +30,18 @@ An SD card 16GB or greater is recommended.
 
 ### OS 
 
-Ubuntu 22.04 LTS or derivative (eg. Armbian 23.8)
+Ubuntu-based 22.04 LTS or derivative (eg. Armbian 23.8)
+Ubuntu 22.04 or 24.04 LTS and derivatives (i.e. Armbian) are working
 
 Other OS versions may work, but the further away from Linux 
 6.1 (either newer or older), the more likely eBPF
 issues may be encountered.
 
-## OS Setup
+## OS Setup - Orange Pi R1+ (Arm64)
+
+> [!NOTE]  
+> I am in the process of streamlining this installation process, 
+> first for the armv7 Orange Pi R1 and eventually for the R1+. 
 
 ### Base image
 For the Orange Pi R1+, you need to get the Armbian 23.8.1 Jammy build:
@@ -49,11 +52,6 @@ This is because we will need to use a BTF-enabled kernel provided
 by [daeuniverse](https://github.com/daeuniverse/armbian-btf-kernel),
 and that is the latest version of Armbian provided there.
 
-A TODO item is to provide updated linux kernels for recent versions 
-of Armbian, or an easy to use build script using the Armbian 
-customize-image hook that preps the image plus the required kernel
-options. 
-
 ### Image prep
 
 The contents of `scripts/customize-image.sh` will need to be run, 

ebpf/Makefile

@@ -25,7 +25,7 @@ vmlinux.h:
 	bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h		
 
 build-throttler:
-	clang -g -O2 -I/usr/include/aarch64-linux-gnu -Wall -target bpf -c throttle.bpf.c -o throttle.o
+	clang -g -O2 -I/usr/include/aarch64-linux-gnu -I/usr/arm-linux-gnueabi/include  -Wall -target bpf -c throttle.bpf.c -o throttle.o
 
 rtarg = dnspkt
 direction = ingress

scripts/customize-image.sh

@@ -1,43 +1,102 @@
 #!/bin/bash
 
-apt-get update
+case "$(arch)" in
+  'aarch64')
+    prom_url='https://github.com/prometheus/prometheus/releases/download/v2.53.1/prometheus-2.53.1.linux-arm64.tar.gz'
+    go_url='https://go.dev/dl/go1.22.3.linux-arm64.tar.gz'
+    kern_url='https://github.com/daeuniverse/armbian-btf-kernel/releases/download/main-2023-06-17/kernel-rockchip64-current_23.08.0-trunk--6.1.34-Sca87-Dbeb1-Pa401-C3053Hfe66-HK01ba-Vc222-B76dc.tar'
+    ;;
+  'armv7l')
+    prom_url='https://github.com/prometheus/prometheus/releases/download/v2.53.1/prometheus-2.53.1.linux-armv7.tar.gz'
+    go_url='https://go.dev/dl/go1.22.3.linux-armv6l.tar.gz'
+    kern_url='https://github.com/atomic77/nethadone/releases/download/btf-kernel/kernel-legacy-sunxi-orangepi-r1.tar.gz'
+    ;;
+  'x86_64')
+    prom_url='https://github.com/prometheus/prometheus/releases/download/v2.53.1/prometheus-2.53.1.linux-amd64.tar.gz'
+    go_url='https://go.dev/dl/go1.22.3.linux-arm64.tar.gz'
+    # Most x86_64 builds for virtual machine testing use should have BTF enabled
+  '*')
+    echo "Unsupported architecture $(arch), exiting."
+    exit 1
+    ;;
 
-# For Debian bookworm
-apt-get install -y apt-transport-https ca-certificates curl clang llvm jq \
-        libelf-dev libpcap-dev libbfd-dev binutils-dev build-essential make \
-        bpfcc-tools python3-pip vim libbpf-dev \
-        avahi-daemon bcc python-is-python3 \
-        python3-dnslib python3-cachetools # for tcpconnect dns tracing in python
+esac
 
-# For ubuntu jammy
-apt-get install linux-tools-common -y
+apt-get update -y
 
-wget https://go.dev/dl/go1.22.3.linux-arm64.tar.gz
-rm -rf /usr/local/go && tar -C /usr/local -xzf go1.22.3.linux-arm64.tar.gz
+apt-get install -y apt-transport-https ca-certificates curl clang llvm jq \
+        libelf-dev libpcap-dev libbfd-dev binutils-dev build-essential make \
+        vim libbpf-dev avahi-daemon linux-tools-common dnsmasq 
 
+wget ${go_url}
+rm -rf /usr/local/go && tar -C /usr/local -xzf go1.*
+rm go*.tar.gz
 echo "export PATH=\$PATH:/usr/local/go/bin" >> /etc/profile
 
+#####
 # Local prometheus for metrics collection
-wget https://github.com/prometheus/prometheus/releases/download/v2.52.0/prometheus-2.52.0.linux-arm64.tar.gz
-tar -C /usr/local -xzf prometheus-2.52.0.linux-arm64.tar.gz
+wget ${prom_url} 
+tar -C /usr/local/bin --strip-components 1 -xzf prometheus-*.tar.gz
+rm prometheus-*.tar.gz
+
+cp /usr/local/bin/prometheus.yml /etc
+mkdir -p /var/lib/prometheus
 
 cat >> /etc/prometheus.yml << EOF
   - job_name: "nethadone"
     static_configs:
       - targets: ["localhost:3000"]
 EOF
 
-# TODO Add prometheus systemd unit, data directory for autostart
-# TODO and ip masq setup / ipv4 forward on boot, or enable
-# in nethadone directly, i.e.
-# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
-# sysctl net.ipv4.ip_forward=1
+cat <<EOF  > /etc/systemd/system/prometheus.service
+[Unit]
+Description=Prometheus
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+User=root
+Restart=on-failure
+
+ExecStart=/usr/local/bin/prometheus \
+  --config.file=/etc/prometheus.yml \
+  --storage.tsdb.path=/var/lib/prometheus
+
+[Install]
+WantedBy=multi-user.target
+
+EOF
+
+systemctl daemon-reload
+systemctl enable prometheus
+
+######
+# NAT forwarding will need to be in place for routing to work,
+# but the interface may change depending on your configuration.
+# TODO Configure this in nethadone directly 
+#echo "iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE" >> /etc/rc.local
+
+echo "net.ipv4.ip_forward = 1" > /etc/sysctl.d/20-nethadone.conf
+
+
+# For wifi-enabled devices, network-manager does a nice job of setting up an AP
+# that automatically proxies DNS requests, eliminating the need for pihole.
+# eg:
+# sudo nmcli dev wifi hotspot ifname wlan0 ssid nethadone password "mypass"
+# 
 
 #####
 # Grab the custom BTF-enabled kernel from daeuniverse' repo
-wget https://github.com/daeuniverse/armbian-btf-kernel/releases/download/main-2023-06-17/kernel-rockchip64-current_23.08.0-trunk--6.1.34-Sca87-Dbeb1-Pa401-C3053Hfe66-HK01ba-Vc222-B76dc.tar
-tar xf kernel-rockchip64-current_23.08.0-trunk--6.1.34-Sca87-Dbeb1-Pa401-C3053Hfe66-HK01ba-Vc222-B76dc.tar
-yum install ./linux-*
+if [ $(arch) != 'x86_64' ]; then
+  wget ${kern_url}
+  tar xvf kernel-*
+  apt-get install -y ./linux-*
+  rm kernel*.tar
+fi
+
+if [ $(arch) == 'armv7' ]; then
+  apt-get install libc6-dev-armel-cross -y
+fi 
 
 # Build BPFTool from source - repo packages don't seem to work properly
 mkdir ~/src
@@ -46,6 +105,8 @@ git clone --recurse-submodules https://github.com/libbpf/bpftool.git
 cd bpftool/src
 make -j 4
 make install
+rm -rf ~/src
 
-
+# TODO - Copy in pre-built nethadone binary from github to avoid git
+# checkout and golang compiler