Skip to content

Commit

Permalink
ms8.2 - usability and documentation improvements
Browse files Browse the repository at this point in the history 
* Revamp documentation to reflect somewhat easier installation process
  using new single-binary builds
* Add strategy target for rockchip64 devices so we're not dependent on
  older kernel builds from the armbian-btf-kernel repo
* Clean up unneeded dependences from image build and set up most items
  to run on reboot automatically
* Small issue fixes here and there
  • Loading branch information
@atomic77
atomic77 committed Aug 12, 2024
1 parent 06d82ad commit b8e4983
Show file tree
Hide file tree
Showing 8 changed files with 150 additions and 170 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/build-btf-kernel.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ jobs:
- boardfamily: sunxi
branch: legacy
representative: orangepi-r1
# TODO Enable for R1plus and others as needed
# - boardfamily: rockchip64
# branch: current
# representative: rockpro64
# TODO Enable for others as needed
- boardfamily: rockchip64
branch: legacy
representative: orangepi-r1plus
fail-fast: false
env:
BOARD_FAMILY: ${{ matrix.boardfamily }}
Expand Down
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ run-root:
all:
GOARCH=arm GOOS=linux go build -o build/nethadone-arm-linux ./cmd/nethadone.go
GOARCH=arm64 GOOS=linux go build -o build/nethadone-arm64-linux ./cmd/nethadone.go
GOARCH=amd64 GOOS=linux go build -o build/nethadone-arm-linux ./cmd/nethadone.go
GOARCH=amd64 GOOS=linux go build -o build/nethadone-amd64-linux ./cmd/nethadone.go

build:
go build ./cmd/nethadone.go
Expand Down
8 changes: 4 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ etc.) while avoiding the downsides of excessive use.

Nethadone is in active development and has been tested on the following devices:

* [Orange Pi R1 Plus](http://www.orangepi.org/orangepiwiki/index.php/Orange_Pi_R1_Plus)
* [Orange Pi R1](http://www.orangepi.org/orangepiwiki/index.php/Orange_Pi_R1)
* [Orange Pi R1 Plus](http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/orange-pi-R1-Plus.html)
* [Orange Pi R1](http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/Orange-Pi-R1.html)
* [Orange Pi Zero](http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/Orange-Pi-Zero-LTS.html)

Any similar device using an armv7 or arm64 chipset should work.
Please check out [doc/install.md](doc/install.md) for details.
Please check out [doc/install.md](doc/install.md) for more details.

For development or testing on a local VM, please see
[doc/development.md](doc/development.md)
Expand Down
12 changes: 8 additions & 4 deletions cmd/nethadone.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,20 @@ import (

func main() {

wanIf := flag.String("wan-interface", "eth0", "Interface connecting out to internet")
lanIf := flag.String("lan-interface", "eth1", "Interface connected to local network")
wanIf := flag.String("wan-interface", "", "Interface connecting out to internet")
lanIf := flag.String("lan-interface", "", "Interface connected to local network")
configFile := flag.String("config-file", "nethadone.yml", "Configuration file")

flag.Parse()

config.ParseConfig(*configFile)
// Command line parameters override anything that might be in the config
config.Cfg.LanInterface = *lanIf
config.Cfg.WanInterface = *wanIf
if *lanIf != "" {
config.Cfg.LanInterface = *lanIf
}
if *wanIf != "" {
config.Cfg.WanInterface = *wanIf
}

log.Println("Configuration: ", repr.String(config.Cfg, repr.Indent(" ")))
database.Connect()
Expand Down
2 changes: 1 addition & 1 deletion config/nethadone.yml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# The WAN interface should have access to the internet
wan_interface: "eth0"
# The LAN interface is where your local devices are
lan_nterface: "eth1"
lan_interface: "eth1"

cfg_db: "/root/cfg.db"
dns_db: "/root/dns.db"
Expand Down
180 changes: 83 additions & 97 deletions doc/install.md
View file
Original file line number Diff line number Diff line change
@@ -1,101 +1,109 @@
# Installation

Nethadone still, unfortunately, requires a lot of manual steps to
get working.
It is a priority to make this installation process less cumbersome,
but anyone brave enough to attempt this in its current state
has my infinite gratitude!
## Requirements

## Requirements
Nethadone has been tested on the latest Ubuntu Noble-based versions of
Armbian (24.04 or 24.08). You can download an image for your board at
the [Armbian download page](https://www.armbian.com/download/?tx_maker=xunlong).

### Hardware

Nethadone has been tested on:
## Image prep

* VMWare-based VM (for development)
* Orange Pi R1Plus / R1Plus LTS
* 64-bit Rockchip RK3328
* 1GB RAM
* 2x GBit ethernet
* Wifi (not functional w/ Armbian yet)
* Orange Pi R1
* 32-bit Allwinner H3
* 2 x 100Mbit ethernet
* Wifi (working)
* 256MB / 512MB RAM
The contents of `scripts/customize-image.sh` will need to be run,
as root, in a fresh installation of the image you downloaded.
If you want to save time, you can prep using the chroot build
script, eg:

In theory, any device with two network interfaces should work, but I have not had a chance to test others.
```bash
cd scripts
./build.sh -i /tmp/Armbian_community_24.8.0-trunk.554_Orangepizero_noble_current_6.6.43.img -m ~/mnt -a arm
```

An SD card 16GB or greater is recommended.
`build.sh` makes liberal use of `sudo` so you may want to run this
on a VM or cloud server with a fast link.

### OS
Replace the `-i` and `-m` options with wherever you uncompressed the
Armbian image above, and an empty folder to use for a mount point.
The `-a` option indicates whether you are building for an arm64
or armv7 image.

> [!TIP]
> `build.sh` is just a chroot wrapper that pre-installs everything
required for nethadone to run. If you prefer to do the installation
of required packages directly on the SBC, you can copy paste from
`customize-image.sh` after you have flashed the stock image using
[Balena etcher](https://etcher.balena.io/) or whichever flashing tool you prefer.

Ubuntu-based 22.04 LTS or derivative (eg. Armbian 23.8)
Ubuntu 22.04 or 24.04 LTS and derivatives (i.e. Armbian) are working

Other OS versions may work, but the further away from Linux
6.1 (either newer or older), the more likely eBPF
issues may be encountered.
At a high level, the `customize-image.sh` script:

## OS Setup - Orange Pi R1+ (Arm64)
* Installs and configures compiler dependencies, bpftool and prometheus
* Creates a systemd service for nethadone to run on boot
* Downloads and installs a BTF-enabled kernel
* Downloads the nethadone binary from Github

> [!NOTE]
> I am in the process of streamlining this installation process,
> first for the armv7 Orange Pi R1 and eventually for the R1+.

### Base image
For the Orange Pi R1+, you need to get the Armbian 23.8.1 Jammy build:
## Nethadone post-installation

https://xogium.performanceservers.nl/archive/orangepi-r1plus/archive/Armbian_23.8.1_Orangepi-r1plus_jammy_current_6.1.50.img.xz
The easiest way to test out Nethadone is with a board like the
Zero or R1 that has functional wifi with Armbian. You can spin
up a [hotspot](https://ubuntu.com/core/docs/networkmanager/configure-wifi-access-points) with `nmcli`, eg:

This is because we will need to use a BTF-enabled kernel provided
by [daeuniverse](https://github.com/daeuniverse/armbian-btf-kernel),
and that is the latest version of Armbian provided there.
```bash
sudo nmcli dev wifi hotspot ifname wlan0 ssid nethadone password $password channel 8 band bg
```

### Image prep
The network-manager hotspot conveniently sets up a LAN for you
for any devices that connect wirelessly (default seems to be `10.42.0.0/24`), and NAT out to your wired interface, which should be plugged in
directly to your primary router.

The contents of `scripts/customize-image.sh` will need to be run,
as root, in a fresh installation of the image you downlaoded.
If you want to save time, you can prep this on your machine using
the chroot build script, eg:
> [!NOTE]
> The network-manager hotspot also has the advantage of setting up DNS
> caching automatically with `dnsmasq`. This makes it easier for
> nethadone to capture these packets, as clients receiving a local
> IP as a DNS server in DHCP configuration typically do not use DNS-over-HTTPS.
```bash
cd scripts
./build.sh -i /tmp/Armbian_23.8.1_Orangepi-r1plus_jammy_current_6.1.50.img -m ~/mnt
```
You can drop the `nmcli` command you used into `/etc/rc.local` to have
it start up on reboot.

Replace the `-i` and `-m` options with wherever you uncompressed the
Armbian image above, and an empty folder to use for a mount point.
### Adjusting cofiguration

`build.sh` is just a chroot wrapper that pre-installs everything
required for nethadone to run; if you prefer to do the installation
of required packages directly, you can copy paste from
`customize-image.sh` after you have flashed the base image using
[Balena etcher](https://etcher.balena.io/) or whichever flashing tool you prefer.
If you want to customize any parameters, or have different interface
names, you can create a `/etc/nethadone.yml` file.
See the sample [config](../config/nethadone.yml) for more details.

### Nethadone installation
## Wired-only Deployment

Once you have the image flashed and the device booted, at this
point you should be ready to install Nethadone.
Faster devices like the OrangePi R1+, as of my last tests, do not have
functional wifi with stock Armbian. This means they need to be introduced
between your wifi router and your device providing internet access using
the two ethernet ports.

First clone the repo as regular user (but one with sudo privilege)
This provides fast, full coverage for all devices in your home, but could take down your internet in the event of an issue with nethadone.
As this is still very much in development, it's recommended to stick
with a secondary access point for now.

```bash
git clone https://github.com/atomic77/nethadone
```
If you have access to an extra wifi router, this is an example of how
you can still make use of a wired-only device:

The default configuration file should work on an r1plus, and you
can run this to build and launch:
![Secondary network](nethadone-secondary-network.drawio.png)

In such as case, you will need to set up NAT for the interface
connected to the , eg. if `eth0` is your wan interface:

```bash
make run-root
iptables -t nat -A POSTROUTING -o end0 -j MASQUERADE
```

*TODO - properly document other steps required for this scenario*

## Runtime logs

If all goes well and nethadone is routing and inspecting traffic,
you should see log entries like:
you should see be able to see log entries like:

```bash
$ journalctl -u nethadone.service --follow
...
2024/07/20 15:48:58 Registering prometheus metrics
2024/07/20 15:48:58 Setting up SimpleLoadAverage policy
Expand Down Expand Up @@ -161,7 +169,7 @@ There is a basic admin interface where you can configure glob
groups, see the currently active policy and inspect currently
mapped DNS to IP addresses, it will be running at:

http://r1plus.local:3000/
http://orangepi-r1.local:3000/

mDNS is configured by default - if this address does not resolve
on your network, replace `r1plus.local` with whatever IP address
Expand All @@ -183,37 +191,15 @@ are being collected there using similar queries used by nethadone:

![prometheus](prometheus.png)

### Routing Configuration & Client setup

One of the design goals of Nethadone is to introduce minimal or
no client configuration beyond DHCP. The growing use of DNS over
HTTPS (DoH), unfortunately complicates this, as it prevents
Nethadone from using the typical UDP DNS packets to associate
a client-requested domain with an IP address.

In practice, the best way to work around this is to install
[pi-hole](https://github.com/pi-hole/pi-hole)
on the same device. Most client devices on your network, if provided
a local DNS server in the DHCP configuration, will automatically
switch to UDP-based DNS, and Nethadone will be able to inspect these
packets. Installing pi-hole on a non-raspberry pi is
straight-forward though not officially supported, I have created a
[youtube video with instructions](https://www.youtube.com/watch?v=m-mIglWyFcs) for installation on an orangepi zero that should work fine against a R1plus.

Alternatively, you can configure browsers to
[disable the use of DoH](https://www.expressvpn.com/support/troubleshooting/disable-dns-over-https/).


## Deployment

The recommended way of starting out with Nethadone is to create
a separate, 'protected' network, potentially using a old
spare wifi router. This is how it looks in my environment:

![Secondary network](nethadone-secondary-network.drawio.png)
## Other boards

Ideally, the wifi found on a board like the Orange Pi R1+ could
be used directly to create an isolated netwrk, but as of this time, Armbian does not have good support for the hardware.
Armbian does not ship with BTF-enabled kernels out of the box at this
time; the two kernel builds available for Armbian `BOARDFAMILY`
[`rockchip64`](https://github.com/armbian/build/blob/752ba047b3cb600095f981cbabb8be53d12bb9a2/config/boards/orangepi-r1plus.csc#L3)
and [`sunxi`](https://github.com/armbian/build/blob/752ba047b3cb600095f981cbabb8be53d12bb9a2/config/boards/orangepi-r1.csc#L3), are targeted for the Orange Pi R1 Plus
and R1 and Zero respectively. Others with the same family are likely,
but not certain to work.

If you are confident that your setup is good, you can then introduce Nethadone in between your primary access point and
your cable/DSL/fiber provider.
If you are interested in trying out another board and are having
trouble getting a BTF kernel working, please raise an issue and I'll
try adding it to the Github Action.
47 changes: 35 additions & 12 deletions scripts/build.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,43 @@
usage() {
echo "Usage: $0 -i <img> -m <mnt>"
echo ""
echo "This script will mount the provided image into the mount point and run"
echo "customize-image.sh on it. If you have downloaded an .xz file from the Armbian"
echo "site it's a good idea to keep the original compressed copy and uncompress the"
echo "image into /dev/shm if you have the memory"
echo ""
echo "Options:"
echo " -i <img> Armbian base image file (uncompressed)"
echo " -m <mnt> Mount point"
echo " -i <img> Armbian base image file to write into (uncompressed)"
echo " -m <mnt> Mount location to use for chroot "
echo " -a <arch> 'aarch64' for orangepi R1+ and similar, 'arm' for opizero/r1, and similar"
exit 1
}

cleanup () {
echo "Cleaning up.."
sudo umount ${mnt}/dev
sudo umount --lazy ${mnt}
sudo e2fsck -p -f ${loop}p1
sudo losetup -d ${loop}
}

trap cleanup ERR EXIT

img=""
mnt=""
arch=""

while getopts ":i:m:" opt; do
while getopts ":i:m:a:" opt; do
case $opt in
i)
img="$OPTARG"
;;
m)
mnt="$OPTARG"
;;
a)
arch="$OPTARG"
;;
\?)
echo "Invalid option: -$OPTARG" >&2
usage
Expand All @@ -37,13 +57,14 @@ while getopts ":i:m:" opt; do
esac
done

if [ -z "$img" ] || [ -z "$mnt" ]; then
echo "Error: Both -i and -m options are required."
if [ -z "$img" ] || [ -z "$mnt" ] || [ -z "$arch" ]; then
echo "Error: All flags are required."
usage
fi

set -e

# After trimming down the image see if we can reduce this to 3 or 2g
truncate -s 4G $img
loop=$(sudo losetup --partscan --show --nooverlap -f $img)
# This may fail on older versions of growpart and require
Expand All @@ -62,13 +83,15 @@ sudo mount --bind /dev ${mnt}/dev
sudo mkdir ${mnt}/tmp/overlay

# To enter into chroot w/ arm64 emulation
sudo cp /usr/bin/qemu-aarch64-static ${mnt}/usr/bin/
sudo cp /usr/bin/qemu-${arch}-static ${mnt}/usr/bin/
sudo cp customize-image.sh ${mnt}/tmp/

# Ubuntu Noble images have a symlink to /run/systemd/resolve/stub-resolv.conf which
# breaks DNS when in chroot. Swap out the current users' resolv.conf so we can
# install packages while in chroot
sudo mv ${mnt}/etc/resolv.conf ${mnt}/etc/resolv.conf.tmp
sudo cp -L /etc/resolv.conf ${mnt}/etc/
sudo chmod +x ${mnt}/tmp/customize-image.sh
# https://stackoverflow.com/questions/8157931/bash-executing-commands-from-within-a-chroot-and-switch-user#8157973
sudo chroot ${mnt} qemu-aarch64-static /bin/bash -c "/tmp/customize-image.sh"
sudo chroot ${mnt} qemu-${arch}-static /bin/bash -c "/tmp/customize-image.sh"

# Clean up
sudo umount ${mnt}
sudo e2fsck -p -f ${loop}p1
sudo losetup -d ${loop}
sudo mv -f ${mnt}/etc/resolv.conf.tmp ${mnt}/etc/resolv.conf
Loading

0 comments on commit b8e4983

Please sign in to comment.