diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index b2767349..6a03007a 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -9,10 +9,9 @@ jobs:
name: Deploy docs
runs-on: ubuntu-latest
steps:
- - name: Checkout master
- uses: actions/checkout@v1
-
- - name: Deploy docs
- uses: mhausenblas/mkdocs-deploy-gh-pages@master
- env:
- GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ - uses: actions/checkout@v2
+ - uses: actions/setup-python@v2
+ with:
+ python-version: 3.x
+ - run: pip3 install -r requirements-mkdocs.txt
+ - run: mkdocs gh-deploy --force
diff --git a/.templates/wireguard/use-container-dns.sh b/.templates/wireguard/use-container-dns.sh
index 18737322..7f13f219 100644
--- a/.templates/wireguard/use-container-dns.sh
+++ b/.templates/wireguard/use-container-dns.sh
@@ -1,8 +1,9 @@
# Forward DNS requests from remote WireGuard clients to the default
# gateway on the internal bridged network that the WireGuard container
-# is attached to. This results in queries being sent to any other
-# container on the same internal bridged network that is listening
-# on port 53 (eg PiHole, AdGuardHome or bind9).
+# is attached to. The gateway routes queries out from the bridged network to
+# the host's network. This results in queries being sent to any daemon or
+# container that is listening on host port 53 (eg PiHole, AdGuardHome, dnsmasq
+# or bind9).
#
# Acknowledgement: @ukkopahis
diff --git a/docs/Accessing-your-Device-from-the-internet.md b/docs/Basic_setup/Accessing-your-Device-from-the-internet.md
similarity index 95%
rename from docs/Accessing-your-Device-from-the-internet.md
rename to docs/Basic_setup/Accessing-your-Device-from-the-internet.md
index 405d09ee..fca16019 100644
--- a/docs/Accessing-your-Device-from-the-internet.md
+++ b/docs/Basic_setup/Accessing-your-Device-from-the-internet.md
@@ -6,14 +6,14 @@ From time to time the IP address that your ISP assigns changes and it's difficul
Secondly, how do you get into your home network? Your router has a firewall that is designed to keep the rest of the internet out of your network to protect you. The solution to that is a Virtual Private Network (VPN) or "tunnel".
-## Dynamic DNS
+## Dynamic DNS
There are two parts to a Dynamic DNS service:
1. You have to register with a Dynamic DNS service provider and obtain a domain name that is not already taken by someone else.
2. Something on your side of the network needs to propagate updates so that your chosen domain name remains in sync with your router's dynamically-allocated public IP address.
-### Register with a Dynamic DNS service provider
+### Register with a Dynamic DNS service provider
The first part is fairly simple and there are quite a few Dynamic DNS service providers including:
@@ -24,7 +24,7 @@ The first part is fairly simple and there are quite a few Dynamic DNS service pr
Some router vendors also provide their own built-in Dynamic DNS capabilities for registered customers so it's a good idea to check your router's capabilities before you plough ahead.
-### Dynamic DNS propagation
+### Dynamic DNS propagation
The "something" on your side of the network propagating WAN IP address changes can be either:
@@ -39,7 +39,7 @@ A behind-the-router technique usually relies on sending updates according to a s
> This seems to be a problem for DuckDNS which takes a beating because almost every person using it is sending an update bang-on five minutes.
-### DuckDNS client
+### DuckDNS client
IOTstack provides a solution for DuckDNS. The best approach to running it is:
@@ -99,7 +99,7 @@ A null result indicates failure so check your work.
Remember, the Domain Name System is a *distributed* database. It takes *time* for changes to propagate. The response you get from directing a query to ns1.duckdns.org may not be the same as the response you get from any other DNS server. You often have to wait until cached records expire and a recursive query reaches the authoritative DuckDNS name-servers.
-#### Running the DuckDNS client automatically
+#### Running the DuckDNS client automatically
The recommended arrangement for keeping your Dynamic DNS service up-to-date is to invoke `duck.sh` from `cron` at five minute intervals.
@@ -152,7 +152,7 @@ $ cat /dev/null >~/Logs/duck.log
### WireGuard
-WireGuard is supplied as part of IOTstack. See [WireGuard documentation](https://sensorsiot.github.io/IOTstack/Containers/WireGuard.html).
+WireGuard is supplied as part of IOTstack. See [WireGuard documentation](../Containers/WireGuard.md).
### PiVPN
diff --git a/docs/Backup-and-Restore.md b/docs/Basic_setup/Backup-and-Restore.md
similarity index 99%
rename from docs/Backup-and-Restore.md
rename to docs/Basic_setup/Backup-and-Restore.md
index 3dae4ab1..48de688a 100644
--- a/docs/Backup-and-Restore.md
+++ b/docs/Basic_setup/Backup-and-Restore.md
@@ -8,6 +8,7 @@ The backup command can be executed from IOTstack's menu, or from a cronjob.
To ensure that all your data is saved correctly, the stack should be brought down. This is mainly due to databases potentially being in a state that could cause data loss.
There are 2 ways to run backups:
+
* From the menu: `Backup and Restore` > `Run backup`
* Running the following command: `bash ./scripts/backup.sh`
@@ -21,6 +22,7 @@ The current directory of bash must be in IOTstack's directory, to ensure that it
```
./scripts/backup.sh {TYPE=3} {USER=$(whoami)}
```
+
* Types:
* 1 = Backup with Date
* A tarball file will be created that contains the date and time the backup was started, in the filename.
@@ -33,10 +35,12 @@ The current directory of bash must be in IOTstack's directory, to ensure that it
If this parameter is not supplied when run as root, the script will ask for the username as input
Backups:
+
* You can find the backups in the ./backups/ folder. With rolling being in ./backups/rolling/ and date backups in ./backups/backup/
* Log files can also be found in the ./backups/logs/ directory.
### Examples:
+
* `./scripts/backup.sh`
* `./scripts/backup.sh 3`
@@ -52,6 +56,7 @@ This will only produce a backup in the rollowing folder and change all the permi
## Restore
There are 2 ways to run a restore:
+
* From the menu: `Backup and Restore` > `Restore from backup`
* Running the following command: `bash ./scripts/restore.sh`
@@ -64,6 +69,7 @@ There are 2 ways to run a restore:
./scripts/restore.sh {FILENAME=backup.tar.gz} {noask}
```
The restore script takes 2 arguments:
+
* Filename: The name of the backup file. The file must be present in the `./backups/` directory, or a subfolder in it. That means it should be moved from `./backups/backup` to `./backups/`, or that you need to specify the `backup` portion of the directory (see examples)
* NoAsk: If a second parameter is present, is acts as setting the no ask flag to true.
diff --git a/docs/Custom.md b/docs/Basic_setup/Custom.md
similarity index 97%
rename from docs/Custom.md
rename to docs/Basic_setup/Custom.md
index 8879f9ad..03a3508c 100644
--- a/docs/Custom.md
+++ b/docs/Basic_setup/Custom.md
@@ -125,7 +125,7 @@ services:
environment:
```
-This will remove the default environment variables set in the template, and tell docker-compose to use the variables specified in your file. It is not mandatory that the *.env file be placed in the service's service directory, but is strongly suggested. Keep in mind the [PostBuild Script](https://sensorsiot.github.io/IOTstack/PostBuild-Script) functionality to automatically copy your *.env files into their directories on successful build if you need to.
+This will remove the default environment variables set in the template, and tell docker-compose to use the variables specified in your file. It is not mandatory that the *.env file be placed in the service's service directory, but is strongly suggested. Keep in mind the [PostBuild Script](../Developers/PostBuild-Script.md) functionality to automatically copy your *.env files into their directories on successful build if you need to.
### Adding custom services
diff --git a/docs/Default-Configs.md b/docs/Basic_setup/Default-Configs.md
similarity index 98%
rename from docs/Default-Configs.md
rename to docs/Basic_setup/Default-Configs.md
index 6ef88715..51b05314 100644
--- a/docs/Default-Configs.md
+++ b/docs/Basic_setup/Default-Configs.md
@@ -1,4 +1,4 @@
-# Build Stack Default Passwords for Services
+# Default Passwords and ports
Here you can find a list of the default configurations for IOTstack for quick referece.
diff --git a/docs/Docker-commands.md b/docs/Basic_setup/Docker-commands.md
similarity index 100%
rename from docs/Docker-commands.md
rename to docs/Basic_setup/Docker-commands.md
diff --git a/docs/How-the-script-works.md b/docs/Basic_setup/How-the-script-works.md
similarity index 100%
rename from docs/How-the-script-works.md
rename to docs/Basic_setup/How-the-script-works.md
diff --git a/docs/Misc.md b/docs/Basic_setup/Misc.md
similarity index 100%
rename from docs/Misc.md
rename to docs/Basic_setup/Misc.md
diff --git a/docs/Native-RTL_433.md b/docs/Basic_setup/Native-RTL_433.md
similarity index 100%
rename from docs/Native-RTL_433.md
rename to docs/Basic_setup/Native-RTL_433.md
diff --git a/docs/Networking.md b/docs/Basic_setup/Networking.md
similarity index 100%
rename from docs/Networking.md
rename to docs/Basic_setup/Networking.md
diff --git a/docs/RPIEasy_native.md b/docs/Basic_setup/RPIEasy_native.md
similarity index 100%
rename from docs/RPIEasy_native.md
rename to docs/Basic_setup/RPIEasy_native.md
diff --git a/docs/Understanding-Containers.md b/docs/Basic_setup/Understanding-Containers.md
similarity index 100%
rename from docs/Understanding-Containers.md
rename to docs/Basic_setup/Understanding-Containers.md
diff --git a/docs/Basic_setup/What-is-sudo.md b/docs/Basic_setup/What-is-sudo.md
new file mode 100644
index 00000000..9f1a50bf
--- /dev/null
+++ b/docs/Basic_setup/What-is-sudo.md
@@ -0,0 +1,47 @@
+# What is sudo?
+
+Many first-time users of IOTstack get into difficulty by misusing the `sudo` command. The problem is best understood by example. In the following, you would expect `~` (tilde) to expand to `/home/pi`. It does:
+
+```bash
+$ echo ~/IOTstack
+/home/pi/IOTstack
+```
+
+The command below sends the same `echo` command to `bash` for execution. This is what happens when you type the name of a shell script. You get a new instance of `bash` to run the script:
+
+```bash
+$ bash -c 'echo ~/IOTstack'
+/home/pi/IOTstack
+```
+
+Same answer. Again, this is what you expect. But now try it with `sudo` on the front:
+
+```bash
+$ sudo bash -c 'echo ~/IOTstack'
+/root/IOTstack
+```
+
+Different answer. It is different because `sudo` means "become root, and then run the command". The process of becoming root changes the home directory, and that changes the definition of `~`.
+
+Any script designed for working with IOTstack assumes `~` (or the equivalent `$HOME` variable) expands to `/home/pi`. That assumption is invalidated if the script is run by `sudo`.
+
+Of necessity, any script designed for working with IOTstack will have to invoke `sudo` **inside** the script **when it is required**. You do not need to second-guess the script's designer.
+
+Please try to minimise your use of `sudo` when you are working with IOTstack. Here are some rules of thumb:
+
+1. Is what you are about to run a script? If yes, check whether the script already contains `sudo` commands. Using `menu.sh` as the example:
+
+ ```bash
+ $ grep -c 'sudo' ~/IOTstack/menu.sh
+ 28
+ ```
+
+ There are numerous uses of `sudo` within `menu.sh`. That means the designer thought about when `sudo` was needed.
+
+2. Did the command you **just executed** work without `sudo`? Note the emphasis on the past tense. If yes, then your work is done. If no, and the error suggests elevated privileges are necessary, then re-execute the last command like this:
+
+ ```bash
+ $ sudo !!
+ ```
+
+It takes time, patience and practice to learn when `sudo` is **actually** needed. Over-using `sudo` out of habit, or because you were following a bad example you found on the web, is a very good way to find that you have created so many problems for yourself that will need to reinstall your IOTstack. *Please* err on the side of caution!
diff --git a/docs/Getting-Started.md b/docs/Basic_setup/index.md
similarity index 78%
rename from docs/Getting-Started.md
rename to docs/Basic_setup/index.md
index 9c7223f8..75b589c0 100644
--- a/docs/Getting-Started.md
+++ b/docs/Basic_setup/index.md
@@ -1,6 +1,6 @@
# Getting Started
-## introduction to IOTstack - videos
+## introduction to IOTstack - videos
Andreas Spiess Video #295: Raspberry Pi Server based on Docker, with VPN, Dropbox backup, Influx, Grafana, etc: IOTstack
@@ -10,27 +10,25 @@ Andreas Spiess Video #352: Raspberry Pi4 Home Automation Server (incl. Docker, O
[![#352 Raspberry Pi4 Home Automation Server (incl. Docker, OpenHAB, HASSIO, NextCloud)](http://img.youtube.com/vi/KJRMjUzlHI8/0.jpg)](https://www.youtube.com/watch?v=KJRMjUzlHI8)
-## assumptions
+## Assumptions
IOTstack makes the following assumptions:
1. Your hardware is a Raspberry Pi (typically a 3B+ or 4B).
- Note:
-
* The Raspberry Pi Zero W2 has been tested with IOTstack. It works but the 512MB RAM means you should not try to run too many containers concurrently.
+ * Users have also [reported success
+ ](https://github.com/SensorsIot/IOTstack/issues/375) on Orange Pi
+ Win/Plus.
-2. Your Raspberry Pi has a reasonably-recent version of 32-bit Raspberry Pi OS (aka "Raspbian") installed. You can download operating-system images:
+2. Your Raspberry Pi has a reasonably-recent version of 32-bit or 64-bit Raspberry Pi OS (aka "Raspbian") installed. You can download operating-system images:
* [Current release](https://www.raspberrypi.com/software/operating-systems/)
+ : "Raspberry Pi OS with desktop" is recommended.
* [Prior releases](http://downloads.raspberrypi.org/raspios_armhf/images/)
+ : This offers only "Raspberry Pi OS with desktop" images.
- Note:
-
- * If you use the first link, "Raspberry Pi OS with desktop" is recommended.
- * The second link only offers "Raspberry Pi OS with desktop" images.
-
-3. Your operating system has been kept up-to-date with:
+3. Your operating system has been updated:
```bash
$ sudo apt update
@@ -46,15 +44,9 @@ If the first three assumptions hold, assumptions four through six are Raspberry
Please don't read these assumptions as saying that IOTstack will not run on other hardware, other operating systems, or as a different user. It is just that IOTstack gets most of its testing under these conditions. The further you get from these implicit assumptions, the more your mileage may vary.
-### other platforms
-
-Users have reported success on other platforms, including:
-
-* [Orange Pi WinPlus](https://github.com/SensorsIot/IOTstack/issues/375)
+## New installation
-## new installation
-
-### automatic (recommended)
+### automatic (recommended)
1. Install `curl`:
@@ -82,7 +74,7 @@ Users have reported success on other platforms, including:
$ docker-compose up -d
```
-### manual
+### manual
1. Install `git`:
@@ -122,21 +114,17 @@ Users have reported success on other platforms, including:
$ docker-compose up -d
```
-### scripted
+### scripted
If you prefer to automate your installations using scripts, see:
* [Installing Docker for IOTstack](https://gist.github.com/Paraphraser/d119ae81f9e60a94e1209986d8c9e42f#scripting-iotstack-installations).
-## migrating from the old repo (gcgarner)?
-
-If you are still running on gcgarner/IOTstack and need to migrate to SensorsIot/IOTstack, see:
+## Required system patches
-* [Migrating IOTstack from gcgarner to SensorsIot](./gcgarner-migration.md).
+Unless you know what you are doing, assume these are needed.
-## recommended system patches
-
-### patch 1 – restrict DHCP
+### patch 1 – restrict DHCP
Run the following commands:
@@ -147,7 +135,7 @@ $ sudo reboot
See [Issue 219](https://github.com/SensorsIot/IOTstack/issues/219) and [Issue 253](https://github.com/SensorsIot/IOTstack/issues/253) for more information.
-### patch 2 – update libseccomp2
+### patch 2 – update libseccomp2
This patch is **ONLY** for Raspbian Buster. Do **NOT** install this patch if you are running Raspbian Bullseye.
@@ -162,9 +150,9 @@ PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
If you see the word "buster", proceed to step 2. Otherwise, skip this patch.
-#### step 2: if you are running "buster" …
+#### step 2: if you are indeed running "buster"
-You need this patch if you are running Raspbian Buster. Without this patch, Docker images will fail if:
+Without this patch on Buster, Docker images will fail if:
* the image is based on Alpine and the image's maintainer updates to [Alpine 3.13](https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0#time64_requirement); and/or
* an image's maintainer updates to a library that depends on 64-bit values for *Unix epoch time* (the so-called Y2038 problem).
@@ -189,61 +177,14 @@ Enable by running (takes effect after reboot):
echo $(cat /boot/cmdline.txt) cgroup_memory=1 cgroup_enable=memory | sudo tee /boot/cmdline.txt
```
-## a word about the `sudo` command
-
-Many first-time users of IOTstack get into difficulty by misusing the `sudo` command. The problem is best understood by example. In the following, you would expect `~` (tilde) to expand to `/home/pi`. It does:
-
-```bash
-$ echo ~/IOTstack
-/home/pi/IOTstack
-```
-
-The command below sends the same `echo` command to `bash` for execution. This is what happens when you type the name of a shell script. You get a new instance of `bash` to run the script:
-
-```bash
-$ bash -c 'echo ~/IOTstack'
-/home/pi/IOTstack
-```
-
-Same answer. Again, this is what you expect. But now try it with `sudo` on the front:
-
-```bash
-$ sudo bash -c 'echo ~/IOTstack'
-/root/IOTstack
-```
-
-Different answer. It is different because `sudo` means "become root, and then run the command". The process of becoming root changes the home directory, and that changes the definition of `~`.
-
-Any script designed for working with IOTstack assumes `~` (or the equivalent `$HOME` variable) expands to `/home/pi`. That assumption is invalidated if the script is run by `sudo`.
-
-Of necessity, any script designed for working with IOTstack will have to invoke `sudo` **inside** the script **when it is required**. You do not need to second-guess the script's designer.
-
-Please try to minimise your use of `sudo` when you are working with IOTstack. Here are some rules of thumb:
-
-1. Is what you are about to run a script? If yes, check whether the script already contains `sudo` commands. Using `menu.sh` as the example:
-
- ```bash
- $ grep -c 'sudo' ~/IOTstack/menu.sh
- 28
- ```
-
- There are numerous uses of `sudo` within `menu.sh`. That means the designer thought about when `sudo` was needed.
-
-2. Did the command you **just executed** work without `sudo`? Note the emphasis on the past tense. If yes, then your work is done. If no, and the error suggests elevated privileges are necessary, then re-execute the last command like this:
-
- ```bash
- $ sudo !!
- ```
-
-It takes time, patience and practice to learn when `sudo` is **actually** needed. Over-using `sudo` out of habit, or because you were following a bad example you found on the web, is a very good way to find that you have created so many problems for yourself that will need to reinstall your IOTstack. *Please* err on the side of caution!
-## the IOTstack menu
+## the IOTstack menu
The menu is used to install Docker and then build the `docker-compose.yml` file which is necessary for starting the stack.
> The menu is only an aid. It is a good idea to learn the `docker` and `docker-compose` commands if you plan on using Docker in the long run.
-### menu item: Install Docker (old menu only)
+### menu item: Install Docker (old menu only)
Please do **not** try to install `docker` and `docker-compose` via `sudo apt install`. There's more to it than that. Docker needs to be installed by `menu.sh`. The menu will prompt you to install docker if it detects that docker is not already installed. You can manually install it from within the `Native Installs` menu:
@@ -260,7 +201,7 @@ Note:
* New menu (master branch) automates this step.
-### menu item: Build Stack
+### menu item: Build Stack
`docker-compose` uses a `docker-compose.yml` file to configure all your services. The `docker-compose.yml` file is created by the menu:
@@ -292,15 +233,15 @@ Some containers also need to be built locally. Node-RED is an example. Depending
Be patient (and ignore the huge number of warnings).
-### menu item: Docker commands
+### menu item: Docker commands
The commands in this menu execute shell scripts in the root of the project.
-### other menu items
+### other menu items
The old and new menus differ in the options they offer. You should come back and explore them once your stack is built and running.
-## switching menus
+## switching menus
At the time of writing, IOTstack supports three menus:
@@ -334,7 +275,7 @@ $ git checkout -- .templates/mosquitto/Dockerfile
When `git status` reports no more "modified" files, it is safe to switch your branch.
-### current menu (master branch)
+### current menu (master branch)
```bash
$ cd ~/IOTstack/
@@ -343,7 +284,7 @@ $ git checkout master
$ ./menu.sh
```
-### old menu (old-menu branch)
+### old menu (old-menu branch)
```bash
$ cd ~/IOTstack/
@@ -352,7 +293,7 @@ $ git checkout old-menu
$ ./menu.sh
```
-### experimental branch
+### experimental branch
Switch to the experimental branch to try the latest and greatest features.
@@ -377,14 +318,14 @@ Notes:
* The way back is to take down your stack, restore a backup, and bring up your stack again.
-## useful commands: docker & docker-compose
+## useful commands: docker & docker-compose
Handy rules:
* `docker` commands can be executed from anywhere, but
* `docker-compose` commands need to be executed from within `~/IOTstack`
-### starting your IOTstack
+### starting your IOTstack
To start the stack:
@@ -395,7 +336,7 @@ $ docker-compose up -d
Once the stack has been brought up, it will stay up until you take it down. This includes shutdowns and reboots of your Raspberry Pi. If you do not want the stack to start automatically after a reboot, you need to stop the stack before you issue the reboot command.
-#### logging journald errors
+#### logging journald errors
If you get docker logging error like:
@@ -423,9 +364,9 @@ Cannot create container for service [service name here]: unknown log opt 'max-fi
Logging limits were added to prevent Docker using up lots of RAM if log2ram is enabled, or SD cards being filled with log data and degraded from unnecessary IO. See [Docker Logging configurations](https://docs.docker.com/config/containers/logging/configure/)
-You can also turn logging off or set it to use another option for any service by using the IOTstack `docker-compose-override.yml` file mentioned at [IOTstack/Custom](https://sensorsiot.github.io/IOTstack/Custom/).
+You can also turn logging off or set it to use another option for any service by using the IOTstack `docker-compose-override.yml` file mentioned at [IOTstack/Custom](Custom.md).
-### starting an individual container
+### starting an individual container
To start a particular container:
@@ -434,7 +375,7 @@ $ cd ~/IOTstack
$ docker-compose up -d «container»
```
-### stopping your IOTstack
+### stopping your IOTstack
Stopping aka "downing" the stack stops and deletes all containers, and removes the internal network:
@@ -450,7 +391,7 @@ $ cd ~/IOTstack
$ docker-compose stop
```
-### stopping an individual container
+### stopping an individual container
`stop` can also be used to stop individual containers, like this:
@@ -480,7 +421,7 @@ $ cd ~/IOTstack
$ docker-compose up -d «container»
```
-### checking container status
+### checking container status
You can check the status of containers with:
@@ -495,7 +436,7 @@ $ cd ~/IOTstack
$ docker-compose ps
```
-### viewing container logs
+### viewing container logs
You can inspect the logs of most containers like this:
@@ -517,7 +458,7 @@ $ docker logs -f nodered
Terminate with a Control+C. Note that restarting a container will also terminate a followed log.
-### restarting a container
+### restarting a container
You can restart a container in several ways:
@@ -544,9 +485,9 @@ $ cd ~/IOTstack
$ docker-compose up -d --force-recreate «container»
```
-See also [updating images built from Dockerfiles](#updateDockerfile) if you need to force `docker-compose` to notice a change to a Dockerfile.
+See also [updating images built from Dockerfiles](#updating-images-not-built-from-dockerfiles) if you need to force `docker-compose` to notice a change to a Dockerfile.
-## persistent data
+## persistent data
Docker allows a container's designer to map folders inside a container to a folder on your disk (SD, SSD, HD). This is done with the "volumes" key in `docker-compose.yml`. Consider the following snippet for Node-RED:
@@ -588,7 +529,7 @@ is mirrored at the same relative path **inside** the container at:
/data
```
-### deleting persistent data
+### deleting persistent data
If you need a "clean slate" for a container, you can delete its volumes. Using InfluxDB as an example:
@@ -616,9 +557,9 @@ When InfluxDB starts, it sees that the folder on right-hand-side of the volumes
This is how **most** containers behave. There are exceptions so it's always a good idea to keep a backup.
-## stack maintenance
+## stack maintenance
-### update Raspberry Pi OS
+### update Raspberry Pi OS
You should keep your Raspberry Pi up-to-date. Despite the word "container" suggesting that *containers* are fully self-contained, they sometimes depend on operating system components ("WireGuard" is an example).
@@ -627,7 +568,7 @@ $ sudo apt update
$ sudo apt upgrade -y
```
-### git pull
+### git pull
Although the menu will generally do this for you, it does not hurt to keep your local copy of the IOTstack repository in sync with the master version on GitHub.
@@ -636,7 +577,7 @@ $ cd ~/IOTstack
$ git pull
```
-### container image updates
+### container image updates
There are two kinds of images used in IOTstack:
@@ -650,7 +591,7 @@ The easiest way to work out which type of image you are looking at is to inspect
* `image:` keyword then the image is **not** built using a Dockerfile.
* `build:` keyword then the image **is** built using a Dockerfile.
-#### updating images not built from Dockerfiles
+#### updating images not built from Dockerfiles
If new versions of this type of image become available on DockerHub, your local IOTstack copies can be updated by a `pull` command:
@@ -665,7 +606,7 @@ The `pull` downloads any new images. It does this without disrupting the running
The `up -d` notices any newly-downloaded images, builds new containers, and swaps old-for-new. There is barely any downtime for affected containers.
-#### updating images built from Dockerfiles
+#### updating images built from Dockerfiles
Containers built using Dockerfiles have a two-step process:
@@ -685,7 +626,7 @@ Note:
* You can also add nodes to Node-RED using Manage Palette.
-##### when Dockerfile changes (*local* image only)
+##### when Dockerfile changes (*local* image only)
When your Dockerfile changes, you need to rebuild like this:
@@ -697,7 +638,7 @@ $ docker system prune
This only rebuilds the *local* image and, even then, only if `docker-compose` senses a *material* change to the Dockerfile.
-If you are trying to force the inclusion of a later version of an add-on node, you need to treat it like a [DockerHub update](#rebuildDockerfile).
+If you are trying to force the inclusion of a later version of an add-on node, you need to treat it like a [DockerHub update](#updating-images-built-from-dockerfiles).
Key point:
@@ -712,7 +653,7 @@ Note:
$ docker-compose up --build -d nodered
```
-##### when DockerHub updates (*base* and *local* images)
+##### when DockerHub updates (*base* and *local* images)
When a newer version of the *base* image appears on DockerHub, you need to rebuild like this:
@@ -728,7 +669,7 @@ This causes DockerHub to be checked for the later version of the *base* image, d
Then, the Dockerfile is run to produce a new *local* image. The Dockerfile run happens even if a new *base* image was not downloaded in the previous step.
-### deleting unused images
+### deleting unused images
As your system evolves and new images come down from DockerHub, you may find that more disk space is being occupied than you expected. Try running:
@@ -762,9 +703,9 @@ $ docker rmi dbf28ba50432
In general, you can use the repository name to remove an image but the Image ID is sometimes needed. The most common situation where you are likely to need the Image ID is after an image has been updated on DockerHub and pulled down to your Raspberry Pi. You will find two containers with the same name. One will be tagged "latest" (the running version) while the other will be tagged "\" (the prior version). You use the Image ID to resolve the ambiguity.
-### pinning to specific versions
+### pinning to specific versions
-See [container image updates](#imageUpdates) to understand how to tell the difference between images that are used "as is" from DockerHub versus those that are built from local Dockerfiles.
+See [container image updates](#container-image-updates) to understand how to tell the difference between images that are used "as is" from DockerHub versus those that are built from local Dockerfiles.
Note:
@@ -820,7 +761,7 @@ To pin an image to a specific version:
$ docker-compose up -d --build mosquitto
```
-## the nuclear option - use with caution
+## the nuclear option - use with caution
If you create a mess and can't see how to recover, try proceeding like this:
@@ -840,7 +781,7 @@ In words:
4. Move your existing IOTstack directory out of the way. If you get a permissions problem:
* Re-try the command with `sudo`; and
- * Read [a word about the `sudo` command](#aboutSudo). Needing `sudo` in this situation is an example of over-using `sudo`.
+ * Read [a word about the `sudo` command](#a-word-about-the-sudo-command). Needing `sudo` in this situation is an example of over-using `sudo`.
5. Check out a clean copy of IOTstack.
diff --git a/docs/Containers/AdGuardHome.md b/docs/Containers/AdGuardHome.md
index 3404cac8..1fd59f19 100644
--- a/docs/Containers/AdGuardHome.md
+++ b/docs/Containers/AdGuardHome.md
@@ -9,7 +9,7 @@
AdGuard Home and PiHole perform similar functions. They use the same ports so you can **not** run both at the same time. You must choose one or the other.
-## Quick Start
+## Quick Start
When you first install AdGuard Home:
@@ -34,7 +34,7 @@ When you first install AdGuard Home:
Port 8089 is the default administrative user interface for AdGuard Home running under IOTstack.
-Port 8089 is not active until you have completed the [Quick Start](#quickStart) procedure. You must start by connecting to port 3001.
+Port 8089 is not active until you have completed the [Quick Start](#quick-start) procedure. You must start by connecting to port 3001.
Because of AdGuard Home limitations, you must take special precautions if you decide to change to a different port number:
@@ -50,11 +50,11 @@ Because of AdGuard Home limitations, you must take special precautions if you de
$ docker-compose up -d adguardhome
```
-3. Repeat the [Quick Start](#quickStart) procedure, this time substituting the new Admin Web Interface port where you see "8089".
+3. Repeat the [Quick Start](#quick-start) procedure, this time substituting the new Admin Web Interface port where you see "8089".
## About port 3001:3000
-Port 3001 (external, 3000 internal) is only used during [Quick Start](#quickStart) procedure. Once port 8089 becomes active, port 3001 ceases to be active.
+Port 3001 (external, 3000 internal) is only used during [Quick Start](#quick-start) procedure. Once port 8089 becomes active, port 3001 ceases to be active.
In other words, you need to keep port 3001 reserved even though it is only ever used to set up port 8089.
diff --git a/docs/Containers/Blynk_server.md b/docs/Containers/Blynk_server.md
index 6985a867..d5bf3260 100644
--- a/docs/Containers/Blynk_server.md
+++ b/docs/Containers/Blynk_server.md
@@ -2,7 +2,7 @@
This document discusses an IOTstack-specific version of Blynk-Server. It is built on top of an [Ubuntu](https://hub.docker.com/_/ubuntu) base image using a *Dockerfile*.
-## References
+## References
- [Ubuntu base image](https://hub.docker.com/_/ubuntu) at DockerHub
- [Peter Knight Blynk-Server fork](https://github.com/Peterkn2001/blynk-server) at GitHub (includes documentation)
@@ -18,7 +18,7 @@ Acknowledgement:
- Original writeup from @877dev
-## Significant directories and files
+## Significant directories and files
```
~/IOTstack
@@ -56,19 +56,19 @@ Everything in ❽:
* will be replaced if it is not present when the container starts; but
* will never be overwritten if altered by you.
-## How Blynk Server gets built for IOTstack
+## How Blynk Server gets built for IOTstack
-### GitHub Updates
+### GitHub Updates
Periodically, the source code is updated and a new version is released. You can check for the latest version at the [releases page](https://github.com/Peterkn2001/blynk-server/releases/).
-### IOTstack menu
+### IOTstack menu
When you select Blynk Server in the IOTstack menu, the *template service definition* is copied into the *Compose* file.
> Under old menu, it is also copied to the *working service definition* and then not really used.
-### IOTstack first run
+### IOTstack first run
On a first install of IOTstack, you run the menu, choose your containers, and are told to do this:
@@ -131,7 +131,7 @@ You *may* see the same pattern in *Portainer*, which reports the ***base image**
> Whether you see one or two rows depends on the version of `docker-compose` you are using and how your version of `docker-compose` builds local images.
-## Logging
+## Logging
You can inspect Blynk Server's log by:
@@ -139,7 +139,7 @@ You can inspect Blynk Server's log by:
$ docker logs blynk_server
```
-## Changing Blynk Server's configuration
+## Changing Blynk Server's configuration
The first time you launch the `blynk_server` container, the following structure will be created in the persistent storage area:
@@ -158,7 +158,7 @@ $ cd ~/IOTstack
$ docker-compose restart blynk_server
```
-## Getting a clean slate
+## Getting a clean slate
Erasing Blynk Server's persistent storage area triggers self-healing and restores known defaults:
@@ -178,7 +178,7 @@ Note:
$ docker-compose restart blynk_server
```
-## Upgrading Blynk Server
+## Upgrading Blynk Server
To find out when a new version has been released, you need to visit the [Blynk-Server releases](https://github.com/Peterkn2001/blynk-server/releases/) page at GitHub.
@@ -220,11 +220,11 @@ At the time of writing, version 0.41.16 was the most up-to-date. Suppose that ve
The second `prune` will only be needed if there is an old *base image* and that, in turn, depends on the version of `docker-compose` you are using and how your version of `docker-compose` builds local images.
-## Using Blynk Server
+## Using Blynk Server
See the [References](#references) for documentation links.
-### Connecting to the administrative UI
+### Connecting to the administrative UI
To connect to the administrative interface, navigate to:
@@ -237,7 +237,7 @@ You may encounter browser security warnings which you will have to acknowledge i
- username = `admin@blynk.cc`
- password = `admin`
-### Change username and password
+### Change username and password
1. Click on Users > "email address" and edit email, name and password.
2. Save changes.
@@ -248,19 +248,19 @@ You may encounter browser security warnings which you will have to acknowledge i
$ docker-compose restart blynk_server
```
-### Setup gmail
+### Setup gmail
Optional step, useful for getting the auth token emailed to you.
(To be added once confirmed working....)
-### iOS/Android app setup
+### iOS/Android app setup
1. When setting up the application on your mobile be sure to select "custom" setup [see](https://github.com/Peterkn2001/blynk-server#app-and-sketch-changes).
2. Press "New Project"
3. Give it a name, choose device "Raspberry Pi 3 B" so you have plenty of [virtual pins](http://help.blynk.cc/en/articles/512061-what-is-virtual-pins) available, and lastly select WiFi.
4. Create project and the [auth token](https://docs.blynk.cc/#getting-started-getting-started-with-the-blynk-app-4-auth-token) will be emailed to you (if emails configured). You can also find the token in app under the phone app settings, or in the admin web interface by clicking Users>"email address" and scroll down to token.
-### Quick usage guide for app
+### Quick usage guide for app
1. Press on the empty page, the widgets will appear from the right.
2. Select your widget, let's say a button.
@@ -273,7 +273,7 @@ Optional step, useful for getting the auth token emailed to you.
Enter Node-Red.....
-### Node-RED
+### Node-RED
1. Install `node-red-contrib-blynk-ws` from Manage Palette.
2. Drag a "write event" node into your flow, and connect to a debug node
diff --git a/docs/Containers/Chronograf.md b/docs/Containers/Chronograf.md
index c2fa8179..02d276fe 100644
--- a/docs/Containers/Chronograf.md
+++ b/docs/Containers/Chronograf.md
@@ -1,12 +1,12 @@
# Chronograf
-## References
+## References
- [*influxdata Chronograf* documentation](https://docs.influxdata.com/chronograf/)
- [*GitHub*: influxdata/influxdata-docker/chronograf](https://github.com/influxdata/influxdata-docker/tree/master/chronograf)
- [*DockerHub*: influxdata Chronograf](https://hub.docker.com/_/chronograf)
-## Kapacitor integration
+## Kapacitor integration
If you selected Kapacitor in the menu and want Chronograf to be able to interact with it, you need to edit `docker-compose.yml` to un-comment the lines which are commented-out in the following:
@@ -28,7 +28,7 @@ $ cd ~IOTstack
$ docker-compose up -d chronograf
```
-## Upgrading Chronograf
+## Upgrading Chronograf
You can update the container via:
@@ -45,7 +45,7 @@ In words:
* `docker-compose up -d` causes any newly-downloaded images to be instantiated as containers (replacing the old containers); and
* the `prune` gets rid of the outdated images.
-### Chronograf version pinning
+### Chronograf version pinning
If you need to pin to a particular version:
diff --git a/docs/Containers/deconz.md b/docs/Containers/Deconz.md
similarity index 100%
rename from docs/Containers/deconz.md
rename to docs/Containers/Deconz.md
diff --git a/docs/Containers/diyHue.md b/docs/Containers/DiyHue.md
similarity index 100%
rename from docs/Containers/diyHue.md
rename to docs/Containers/DiyHue.md
diff --git a/docs/Containers/Grafana.md b/docs/Containers/Grafana.md
index 824dd8e8..85e64bb6 100644
--- a/docs/Containers/Grafana.md
+++ b/docs/Containers/Grafana.md
@@ -15,6 +15,16 @@ The default *~/IOTstack/services/grafana/grafana.env* contains this line:
Uncomment that line and change the right hand side to [your own timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
+## Adding InfluxDB datasource
+
+Select Data Sources -> Add data source -> InfluxDB.
+
+Set options:
+
+* HTTP / URL: `http://influxdb:8086`
+* InfluxDB Details / Database: `telegraf`
+* InfluxDB Details / User: `nodered`
+* InfluxDB Details / Password: `nodered`
## Security
diff --git a/docs/Containers/Home-Assistant.md b/docs/Containers/Home-Assistant.md
index 3562ad42..0ea204c4 100644
--- a/docs/Containers/Home-Assistant.md
+++ b/docs/Containers/Home-Assistant.md
@@ -2,7 +2,7 @@
Home Assistant is a home automation platform. It is able to track and control all devices at your home and offer a platform for automating control.
-## References
+## References
- [Home Assistant home page](https://www.home-assistant.io/)
@@ -13,7 +13,7 @@ Home Assistant is a home automation platform. It is able to track and control al
- [DockerHub](https://hub.docker.com/r/homeassistant/home-assistant/)
-## Home Assistant: two versions
+## Home Assistant: two versions
There are two versions of Home Assistant:
@@ -31,7 +31,7 @@ Technically, both versions of Home Assistant can be installed on your Raspberry
IOTstack used to offer a menu entry leading to a convenience script that could install Supervised Home Assistant but that stopped working when Home Assistant changed their approach. Now, the only method supported by IOTstack is Home Assistant Container.
-### Installing Home Assistant Container
+### Installing Home Assistant Container
Home Assistant (Container) can be found in the `Build Stack` menu. Selecting it in this menu results in a service definition being added to:
@@ -56,11 +56,13 @@ $ cd ~/IOTstack
$ docker-compose up -d
```
-### Installing Supervised Home Assistant
+### Installing Supervised Home Assistant
The direction being taken by the Home Assistant folks is to supply a ready-to-run image for your Raspberry Pi. That effectively dedicates your Raspberry Pi to Home Assistant and precludes the possibility of running alongside IOTstack and containers like Mosquitto, InfluxDB, Node-RED, Grafana, PiHole and WireGuard.
-It is possible to run Supervised Home Assistant on the same Raspberry Pi as IOTstack. The recommended approach is to start from a clean slate and use [PiBuilder](https://github.com/Paraphraser/PiBuilder).
+Alternatively you can try to manually install Supervised Home Assistant using their [installation instructions for advanced users](https://github.com/home-assistant/supervised-installer) and when it works, install IOTstack. In theory this should work, but isn't tested or supported.
+
+The recommended approach is to start from a clean slate and use [PiBuilder](https://github.com/Paraphraser/PiBuilder).
When you visit the PiBuilder link you may well have a reaction like "all far too complicated" but you should try to get past that. PiBuilder has two main use-cases:
@@ -104,12 +106,12 @@ The first time you use PiBuilder, the process boils down to:
where «name» is the name you give to your Raspberry Pi (eg "iot-hub").
-After step 9, Supervised Home Assistant will be running. The `04_setup.sh` script also deals with the [random MACs](#aboutRandomMACs) problem. After step 11, you'll be able to either:
+After step 9, Supervised Home Assistant will be running. The `04_setup.sh` script also deals with the [random MACs](#why-random-macs-are-such-a-hassle) problem. After step 11, you'll be able to either:
1. Restore a backup; or
2. Run the IOTstack menu and choose your containers.
-## Why random MACs are such a hassle
+## Why random MACs are such a hassle
> This material was originally posted as part of [Issue 312](https://github.com/SensorsIot/IOTstack/issues/312). It was moved here following a suggestion by [lole-elol](https://github.com/lole-elol).
@@ -169,7 +171,7 @@ Random MACs are not a problem for a **client** device like a phone, tablet or la
It is not just configuration-time SSH sessions that break. If you decide to leave Raspberry Pi random Wifi MAC active **and** you have other clients (eq IoT devices) communicating with the Pi over WiFi, you will wrong-foot those clients each time the Raspberry Pi reboots. Data communications services from those clients will be impacted until those client devices time-out and catch up.
-# Using bluetooth from the container
+## Using bluetooth from the container
In order to be able to use BT & BLE devices from HA integrations, make sure that bluetooth is enabled and powered on at the start of the (Rpi) host by editing `/etc/bluetooth/main.conf`:
```conf
@@ -187,3 +189,116 @@ UP
...
```
ref: https://scribles.net/auto-power-on-bluetooth-adapter-on-boot-up/
+
+## HTTPS with a valid SSL certificate
+
+Some HA integrations (e.g google assistant) require your HA API to be
+accessible via https with a valid certificate. You can configure HA to do this:
+[docs](https://www.home-assistant.io/docs/configuration/remote/) /
+[guide](https://www.home-assistant.io/docs/ecosystem/certificates/lets_encrypt/)
+or use a reverse proxy container, as described below.
+
+The linuxserver Secure Web Access Gateway container
+([swag](https://docs.linuxserver.io/general/swag)) ([Docker hub
+docs](https://hub.docker.com/r/linuxserver/swag)) will automatically generate a
+SSL-certificate, update the SSL certificate before it expires and act as a
+reverse proxy.
+
+1. First test your HA is working correctly: `http://raspberrypi.local:8123/` (assuming
+your RPi hostname is raspberrypi)
+2. Make sure you have duckdns working.
+3. On your internet router, forward public port 443 to the RPi port 443
+4. Add swag to ~/IOTstack/docker-compose.yml beneath the `services:`-line:
+```
+ swag:
+ image: ghcr.io/linuxserver/swag
+ cap_add:
+ - NET_ADMIN
+ environment:
+ - PUID=1000
+ - PGID=1000
+ - TZ=Etc/UTC
+ - URL=.duckdns.org
+ - SUBDOMAINS=wildcard
+ - VALIDATION=duckdns
+ - DUCKDNSTOKEN=
+ - CERTPROVIDER=zerossl
+ - EMAIL= # required when using zerossl
+ volumes:
+ - ./volumes/swag/config:/config
+ ports:
+ - 443:443
+ restart: unless-stopped
+```
+ Replace the bracketed values. Do NOT use any "-characters to enclose the values.
+
+5. Start the swag container, this creates the file to be edited in the next step:
+ ```
+ cd ~/IOTstack && docker-compose up -d
+ ```
+
+ Check it starts up OK: `docker-compose logs -f swag`. It will take a minute or two before it finally logs "Server ready".
+
+6. Enable reverse proxy for `raspberrypi.local`. `homassistant.*` is already by default. and fix homeassistant container name ("upstream_app"):
+ ```
+ sed -e 's/server_name/server_name *.local/' \
+ volumes/swag/config/nginx/proxy-confs/homeassistant.subdomain.conf.sample \
+ > volumes/swag/config/nginx/proxy-confs/homeassistant.subdomain.conf
+ ```
+
+7. Forward to correct IP when target is a container running in "network_mode:
+ host" (like Home Assistant does):
+ ```
+ cat << 'EOF' | sudo tee volumes/swag/config/custom-cont-init.d/add-host.docker.internal.sh
+ #!/bin/sh
+ DOCKER_GW=$(ip route | awk 'NR==1 {print $3}')
+
+ sed -i -e "s/upstream_app .*/upstream_app ${DOCKER_GW};/" \
+ /config/nginx/proxy-confs/homeassistant.subdomain.conf
+ EOF
+ sudo chmod u+x volumes/swag/config/custom-cont-init.d/add-host.docker.internal.sh
+ ```
+ (This needs to be copy-pasted/entered as-is, ignore any "> "-prefixes printed
+ by bash)
+
+8. (optional) Add reverse proxy password protection if you don't want to rely
+ on the HA login for security, doesn't affect API-access:
+ ```
+ sed -i -e 's/#auth_basic/auth_basic/' \
+ volumes/swag/config/nginx/proxy-confs/homeassistant.subdomain.conf
+ docker-compose exec swag htpasswd -c /config/nginx/.htpasswd anyusername
+ ```
+9. Add `use_x_forwarded_for` and `trusted_proxies` to your homeassistant [http
+ config](https://www.home-assistant.io/integrations/http). The configuration
+ file is at `volumes/home_assistant/configuration.yaml` For a default install
+ the resulting http-section should be:
+ ```
+ http:
+ use_x_forwarded_for: true
+ trusted_proxies:
+ - 192.168.0.0/16
+ - 172.16.0.0/12
+ - 10.77.0.0/16
+ ```
+10. Refresh the stack: `cd ~/IOTstack && docker-compose stop && docker-compose
+ up -d` (again may take 1-3 minutes for swag to start if it recreates
+ certificates)
+11. Test homeassistant is still working correctly:
+ `http://raspberrypi.local:8123/` (assuming your RPi hostname is
+ raspberrypi)
+12. Test the reverse proxy https is working correctly:
+ `https://raspberrypi.local/` (browser will issue a warning about wrong
+ certificate domain, as the certificate is issued for you duckdns-domain, we
+ are just testing)
+
+ Or from the command line in the RPi:
+ ```
+ curl --resolve homeassistant..duckdns.org:443:127.0.0.1 \
+ https://homeassistant..duckdns.org/
+ ```
+ (output should end in `if (!window.latestJS) { }