Merge pull request #1 from Willem-Dekker/mkdocks

first mkdock test

.github/workflows/main.yml

@@ -0,0 +1,18 @@
+name: Publish docs via GitHub Pages
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  build:
+    name: Deploy docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout master
+        uses: actions/checkout@v1
+
+      - name: Deploy docs
+        uses: mhausenblas/mkdocs-deploy-gh-pages@master
+        env:
+          PERSONAL_TOKEN: ${{ secrets.PERSONAL_TOKEN }}

docs/Accessing-your-Device-from-the-internet.md

@@ -0,0 +1,43 @@
+The challenge most of us face with remotely accessing your home network is that you don't have a static IP. From time to time the IP that your ISP assigns to you changes and it's difficult to keep up. Fortunately, there is a solution, a DynamicDNS. The section below shows you how to set up an easy to remember address that follows your public IP no matter when it changes.
+
+Secondly, how do you get into your home network? Your router has a firewall that is designed to keep the rest of the internet out of your network to protect you. Here we install a VPN and configure the firewall to only allow very secure VPN traffic in. 
+
+# DuckDNS
+If you want to have a dynamic DNS point to your Public IP I added a helper script.
+Register with duckdns.org and create a subdomain name. Then edit the `nano ~/IOTstack/duck/duck.sh` file and add your
+
+```bash
+DOMAINS="YOUR_DOMAINS"
+DUCKDNS_TOKEN="YOUR_DUCKDNS_TOKEN"
+```
+
+first test the script to make sure it works `sudo ~/IOTstack/duck/duck.sh` then `cat /var/log/duck.log`. If you get KO then something has gone wrong and you should check out your settings in the script. If you get an OK then you can do the next step. 
+
+Create a cron job by running the following command `crontab -e`
+
+You will be asked to use an editor option 1 for nano should be fine
+paste the following in the editor `*/5 * * * * sudo ~/IOTstack/duck/duck.sh >/dev/null 2>&1` then ctrl+s and ctrl+x to save
+
+Your Public IP should be updated every five minutes
+
+# PiVPN
+pimylifeup.com has an excellent tutorial on how to install [PiVPN](https://pimylifeup.com/raspberry-pi-vpn-server/)
+
+In point 17 and 18 they mention using noip for their dynamic DNS. Here you can use the DuckDNS address if you created one.
+
+Don't forget you need to open the port 1194 on your firewall. Most people won't be able to VPN from inside their network so download OpenVPN client for your mobile phone and try to connect over mobile data. ([More info.](https://en.wikipedia.org/wiki/Hairpinning))
+
+Once you activate your VPN (from your phone/laptop/work computer) you will effectively be on your home network and you can access your devices as if you were on the wifi at home.
+
+I personally use the VPN any time I'm on public wifi, all your traffic is secure.
+
+# Zerotier
+https://www.zerotier.com/
+
+Zerotier is an alternative to PiVPN that doesn't require port forwarding on your router. It does however require registering for their free tier service [here](https://my.zerotier.com/login). 
+
+Kevin Zhang has written a how to guide [here](https://iamkelv.in/blog/2017/06/zerotier.html). Just note that the install link is outdated and should be:
+```
+curl -s 'https://raw.githubusercontent.com/zerotier/ZeroTierOne/master/doc/contact%40zerotier.com.gpg' | gpg --import && \
+if z=$(curl -s 'https://install.zerotier.com/' | gpg); then echo "$z" | sudo bash; fi
+```

docs/Adminer.md

@@ -0,0 +1,11 @@
+# References
+- [Docker](https://hub.docker.com/_/adminer)
+- [Website](https://www.adminer.org/)
+
+# About
+
+This is a nice tool for managing databases. Web interface has moved to port 9080. There was an issue where openHAB and Adminer were using the same ports. If you have an port conflict edit the docker-compose.yml and under the adminer service change the line to read:
+```
+    ports:
+      - 9080:8080
+```

docs/Backups.md

@@ -0,0 +1,112 @@
+Because containers can easily be rebuilt from docker hub we only have to back up the data in the "volumes" directory.
+
+# Cloud Backups
+## Dropbox-Uploader
+This a great utility to easily upload data from your Pi to the cloud. https://magpi.raspberrypi.org/articles/dropbox-raspberry-pi. It can be installed from the Menu under Backups.
+## rclone (Google Drive)
+This is a service to upload to Google Drive. The config is described [here]( https://medium.com/@artur.klauser/mounting-google-drive-on-raspberry-pi-f5002c7095c2). Install it from the menu then follow the link for these sections:
+* Getting a Google Drive Client ID
+* Setting up the Rclone Configuration
+
+When naming the service in `rclone config` ensure to call it "gdrive"
+
+**The Auto-mounting instructions for the drive in the link don't work on Rasbian**. Auto-mounting of the drive isn't necessary for the backup script.
+
+If you want your Google Drive to mount on every boot then follow the instructions at the bottom of the wiki page
+
+
+# Influxdb
+`~/IOTstack/scripts/backup_influxdb.sh` does a database snapshot and stores it in ~/IOTstack/backups/influxdb/db . This can be restored with the help a script (that I still need to write)
+
+# Docker backups
+The script `~/IOTstack/scripts/docker_backup.sh` performs the master backup for the stack. 
+
+This script can be placed in a cron job to backup on a schedule.
+Edit the crontab with`crontab -e`
+Then add `0 23 * * * ~/IOTstack/scripts/docker_backup.sh >/dev/null 2>&1` to have a backup every night at 23:00.
+
+This script cheats by copying the volume folder live. The correct way would be to stop the stack first then copy the volumes and restart. The cheating method shouldn't be a problem unless you have fast changing data like in influxdb. This is why the script makes a database export of influxdb and ignores its volume. 
+
+## Cloud integration
+The docker_backup.sh script now no longer requires modification to enable cloud backups. It now tests for the presence of and enable file in the backups folder
+### Drobox-Uploader
+The backup tests for a file called `~/IOTstack/backups/dropbox`, if it is present it will upload to dropbox. To disable dropbox upload delete the file. To enable run `sudo touch ~/IOTstack/backups/dropbox`
+### rclone
+The backup tests for a file called `~/IOTstack/backups/rclone`, if it is present it will upload to google drive. To disable rclone upload delete the file. To enable run `sudo touch ~/IOTstack/backups/rclone`
+
+### Pruning online backups
+@877dev has added functionality to prune both local and cloud backups. For dropbox make sure you dont have any files that contain spaces in your backup directory as the script cannot handle it at this time.
+
+## Restoring a backup
+The "volumes" directory contains all the persistent data necessary to recreate the container. The docker-compose.yml and the environment files are optional as they can be regenerated with the menu. Simply copy the volumes directory into the IOTstack directory, Rebuild the stack and start. 
+
+# Added your Dropbox token incorrectly or aborted the install at the token screen
+
+Make sure you are running the latest version of the project [link](https://github.com/gcgarner/IOTstack/wiki/Updating-the-Project).
+
+Run `~/Dropbox-Uploader/dropbox_uploader.sh unlink` and if you have added it key then it will prompt you to confirm its removal. If no key was found it will ask you for a new key.
+
+Confirm by running `~/Dropbox-Uploader/dropbox_uploader.sh` it should ask you for your key if you removed it or show you the following prompt if it has the key:
+
+```
+ $ ~/Dropbox-Uploader/dropbox_uploader.sh
+Dropbox Uploader v1.0
+Andrea Fabrizi - andrea.fabrizi@gmail.com
+
+Usage: /home/pi/Dropbox-Uploader/dropbox_uploader.sh [PARAMETERS] COMMAND...
+
+Commands:
+	 upload   <LOCAL_FILE/DIR ...>  <REMOTE_FILE/DIR>
+	 download <REMOTE_FILE/DIR> [LOCAL_FILE/DIR]
+	 delete   <REMOTE_FILE/DIR>
+	 move     <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
+	 copy     <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
+	 mkdir    <REMOTE_DIR>
+....
+
+```
+
+Ensure you **are not** running as sudo as this will store your api in the /root directory as `/root/.dropbox_uploader`
+
+If you ran the command with sudo the remove the old token file if it exists with either `sudo rm /root/.dropbox_uploader` or `sudo ~/Dropbox-Uploader/dropbox_uploader.sh unlink`
+
+# Auto-mount Gdrive with rclone
+
+To enable rclone to mount on boot you will need to make a user service. Run the following commands
+
+```bash
+mkdir -p ~/.config/systemd/user
+nano ~/.config/systemd/user/gdrive.service
+```
+Copy the following code into the editor, save and exit
+
+```
+[Unit]
+Description=rclone: Remote FUSE filesystem for cloud storage
+Documentation=man:rclone(1)
+
+[Service]
+Type=notify
+ExecStartPre=/bin/mkdir -p %h/mnt/gdrive
+ExecStart= \
+  /usr/bin/rclone mount \
+  --fast-list \
+  --vfs-cache-mode writes \
+  gdrive: %h/mnt/gdrive
+
+[Install]
+WantedBy=default.target
+```
+enable it to start on boot with: (no sudo)
+```bash
+systemctl --user enable gdrive.service
+```
+start with 
+```bash
+systemctl --user start gdrive.service
+```
+if you no longer want it to start on boot then type:
+```bash
+systemctl --user disable gdrive.service
+```
+

docs/Blynk_server.md

@@ -0,0 +1,81 @@
+This is a custom implementation of Blynk Server
+
+```yml
+  blynk_server:
+    build: ./services/blynk_server/.
+    container_name: blynk_server
+    restart: unless-stopped
+    ports:
+      - 8180:8080
+      - 8441:8441
+      - 9443:9443
+    volumes:
+      - ./volumes/blynk_server/data:/data
+```
+
+To connect to the admin interface navigate to `<your pis IP>:9443/admin`
+
+I don't know anything about this service so you will need to read though the setup on the [Project Homepage](https://github.com/blynkkk/blynk-server)
+
+When setting up the application on your mobile be sure to select custom setup [here](https://github.com/blynkkk/blynk-server#app-and-sketch-changes)
+
+Writeup From @877dev
+
+**Getting started:**
+Log into admin panel at https://youripaddress:9443/admin
+(Use your Pi's IP address, and ignore Chrome warning).
+
+Default credentials:
+user:admin@blynk.cc
+pass:admin
+
+**Change username and password:**
+Click on Users > "email address" and edit email, name and password. 
+Save changes
+Restarting the container using Portainer may be required to take effect.
+
+**Setup gmail:**
+Optional step, useful for getting the auth token emailed to you.
+(To be added once confirmed working....)
+
+**iOS/Android app setup:**
+Login the app as per the photos [HERE](https://github.com/blynkkk/blynk-server#app-and-sketch-changes)
+Press "New Project"
+Give it a name, choose device "Raspberry Pi 3 B" so you have plenty of [virtual pins](http://help.blynk.cc/en/articles/512061-what-is-virtual-pins) available, and lastly select WiFi.
+Create project and the [auth token](https://docs.blynk.cc/#getting-started-getting-started-with-the-blynk-app-4-auth-token) will be emailed to you (if emails configured). You can also find the token in app under the phone app settings, or in the admin web interface by clicking Users>"email address" and scroll down to token.
+
+**Quick usage guide for app:**
+Press on the empty page, the widgets will appear from the right.
+Select your widget, let's say a button.
+It appears on the page, press on it to configure.
+Give it a name and colour if you want. 
+Press on PIN, and select virtual. Choose any pin i.e. V0
+Press ok.
+To start the project running, press top right Play button.
+You will get an offline message, because no devices are connected to your project via the token.
+Enter node red.....
+
+**Node red:**
+Install node-red-contrib-blynk-ws from pallette manager
+Drag a "write event" node into your flow, and connect to a debug node
+Configure the Blynk node for the first time:
+```URL: wss://youripaddress:9443/websockets``` more info [HERE](https://github.com/gablau/node-red-contrib-blynk-ws/blob/master/README.md#how-to-use)
+Enter your [auth token](https://docs.blynk.cc/#getting-started-getting-started-with-the-blynk-app-4-auth-token) from before and save/exit.
+When you deploy the flow, notice the app shows connected message, as does the Blynk node.
+Press the button on the app, you will notice the payload is sent to the debug node.
+
+**What next?**
+Further information and advanced setup:
+https://github.com/blynkkk/blynk-server
+
+Check the documentation:
+https://docs.blynk.cc/
+
+Visit the community forum pages:
+https://community.blynk.cc/
+
+Interesting post by Peter Knight on MQTT/Node Red flows:
+https://community.blynk.cc/t/my-home-automation-projects-built-with-mqtt-and-node-red/29045
+
+Some Blynk flow examples:
+https://github.com/877dev/Node-Red-flow-examples

docs/Custom.md

@@ -0,0 +1,13 @@
+If you have a container that you want to stop and start with the stack you can now use the custom container option. This you can use for testing or in prep for a Pull Request.
+
+You will need to create a directory for your container call `IOTstack/services/<container>`
+
+Inside that container create a `service.yml` containing your service and configurations. Have a look at one of the other services for inspiration.
+
+Create a file called `IOTstack/services/custom.txt` and and enter your container names, one per line
+
+Run the menu.sh and build the stack. After you have made the selection you will be asked if you want to add the custom containers.
+
+Now your container will be part of the docker-compose.yml file and will respond to the docker-compose up -d commands.
+
+Docker creates volumes under root and your container may require different ownership on the volume directory. You can see an example of the workaround for this in the python template in `IOTstack/.templates/python/directoryfix.sh`

docs/Docker-commands.md

@@ -0,0 +1,14 @@
+# Aliases
+
+I've added bash aliases for stopping and starting the stack. They can be installed in the docker commands menu. These commands no longer need to be executed from the IOTstack directory and can be executed in any directory
+
+```bash
+alias iotstack_up="docker-compose -f ~/IOTstack/docker-compose.yml up -d"
+alias iotstack_down="docker-compose -f ~/IOTstack/docker-compose.yml down"
+alias iotstack_start="docker-compose -f ~/IOTstack/docker-compose.yml start"
+alias iotstack_stop="docker-compose -f ~/IOTstack/docker-compose.yml stop"
+alias iotstack_update="docker-compose -f ~/IOTstack/docker-compose.yml pull"
+alias iotstack_build="docker-compose -f ~/IOTstack/docker-compose.yml build"
+```
+
+You can now type `iotstack_up`, they even accept additional parameters `iotstack_stop portainer`

docs/EspruinoHub.md

@@ -0,0 +1,11 @@
+This is a testing container
+
+I tried it however the container keeps restarting `docker logs espruinohub` I get "BLE Broken?" but could just be i dont have any BLE devices nearby
+
+web interface is on "{your_Pis_IP}:1888"
+
+see https://github.com/espruino/EspruinoHub#status--websocket-mqtt--espruino-web-ide for other details
+
+there were no recommendations for persistent data volumes. so `docker-compose down` may destroy all you configurations so use `docker-compose stop` in stead 
+
+please let me know about your success or issues [here](https://github.com/gcgarner/IOTstack/issues/84)

docs/Getting-Started.md

@@ -0,0 +1,82 @@
+# Download the project
+
+On the lite image you will need to install git first 
+```
+sudo apt-get install git
+```
+Then download with
+```
+git clone https://github.com/gcgarner/IOTstack.git ~/IOTstack
+```
+Due to some script restraints, this project needs to be stored in ~/IOTstack
+
+To enter the directory run:
+```
+cd ~/IOTstack
+```
+# The Menu
+I've added a menu to make things easier. It is good however to familiarise yourself with how things are installed.
+The menu can be used to install docker and then build the docker-compose.yml file necessary for starting the stack and it runs a few common commands. I do recommend you start to learn the docker and docker-compose commands if you plan using docker in the long run. I've added several helper scripts, have a look inside.
+
+Navigate to the project folder and run `./menu.sh`
+
+## Installing from the menu
+Select the first option and follow the prompts
+
+## Build the docker-compose file
+docker-compose uses the `docker-compose.yml` file to configure all the services. Run through the menu to select the options you want to install.
+
+## Docker commands
+This menu executes shell scripts in the root of the project. It is not necessary to run them from the menu. Open up the shell script files to see what is inside and what they do
+
+## Miscellaneous commands
+Some helpful commands have been added like disabling swap
+
+# Running Docker commands
+From this point on make sure you are executing the commands from inside the project folder. Docker-compose commands need to be run from the folder where the docker-compose.yml is located. If you want to move the folder make sure you move the whole project folder.
+
+## Starting and Stopping containers
+to start the stack navigate to the project folder containing the docker-compose.yml file
+
+To start the stack run:
+`docker-compose up -d` or `./scripts/start.sh`
+
+To stop:
+`docker-compose stop` stops without removing containers
+
+To remove the stack: 
+`docker-compose down` stops containers, deletes them and removes the network
+
+The first time you run 'start' the stack docker will download all the images for the web. Depending on how many containers you selected and your internet speed this can take a long while.
+
+## Persistent data
+Docker allows you to map folders inside your containers to folders on the disk. This is done with the "volume" key. There are two types of volumes. Any modification to the container reflects in the volume.
+
+### Sharing files between the Pi and containers
+Have a look a the wiki on how to share files between Node-RED and the Pi. [Wiki](https://github.com/gcgarner/IOTstack/wiki/Node-RED#sharing-files-between-node-red-and-the-host)
+
+## Updating the images
+If a new version of a container image is available on docker hub it can be updated by a pull command.
+
+Use the `docker-compose stop` command to stop the stack
+
+Pull the latest version from docker hub with one of the following command
+
+`docker-compose pull` or the script `./scripts/update.sh`
+
+Start the new stack based on the updated images
+
+`docker-compose up -d`
+
+## Node-RED error after modifications to setup files
+The Node-RED image differs from the rest of the images in this project. It uses the "build" key. It uses a dockerfile for the setup to inject the nodes for pre-installation. If you get an error for Node-RED run `docker-compose build` then `docker-compose up -d`
+
+## Deleting containers, volumes and images
+
+`./prune-images.sh` will remove all images not associated with a container. If you run it while the stack is up it will ignore any in-use images. If you run this while you stack is down it will delete all images and you will have to redownload all images from scratch. This command can be helpful to reclaim disk space after updating your images, just make sure to run it while your stack is running as not to delete the images in use. (your data will still be safe in your volume mapping)
+
+## Deleting folder volumes
+If you want to delete the influxdb data folder run the following command `sudo rm -r volumes/influxdb/`. Only the data folder is deleted leaving the env file intact. review the docker-compose.yml file to see where the file volumes are stored.
+
+You can use git to delete all files and folders to return your folder to the freshly cloned state, AS IN YOU WILL LOSE ALL YOUR DATA.
+`sudo git clean -d -x -f` will return the working tree to its clean state. USE WITH CAUTION!