Dumputils Container

🚀 blazingly-usefull! Please contribute if you have anything to add!

Table of Contents

• What is it
• What's inside
  • Tools
  • Aliases
  • Helper Scripts
• Usage
  • Run
  • Build
• Example Usage
  • Backup PostgreSQL and upload to S3
  • Use your existing credentials
  • Run inside Kubernetes
  • Add to docker-compose

What is it

This container simplifies the process of managing database backups by providing easy access to a range of backup and restore utilities. It includes additional aliases and helpful scripts to make it easier to use, and is designed to solve the problem of installing and managing multiple versions of tools and their dependencies on your local machine or server. It presents you with a structured shell environment where you can make your backup/restore procedures manually or automate with scripts while running inside Docker Compose or Kubernetes.

What's inside

Tools

• PostgreSQL Client Tools Version: 10, 11, 12, 13, 14 and 15

  • psql
  • pg_dump
  • pg_dumpall
  • pg_restore
  • pg_basebackup

• timescaledb-parallel-copy version v0.4.0

• percona-toolkit version 3.5.0-2

• Mongo tools version 100.5.2

  • mongodump
  • mongoexport
  • mongofiles
  • mongoimport
  • mongorestore
  • mongostat
  • mongotop

• MariaDB Client Tools version 10.8

  • mysql
  • mysql_find_rows
  • mysql_fix_extensions
  • mysql_waitpid
  • mysqlaccess
  • mysqladmin
  • mysqlanalyze
  • mysqlcheck
  • mysqldump
  • mysqlimport
  • mysqlrepair
  • mysqlreport
  • mysqlshow
  • mysqlslap

• kubectl latest stable version

• MinIO Client latest stable version

• AWS cli latest stable version

• Redis cli latest stable version

• kcat (kafkacat) Version 1.6.0

• bat v0.22.1 for Syntax highlighting

• net-utils, curl, wget

• zip, unzip, tar, etc...

• aliases and nice shell prompt

• shell utils (tmux, bat, htop)

• How To common tricks

Usage

Run

The prebuild images are pushed to Github (ghcr.io) and Dockerhub (docker.io). You can use whatever meets your expectations to run the image as follows:

The latest tag is stable and suitable for use. It reflects to the latest release

• Github Registry

docker run --rm --name dumputils \
-h dumputils -v /opt/backups:/opt/backups \
-it ghcr.io/st3ga/dumputils:latest

• Docker Registry

docker run --rm --name dumputils \
-h dumputils -v /opt/backups:/opt/backups \
-it st3ga/dumputils:latest

To run specific release version of your choice just use the desired release tag:

• Run version 1.3.0

docker run --rm --name dumputils \
-h dumputils -v /opt/backups:/opt/backups \
-it st3ga/dumputils:1.3.0

Build

If you are on restricted environment, you can build the image using the above commands. Please note that the main branch is considered unstable, so it is recommended to use the latest release tag for the most stable version.

git clone https://github.com/st3ga/dumputils-container.git
git checkout v1.3.0
docker build -f Dockerfile -t registry.yourdomain.com/dumputils:1.3.0 .

Examples

Backup PostgreSQL and upload to S3

Using the container to do manual backup of database called sales inside PostgreSQL Server Version 12 running in 192.168.1.5:5432. Upload to AWS s3 bucket called test after

user@machine: mkdir -p /opt/backups
user@machine: docker run --rm --name dumputils -h dumputils -v /opt/backups:/opt/backups -it st3ga/dumputils:latest

[root@dumputils] cd /opt/backups/ 
[root@dumputils] export PGPASSWORD="<password>" 
[root@dumputils] pg_dump12 -h 192.168.1.5 -p 5432 -U <user> -F t sales > sales.tar
[root@dumputils] export AWS_ACCESS_KEY_ID=<key-id> && export AWS_SECRET_ACCESS_KEY=<acces-key>
[root@dumputils] aws s3 cp sales.tar s3://test/backups/

You can find the tar file in /opt/backups inside your host machine after you exit the container.

Use your existing credentials

docker run --rm --name dumputils -h dumputils \
-v /opt/backups:/opt/backups \
-v $HOME/.config/kube:/root/.config/kube \
-v $HOME/.aws/:/root/.aws \
-v $HOME/.azure:/root/.azure \
-it st3ga/dumputils:latest

Run inside Kubernetes

• Deploy the stack

Increase the volume size of your data mounted in /opt/backups to match your needs. The default value is 5 Gigabytes which is suitable only for small backups and test purposes. You can entirely remove the Persistence Volume Claim if you plan to directly upload your backups somewhere right after creation.

kubectl apply -f examples/k8s/deployment/ 

• Validate

kubectl get all -n dumputils

NAME                             READY   STATUS    RESTARTS   AGE
pod/dumputils-xxxx               1/1     Running   0          1m

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/dumputils        1/1      1            1           1m

NAME                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/dumputils-xxxx   1         1         1       1m

kubectl get pvc -n dumputils

NAME                             STATUS   VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
dumputils-storage                Bound    pvc-xxx   5Gi        RWO            local-path     1m

• Use

From now on you can use your dumputils instance by starting a shell with the following commands:

kubectl exec -it \
$(kubectl get pods --template \
'{{range .items}}{{.metadata.name}}{{end}}' \
--selector=app=dumputils -n dumputils) \
-n dumputils -- /bin/bash

-----------------------------------------------------------------------------------------------

      ██████╗ ██╗   ██╗███╗   ███╗██████╗       ██╗   ██╗████████╗██╗██╗     ███████╗
      ██╔══██╗██║   ██║████╗ ████║██╔══██╗      ██║   ██║╚══██╔══╝██║██║     ██╔════╝
      ██║  ██║██║   ██║██╔████╔██║██████╔╝█████╗██║   ██║   ██║   ██║██║     ███████╗
      ██║  ██║██║   ██║██║╚██╔╝██║██╔═══╝ ╚════╝██║   ██║   ██║   ██║██║     ╚════██║
      ██████╔╝╚██████╔╝██║ ╚═╝ ██║██║           ╚██████╔╝   ██║   ██║███████╗███████║
      ╚═════╝  ╚═════╝ ╚═╝     ╚═╝╚═╝            ╚═════╝    ╚═╝   ╚═╝╚══════╝╚══════╝

┌─[root@dumputils]─[~]
└──╼ #

Add to docker-compose

To add dumputils to your existing docker-compose stack refer to this example.

# /opt/shared/backups will be mounted in dumputils so you can use 
# created backups outside of the container
mkdir -p /opt/shared/backups

cd examples/docker-compose/ 

docker-compose up -d 

docker exec -it dumputils /bin/bash
[root@dumputils] cd /opt/shared/backups/
[root@dumputils] pg_dump14 ....