zfs-replicate.readme

ZFS REPLICATION SCRIPT


Tested in:
	Ubuntu 12.04 + zfsonlinux	(all protocols)
	Nexentastor 3.1.3.5		(SSH,SSH+GZIP,NETCAT)


Support multiple simultaneous replication schemas

This script can be configured in cron to be run every "n" time.

It support multiple cron entries using diferent prefix snapshots
For example you can create 3 cron entries:

a) run every minute from 8:00 to 18:00. Options: -f min -t "60 minutes ago"
     We set "min" as prefix for the snapshots and we keep snapshots for 60 minutes. 
    
b) run every day at 20:00. Options: -f day -t "30 days ago"
     We set "day" as prefix for the snapshots and we keep snapshots for 30 days. 


c) run every day 1 of month at 21:00. Options: -f mth -t "3 month ago"
     We set "mth" as prefix for the snapshots and we keep snapshots for 3 months. 


Comunication protocols

It support diferent protocols:

SSH
SSH+GZIP
NETCAT
SOCAT
SOCAT CLIENT + NETCAT SERVER

In my test I usualy get 50MB/s using SSH and 250MB/s using NETCAT

For NETCAT you need to uninstall netcad-openbsd package and install netcat-traditional, because the openbsd version does not support listening timeout.


Filesystem check

After replication It can compare source and target using a dry-run rsync with checksum. This is for security because zfs for linux is still release candidate.


Common Issues:

For first time replication use parameter "-z"
Dataset should not exists in target for first time replication. If targert exists and there is no snapshot it will fail saying "There is no snapshots in target filesystem"


The Script parameters:

-h  target host 
    ip or hostname of target server

-p  target ssh port
    port number of target server 

-s  source zfs dataset
 
-d  target zfs dataset (default = source) 

-f  snapshot prefix
    is the prefix used to name the snapshots.
    it let setup multiple cron programs
    whith different time preservation

-t  max time to preserve snapshots (default infinite).
    Examples:
    -t "7 days ago"
    -t "12 hours ago"
    -t "10 minutes ago"


-v  verbose
    verbose zfs receive streams 

-c  clean snapshots in target that are not in source
 
-k  compare source and target with checksum using rsync
 
-n  no snapshot
 
-r  no replicate
 
-z  create target dataset if needed (for firt time replication)
 
-D  use deduplication in target fs and send stream

-o  replication protocol
    -o SSH
       for internet connections with good bandwith. >= 50 MB/s 
    -o SSHGZIP  compresed with gzip
       for medium/low bandwith conections < 50 MB/s
    -o NETCAT  it replicate using NETCAT on port 8023
       for lan networks >= 100 MB/s
       requires package netcat-traditional
       netcat-openbsd is not supported
    -o SOCAT same as NETCAT but use SOCAT package
    -o NETSOCAT  this is the recomended option for local lan
       it use netcat in server and socat in client
       it's the most stable and fault tolerant
       because netcat support listening timeout
       and socat support conection retries
 
-l  compression level 1..9 (default 6)
    for use with SSHGZIP protocol