Skip to content

Utility to download only missing files from remote server based on file list

License

Notifications You must be signed in to change notification settings

shoobyban/gwget

Folders and files

NameName
Last commit message
Last commit date

Latest commit

author
Sam Ban
Apr 25, 2019
60e05c4 · Apr 25, 2019

History

7 Commits
Mar 15, 2018
Mar 16, 2018
Mar 16, 2018
Aug 15, 2018
Apr 25, 2019
Apr 25, 2019
Apr 25, 2019

Repository files navigation

Wget based on file list

Documentation Go Report Card Build Status

The tool can (for example) mirror a media folder to a staging or developer server from the current live, over standard http requests, when scp is way too slow, creating tarball is not an option because of space requirements, VPN makes it hard to run tar | tar or rsync.

Usage

source$ cd ~/web/media
source$ find . -type f | grep -v '.thumb' | grep -v product/cache | sed 's#^./##' > ~/media.txt

scp media.txt onto the target server with a gwget binary

target$ cd ~/web/media
target$ ~/gwget ~/media.txt https://www.somesite.com/media/

About

Utility to download only missing files from remote server based on file list

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages