Temporary files on receiver side vanished

[dpstrand]

I have seen some discussion on various forums about vanished files. However they seem to all refer to copying live filesystems where the source has removed a file.

I am seeing vanished on the receiving side, reporting the temporary file used for the transfer has vanished. The source file was not removed or modified. The file is successfully moved into place on the destination so this message seems spurious. This happens on a few of the files during a large copy.

file has vanished: "/media/.qra021900399ma.abs.WBQAUQ" (in disk)
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1816) [generator=3.2.3]

This does not occur with --inplace

[WayneD]

That message is only output for the sending side. I'd imagine that someone is copying files into the affected directory while another rsync is copying files out. When an error happens, the outgoing copy has found an incoming tempory file but discovered that it vanished (since it was moved into place).

It looks like you are using a daemon configuration, so you might consider adding a rule to hide dot files using filter/include/exclude rules in the daemon config (which can allow certain dot files through and exclude the rest).

[dpstrand]

Thank you for the reply. In this case the rsync daemon is running on another host (the source), and multiple rsync commands are initiated on the destination to copy all the remote files from that source to itself (the destination where this is being logged). Nothing is being simultaneously copied out of this destination location from any other processes. I am using the following sequence of commands run on the destination:

The destination file system is emptied or newly created before beginning, so there are no partial transfers or existing files prior to this.

To get a list of files to be copied:
rsync --no-h -rW8 src_hostname::module > files.tmp

This list is processed to produce 1 list of directories, and multiple lists of files of equal total file size

First I create all the directories:
rsync -aRlW8 --delete --stats --log-file=${LOG_DIR}/${DST}_copy.log --files-from=list.dir src_hostname::module /path/to/dest

Then multiple parallel copies of files:
for ((I=0; I<Num_Parallel; ++I)); do
rsync -asRlW8 --delete --stats --log-file=${LOG_DIR}/${DST}_copy.log --files-from=file_list.$I src_hostname::module /path/to/dest &
done

The vanished messages above are seen on the destination where the rsync commands are run which is why I had labelled it "the receiving side". I pictured it as the equivalent of an ftp pull, but perhaps that's not a proper analogy.

Does this make more sense? I believe the files being logged as vanished are the temporary files just prior to moving partial files to their final name. They are definitely not files that exist on the source file system.

For example: on the source host, the file from the original post is named /media/qra021900399ma.abs

Do you see any issue with parallelizing rsync in this way that would cause this?

[WayneD]

The client displays the message, but it's reporting the issue on the sending side. Something has to be creating files on the sending side for the rsync that is complaining to be able to find those in-coming (server) temp files. This may well be something that is rsyncing changes into the dirs that the module is serving up. Thus, the module-list command you mentioned finds some temp-file names in the module hierarchy.

One thing that could help is to copy files into the module in a more atomic manner. For instance, if you had 2 dir hierarchies and a symlink pointing at the live one, you could update files into the non-live dir and then do an atomic symlink update (create symlink under a new name and rename it into place) to swap the module's files from old to new. Or, if that's overkill for you, do all the incoming rsync using --delay-updates and --partial-dir=.rsync-tmp and add an exclude of .rsync-tmp into your rsyncd.conf file.