Rsync's behavior when uploading a directory

[cdkharris]

Rsync does not behave as described in this paragraph. I have set up an illustrating example for clarity.

These are my starting points on my local machine (a directory bar containing a file subbar) and the remote cluster.

Local:

$ ls -ld bar
drwxr-xr-x  3 ccaech0  staff  96 25 Jul 11:54 bar
$ ls -l bar 
total 0
-rw-r--r--  1 ccaech0  staff  0 25 Jul 11:54 subbar

Remote:

myriad $ ls -l bar subbar
ls: cannot access bar: No such file or directory
ls: cannot access subbar: No such file or directory

The example of L373 works as written.
Local:

$ rsync -avP bar myriad:~/
building file list ... 
2 files to consider
bar/
bar/subbar
           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/2)

sent 142 bytes  received 48 bytes  42.22 bytes/sec
total size is 0  speedup is 0.00

Remote:

myriad $ ls -l bar subbar
ls: cannot access subbar: No such file or directory
bar:
total 0
-rw-r--r-- 1 ccaech0 staff 0 Jul 25 11:54 subbar
myriad $ rm -rf bar subbar

Regarding the paragraph starting on L377, Rsync behaves identically if the trailing / on the target directory is omitted.

Local:

$ rsync -avP bar myriad:~ 
building file list ... 
2 files to consider
bar/
bar/subbar
           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/2)

sent 142 bytes  received 48 bytes  126.67 bytes/sec
total size is 0  speedup is 0.00

Remote:

$ ls -l bar subbar
ls: cannot access subbar: No such file or directory
bar:
total 0
-rw-r--r-- 1 ccaech0 staff 0 Jul 25 11:54 subbar
myriad $ rm -rf bar subbar

However, adding a trailing / on the source directory will result in the behaviour described.

Local:

$ rsync -avP bar/ myriad:~/
building file list ... 
2 files to consider
./
subbar
           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/2)

sent 138 bytes  received 48 bytes  74.40 bytes/sec
total size is 0  speedup is 0.00

Remote:

$ ls -l bar subbar
ls: cannot access bar: No such file or directory
-rw-r--r-- 1 ccaech0 staff 0 Jul 25 11:54 subbar

We have updated the text in the lesson to clarify this.

[reid-a]

So both of your recent PRs reference a recent issue that's on a different repo. Propagating fixes across repos is awesome, please don't misunderstand, but it also looks like maybe you've issued a PR against the wrong repo? Can you clarify your intentions?

I'm going to look into it anyways, of course, a fix is a fix!

[reid-a]

So having reviewed this, you're right that the text on our site is incorrect, for directory transfers, directory creation on the remote site occurs with or without the trailing slash on the destination, and this is true whether or not the target directory exists. The relevant trailing slash is on the source, as you correctly point out.

Interestingly, I also found this, which points out that a trailing slash on the destination can be important in the case where the source is a single file and the destination does not exist. rsync source path/to/dest creates a file at path/to/dest with the contents of source, but rsync source path/to/dest/ creates a directory path/to/dest with a file named source in it, at path/to/dest/source.

I like the second half of the PR. The comment about -r for scp is redundant with the immediately preceding discussion on L309 and L319.

I suggest you revise your PR to remove the changes around L344, but retain the others?

[reid-a]

Good to go, thanks!

[cdkharris]

So both of your recent PRs reference a recent issue that's on a different repo. Propagating fixes across repos is awesome, please don't misunderstand, but it also looks like maybe you've issued a PR against the wrong repo? Can you clarify your intentions?

I'm going to look into it anyways, of course, a fix is a fix!

Thanks! I'll explain what's happening:

At UCL we had not merged our site with the upstream carpentries-incubator/hpc-intro course for about 2 years, so the two have diverged. We are currently sorting through our local changes and grouping them into PRs. Most of these changes are updates to our site snippets, but there are some contributions to the lessons.

We made a plan for getting our site sufficiently up-to-date with the upstream so we can resume regular merges, which you've discovered in our issue that you've linked above. As part of that, I've been making PRs where it looked like our changes could be appropriate to contribute back. Happy to take a different approach :)

[reid-a]

IMO this is a fine approach. We have had people do PRs against our repos from forks by mistake, so I wanted to be careful. Will dig into the other one soon!

[cdkharris]

One of those mistake PRs was me ;)

Great, thank you!