[Bug] yarsync push overwrite uncommitted modified files on the remote

[r888800009]

Hello, I am looking for a synchronization tool similar to git but that can handle FS metadata (timestamp), and found your project that seems to be a good match. Now I am trying to evaluate whether it is suitable for my use case.

But when push, that some data that has not yet been committed will be overwritten.

Version

yarsync version 0.2.1

Operating System

macos ventura and archlinux

What happened

We have two computers A and B, create a repo, create a file1 and commit.
And make sure that A and B each have a copy of the repo.

Then modify file1 on B then push from A, we can see that the file1 has been overwritten.

How to reproduce

init repo

# on A
mkdir test1
cd test1
yarsync init
yarsync commit -m "Initial commit"
yarsync remote add my_remote host:/tmp/test1-repo

# on B
mkdir /tmp/test1-repo
cd /tmp/test1-repo
yarsync init

then push repo from A to B, and make file1 push again

# on A
yarsync push my_remote

echo testfile > file1
yarsync commit
yarsync push my_remote

Modify file1 on B

$ echo 'should not be overwritten' > file1
$ cat file1
should not be overwritten

Then push from A again, We can see that the file is overwritten

# on A
yarsync push my_remote

# on B
$ cat file1
testfile

What you think should happen instead

yarsync should detect and avoid overwriting data, shows that the remote has not yet committed.

If this is not the intended use, it must be aborted to avoid data loss.

[ynikitenko]

Hey @r888800009 , thanks for your pull request and bug report.

yarsync tests that there are no uncommitted changes in the local repository.
I can't see any way of testing that for the remote repository.
The way I do it is to use yarsync -n push <remote>. -n is dry run and it will show you the transferred data (without making changes).

Otherwise seeing "uncommitted changes" on the remote would mean to compare (push data) between two remote folders. Do you have an idea on how it could be implemented?.. With rsync it is not possible, unfortunately.

[r888800009]

Thanks for your reply, I have some ideas to discuss.

Run local test on the remote server

The first idea, if we expect remote to also install yarsync, and your previous consideration is traffic overhead

There is a hacky way to use --rsync-path to run local test on the remote server, but it is only for ssh connections and does not support rsync://host (rsyncd)

For example, we can run sleep on the remote server with --rsync-path

rsync --rsync-path 'sleep 10 && rsync' remote_host:/

One possible solution is to use the rsync error message to detect whether the local test was successful.

rsync  --rsync-path 'sleep 1 && ls / && rsync' remote_host:/temp -vvvv

We can compare the difference between these two outputs

In this case, it would be better to use the ssh command to avoid hacky code. The disadvantage is assumed the remote repo user uses rsync through ssh.

Dry run as default behavior

Another idea is to run dry-run checking by default when remote push, and the user can set the configuration like config.ini to turn off checking.

This may be useful for new users to avoid accidental overwriting.

If we always run --dry-run first, why not just let it run automatically?

Bare Repository

There is also something like git bare repository, which is divided into multiple repo, one is bare for sync between computers, but I have not tried it with yarsync. If it is a possible solution, maybe users can refer to it.

[ynikitenko]

Thanks for the ideas. They are all rather nice.

Run local test on the remote server

Yes, this should work. Since rsync (and servers) are mostly used by (use) Linux, ssh should also be present in most cases.

ssh <my_server> 'cd <my_repo_path>; yarsync status'

shall print the status.
The downside: (as in git), it doesn't give an error code if something was changed.
I'm not sure one should rely solely on the text output (for a human it's easy to make a mistake).

Dry run as default behavior

Yes, this might be useful.
I would add an option -i, so that it first runs -n, then asks whether the user wants to continue.
Then again, the problem of the reliance upon the user stays.

Bare Repository

There is a kind of a "bare" repository (with detached working and configuration directories), but it is different.
However, when you don't work on a server, you simply don't change the working directory. So you either need a non-bare repository there, or you can go with a simple repo (just don't change it directly).

There are indeed several options to deal with this problem, that's why I'm not sure yet which one to apply. In general yes, it would be good (namely safe) to test for uncommitted changes.

For me, I try to always commit my changes. Also, the files in yarsync repositories are usually not crucial for me (and if they are, I carefully look at the results of dry run - and I synchronise important repositories rarely; I don't update them often).

What is your situation (working routine)? What kind of files do you use? Do you really use a server and add files there? Maybe a better solution exists.