Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding offset and size parameters #15

Open
Evengard opened this issue Apr 2, 2022 · 1 comment
Open

Adding offset and size parameters #15

Evengard opened this issue Apr 2, 2022 · 1 comment

Comments

@Evengard
Copy link

Evengard commented Apr 2, 2022

Hello!

First of all, big thanks for this tool. You saved my life with this.

Now, an enhancement request. Would it be possible to add --offset and --size options for the source file/drive? Possibly even a --target-offset as well!

Rationale: I have a 2 TB device I need to sync to a remote host. The internet connection is imperfect (and slow), so I need to restart it sometimes.
Now, on reconnect rechecking even 30% of a 2TB drive takes about 2 hours (because of the actual device speed). This is 2 hours of checking for something which is known to be OK - or which could be easily and quickly fixed by one last full file sweep at the end of the full file transfer.

Now, with --offset and --size parameters I could split the same transfers into chunks of, let's say, 100 GB each, syncing each of them, checking for integrity and moving on to the next. This would allow for faster checks on reconnections, and in the end I could just concatenate the resulting files back to the full image and be done with it.

With --target-offset I won't even need to concatenate the files, it'll just happen automatically.

I'm pretty positive that operating with offsets would be beneficial for other uses as well, basically allowing diskrsync to become something like dd over ssh.

Thanks again for the amazing tool!

@grinapo
Copy link

grinapo commented Sep 6, 2023

Without actually checking the code I'd guess that source and destination must be the same (length and content) since the hashing wouldn't work otherwise.

Same offset for both src and tgt could work, or, as you seem to suggest, --begin-pos and --end-pos of the transferred data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants