Syncronization of large files fails on Linux systems with too-small /tmp

[alerighi]

On some Linux distributions (e.g. ArchLinux) the /tmp directory is mounted as a tmpfs in RAM by default. Since it's mounted in RAM, it has a limited capacity (on my system, that has 8Gb of RAM, the tmpfs is 4Gb in capacity).

I've noticed that kDrive was unable to sync some large (5+Gb) files that are on my drive. I've seen from the logfile that the download did indeed fail because the space on /tmp was finished. I've then modified the configuration of my system to mount /tmp on a real disk partition (the same that is done on Debian and other distributions) and the synchronization errors did indeed disappear.

/tmp shouldn't be used to store large files, not only because it has a limited capacity but also because storing data here wastes RAM on systems where the /tmp is a RAM filesystem.

In my opinion it would be better if kDrive would store temporary files inside the user home directory (e.g. in ~/.cache/kDrive). Alternatively you would need to document this behavior, and provide an option for the user to set the path of the temporary directory (or document it if there is already such a parameter).

Thanks!

[alerighi]

Also, I've noticed that these temporary files are not removed even after the file is completely downloaded. This causes a waste of RAM/disk space, till the next reboot (if /tmp is emptied at reboot or in a RAM tmpfs).

[ClementKunz]

Hi,
Thanks for report. Yes, that is something we noticed on Manjaro Linux as well.
This is in our backlog. We hope to be able to deliver a fix in in upcoming versions.

[eylenburg]

I noticed the same problem on Fedora and OpenSUSE. Not only does the sync stop working when /tmp is full, it also creates duplicates of files (conflicted copies).

A crude workaround when this happens is to pause the sync, run sudo rm /tmp/kdrive*, and unpause the sync.

[herve-er]

Hi,
After investigation, there is currently a bug where files are not removed from tmp when tmp is on a different file system than the synchronization. It is typically the case on Linux distributions that set the /tmp on RAM.

I have raised the priority of this issue in our roadmap for it to be fixed in the next release (Unfortunately not in 3.6.6 as it is already in its release process).

@alerighi, regarding the possibility of setting a location other than /tmp, we will discuss this and update this issue soon.

(@beankylla)

[beankylla]

Hi,
Just a note about this: what is meant with "the files are not on the same filesystem than the synchronization"?

I initially had a separate /tmp partition, but then removed it to solve the issue (it was then caching on the / filesystem. I have a separate /home partition though if that's what is meant here.

[herve-er]

Hi,
@beankylla
Your /tmp was probably a tmpfs filesystem, while /home is likely an ext4 filesystem. This difference is likely what caused the bug on our side. (A fix is on the track).

@alerighi, after discussion, we will investigate the possibility of using /cache instead of /tmp. However, for now, we won’t implement the option to manually choose a custom temp folder.
We haven’t yet decided on a target version for using /cache instead of /tmp.

[alerighi]

Hi @herve-er, thanks!

In my option on Linux the best would be to use the XDG Base Directory Specification, thus use the value of the environment variable XDG_CACHE_HOME (if defined), or else $HOME/.cache as specified by the standard.

[herve-er]

Hi @alerighi,
Thank you for the suggestion.

Hi @beankylla,
Here is the PR to resolve the issue of files remaining in the tmp directory. (This one does not include changes to the location of the tmp folder).

[SohKa]

Hi,
I've also experienced this issue with a tmpfs /tmp partition. Its size is 9.8 GiB, I wouldn't call that a small partition. It depends of course of the amount of data to synchronize.
I'm available if you need to perform some test.

[herve-er]

Hi @SohKa

There are currently two issues related to the /tmp partition on Linux.

For context, when the app downloads your files, they are first downloaded to /tmp before being copied to their final destination.

The first issue is that, in some cases, the app does not delete the temporary file after the copy, causing /tmp to eventually fill up with no space available for further downloads.

The second issue affects large files. If you have files on your drive that are larger than the space available in your /tmp partition, we are unable to download them, as they do not fit in the /tmp folder.

For the first issue, a PR is under review to fix it (#389).

For the second issue, we are currently considering a better location for storing temporary files, but we cannot provide a release version or date for this change yet (#204 (comment)).