Use Openat, Renameat syscalls

[adam-azarchs]

There are some situations where the atomicity of these operations isn't currently guaranteed. Specifically, there will be a problem if you want to atomically write /path/to/dir/file.txt but in between opening the temp file and renaming over file.txt, the directory gets renamed or deleted. This can be particularly problematic on NFS where, under certain circumstances (generally very heavy load), renaming the file can actually cause the directory to be recreated at its original location. You can similarly run into trouble if the directory containing the temporary file (which may or may not be the same as the destination directory) gets renamed.

At least in the case where the temporary file is created in the same directory as the final destination, this can be solved by doing

1. Open the destination directory with O_DIRECTORY|O_PATH
2. Use openat to open the temporary file using the file descriptor obtained in 1.
3. Write the file as normal
4. Use renameat to rename it, again using the file descriptor obtained in 1.

In the case where the temporary file is created in TEMPDIR rather than the destination directory, it's less clear what to do about the destination directory being renamed, however this technique can at least protect against the directory containing the temporary file getting renamed during the operation, which could otherwise in theory lead to renaming the wrong file.

Linux 3.11+ option

If you're on linux 3.11 or later (which would unfortunately require runtime feature detection¹), you can do even better:

1. Open the destination directory with O_DIRECTORY|O_PATH
2. Open the destination file using O_TMPFILE, which creates an unnamed regular file inode in the destination directory.
3. Once the file is ready to be exposed, use linkat to give the file a name.

If this is available, it has some significant advantages:

1. There's no need to manage cleanup state of the temporary file. It gets cleaned up automatically when the file handle closes, even if the program terminates unexpectedly (e.g. power-off, or OOM).
2. The file is not available on any filesystem path until it's complete.

Footnotes

1. upcoming versions of go will require kernel 3.17 or 3.10 with urandom support backported. The latter is still in widespread use, and would not support this. ↩

[stapelberg]

I’m open to using openat etc. once golang/go#67002 is implemented and available, but prior to that, it seems like a lot of effort to address a niche use-case.

In general, when I wrote the package, I had in mind the use-case of modifying files in an output directory that is not moved. I recommend you use the package similarly, at least for now.

BTW, using O_TMPFILE has most recently been discussed at #39 with the conclusion that linkat does not allow replacing a file. Was that not an accurate conclusion?

[adam-azarchs]

Perhaps my use cases are a bit unusual but I've encountered the directory being renamed (or deleted) out from under me quite often. It's especially nasty with NFS, where the local attribute cache means that a client might be unaware of that rename/delete. I feel like a package that aims to be a more robust alternative to managing this flow oneself should probably handle that too. If that functionality is being added to the standard library, it certainly seems reasonable to wait for that rather than reinventing that wheel, however.

I hadn't seen that closed issue. It does seem that indeed linkat can't be used to replace an existing file. Disappointing. That doesn't mean it's not still useful, but I'll refrain from discussing that further in this issue.