fix[next]: Guard diskcache creation by file lock #1745
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tehrengruber
force-pushed
the
fix_caching_issue
branch
from
ab3f3ff to
92c1139
Compare
havogt
reviewed
Nov 25, 2024
| i.e. it ensures that any resources associated with the cache are properly | ||
| released when the instance is garbage collected. | ||
| This class extends `diskcache.Cache` to ensure the cache is properly | ||
| - opened on a distributed file system by using a file lock. This guards the creating of the |
There was a problem hiding this comment.
Is writing not a problem? Except that it's much harder to make all processes write at the same time and there less likely to trigger.
There was a problem hiding this comment.
No, writing to the cache is actually fine. SQLite which is used internally by diskcache.Cache handles concurrency very well in most cases, except for changing settings of the database itself, and that is the problem here. On creation of the diskcache.Cache object, some sql PRAGMA statements are executed and they lead to the observed issues. It is not really clear why, this should work, but it doesn't.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The disk cache used to cache compilation in the gtfn backend has a race condition manifesting itself in
sqlite3.OperationalError: database is lockederrors if multiple python processes try to initialize thediskcache.Cacheobject concurrently. This PR fixes this by guarding the object creation by a file-based lock in the same directory as the database.While this issue occurred frequently and was observed to be fixed on distributed file systems, the lock does not guarantee correct behavior in particular for accesses to the cache (beyond opening) since the underlying SQLite database is unreliable when stored on an NFS based file system. It does however ensure correctness of concurrent cache accesses on a local file system. See more information here:
https://grantjenks.com/docs/diskcache/tutorial.html#settings
https://www.sqlite.org/faq.html#q5
tox-dev/filelock#73
NFS safe locking:
https://gitlab.com/warsaw/flufl.lock
Barry Warsaw / FLUFL Lock · GitLab