You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There should be an agent that periodically runs that clears data that is no longer used in the S3 bucket and in the main /var/lib folders. It should include:
Getting rid of files with no Mongo DB entry
Getting rid of engines with no Mongo DB entry in /var/lib/engines
Getting rid of S3 bucket experiments with no Mongo DB entry, or that have been completed successfully and finished.
Note that:
We need to ensure that the Mongo DB entry will be created before the files are created, otherwise, we could have race conditions and delete files before they are entered into the database. Same with pulling all files from the S3 bucket before calling the job "ready do delete" or some such thing.
This job should run periodically (once a day?) to clean up files. It needs to clear files even if there was an unexpected shutdown.
The text was updated successfully, but these errors were encountered:
There should be an agent that periodically runs that clears data that is no longer used in the S3 bucket and in the main /var/lib folders. It should include:
Note that:
The text was updated successfully, but these errors were encountered: