-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failure to delete large project #1002
Comments
This happens also when deleting many projects. "504 Gateway Time-out I would suggest that the deletion should happen in two steps:
|
Note that things work fine from the command line. |
Working on this |
jayanth-kumar-morem
added a commit
to jayanth-kumar-morem/scancode.io
that referenced
this issue
Jan 26, 2024
1. Fix Default Filtering in ProjectFilterSet: - Modified the `ProjectFilterSet` to include `is_marked_for_deletion` in the default filtering when `is_archived` is not provided. - This ensures that projects marked for deletion are excluded from default views. 2. Migration for `is_marked_for_deletion`: - Added a migration (`0052_project_is_marked_for_deletion.py`) to introduce the `is_marked_for_deletion` field in the `Project` model. 3. Updated `Project` Model: - Added `is_marked_for_deletion` field to the `Project` model with a default value of `False`. 4. Modified `ProjectListView` to Exclude Marked Projects: - Modified the `get_queryset` method in `ProjectListView` to only fetch projects with `is_marked_for_deletion=False`. 5. Introduced `mark_for_deletion` Method: - Added a `mark_for_deletion` method in the `Project` model to set the `is_marked_for_deletion` flag and save the model. 6. Changed `delete` Method in `Project` Model: - Renamed the `delete` method to `delete_action` in the `Project` model to avoid conflicts. - Introduced a new `delete` method that marks the project for deletion and enqueues a background deletion task. 7. Background Deletion Task: - Created a background deletion task using `django_rq` to handle project deletion asynchronously. - Checks if the project is still marked for deletion before proceeding. - If an error occurs during deletion, updates project status and logs an error. 8. Updated `ProjectActionView` Success Message: - Modified the success message in `ProjectActionView` for the "delete" action to provide a more informative message based on the count. Fixes: aboutcode-org#1002 Signed-off-by: Jayanth Kumar <[email protected]>
jayanth-kumar-morem
added a commit
to jayanth-kumar-morem/scancode.io
that referenced
this issue
Jan 27, 2024
1. Fix Default Filtering in ProjectFilterSet: - Modified the `ProjectFilterSet` to include `is_marked_for_deletion` in the default filtering when `is_archived` is not provided. - This ensures that projects marked for deletion are excluded from default views. 2. Migration for `is_marked_for_deletion`: - Added a migration (`0052_project_is_marked_for_deletion.py`) to introduce the `is_marked_for_deletion` field in the `Project` model. 3. Updated `Project` Model: - Added `is_marked_for_deletion` field to the `Project` model with a default value of `False`. 4. Modified `ProjectListView` to Exclude Marked Projects: - Modified the `get_queryset` method in `ProjectListView` to only fetch projects with `is_marked_for_deletion=False`. 5. Introduced `mark_for_deletion` Method: - Added a `mark_for_deletion` method in the `Project` model to set the `is_marked_for_deletion` flag and save the model. 6. Add `delete_in_background` Method in `Project` Model: - Introduced a new `delete_in_background` method that marks the project for deletion and enqueues a background deletion task. 7. Background Deletion Task: - Created a background deletion task using `django_rq` to handle project deletion asynchronously. - Checks if the project is still marked for deletion before proceeding. - If an error occurs during deletion, updates project status and logs an error. 8. Updated `ProjectActionView` Success Message: - Modified the success message in `ProjectActionView` for the "delete" action to provide a more informative message based on the count. Fixes: aboutcode-org#1002 Signed-off-by: Jayanth Kumar <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have a project with about 1M files and 2GB compressed inputs (~10G uncompressed)
Deleting failed now and then with a "Gateway timeout" error
The text was updated successfully, but these errors were encountered: