fix: delete references if schema is deleted #1027
Conversation
keejon
force-pushed
the
keejon/fix-delete-references
branch
2 times, most recently
from
73711ca to
ba65ad9
Compare
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
keejon
force-pushed
the
keejon/fix-delete-references
branch
2 times, most recently
from
61fc7c5 to
a8cbf98
Compare
src/karapace/in_memory_database.py
Outdated
| @@ -251,6 +251,10 @@ def insert_schema_version( | |||
| references=references, | |||
| ) | |||
|
|
|||
| if references: | |||
| for ref in references: | |||
| self.insert_referenced_by(subject=ref.subject, version=ref.version, schema_id=schema_id) | |||
There was a problem hiding this comment.
The insert_referenced_by locks self.schema_lock_thread, lock is re-entrant but maybe this function should be private and skip locking as this function does it already?
Similar locking happens with removal.
There was a problem hiding this comment.
I made insert and remove referenced_by private and removed it from API level because messaging should be blocking until schema_ready has read and handled message with the latest offset. (have left locking in though because it does not hurt and we can be safe that nobody misses it in the future) Strange thing is that this is not working as expected though and i get 422 in integration tests. 🤔
There was a problem hiding this comment.
@jjaakola-aiven tests are green again
keejon
force-pushed
the
keejon/fix-delete-references
branch
3 times, most recently
from
569aa1f to
a662bdb
Compare
to address recent mypy changes
keejon
force-pushed
the
keejon/fix-delete-references
branch
from
a662bdb to
485a95c
Compare
Moves handling of references (basically maintaining the reference index) to InMemoryDatabase to prevent issue where a schema is deleted but the references are still indexed in memory which can lead to issues e.g. not being able to delete schemas that are referenced.
Removal of references is still left on API level in addition as otherwise this will introduce some delay on the primary node until references are synced (which could lead to issues if someone wants to delete multiple schemas in consecutive calls)Also removed removal of references on API level as sending removal message is blocking and waiting for schema_reader to sync (See https://github.com/Aiven-Open/karapace/blob/keejon/fix-delete-references/src/karapace/messaging.py#L63)