Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metamorph records clearing #217

Merged
merged 17 commits into from
Jan 5, 2024
Merged

Add metamorph records clearing #217

merged 17 commits into from
Jan 5, 2024

Conversation

nozim
Copy link
Contributor

@nozim nozim commented Dec 17, 2023

  • add background job for clearing expired transactions in metamorph tables

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you copy these files from this path: metamorph/store/postgresql/migrations
then they should be exactly the same as they are the migration files which are actually used in the deployment. Right now they're not the same

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The migrations are still not exaclty the same. This schema is missing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does it have to have custom schema?

@nozim nozim requested a review from boecklim December 21, 2023 12:15
}

func ClearMetamorph(params ClearRecordsParams) error {
if err := runDelete("blocks", params); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After adding the schema the from clause would be metamorph.blocks like here

q := `INSERT INTO metamorph.blocks (

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it's a good idea to hardcode database name inside the query. Because if the name inside db connection can be different from the one inside the query.

if err := runDelete("blocks", params); err != nil {
return err
}
if err := runDelete("transactions", params); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same here

q := `UPDATE metamorph.transactions SET locked_by = 'NONE' WHERE hash = ANY($1);`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, why do we need to hardcode the database name?

Copy link

sonarqubecloud bot commented Jan 5, 2024

Quality Gate Failed Quality Gate failed

Failed conditions

63.6% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

@nozim nozim requested a review from boecklim January 5, 2024 13:18
@nozim nozim merged commit 08028e1 into main Jan 5, 2024
5 of 6 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 5, 2024
* Add metamorph records clearing

* Add metamorph database testing utility code

* Fix compilation errors

* Fix typo

* Amend blocktx tests

* Fix compilation errors

* Fix test failure

* Fix compilation error

* Fix formatting in blocktx db utility

* Add timestamp columnes for metamorph tables

* Rename migrations

* Fix test errors, amend migrations

* Amend schema name migrations

* Amend metamorph tests

---------

Co-authored-by: Nozim Mehrubonov <[email protected]>
@boecklim boecklim deleted the add-metamorph-clearing branch August 28, 2024 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants