Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autopilot refactor 4/4: Use domain::Settlement in OnSettlementEventUpdater #2861

Merged
merged 14 commits into from
Aug 20, 2024

Conversation

sunce86
Copy link
Contributor

@sunce86 sunce86 commented Aug 5, 2024

Dependent on #2875

Description

Actually uses domain::Settlement object in OnSettlementEventUpdater.

Changes

  • ...
  • ...

How to test

  • e2e tests
  • also manually tested values for surplus/score versus old code
  • mostly worried around handling of different error variants so will extensively monitor staging

@sunce86 sunce86 self-assigned this Aug 5, 2024
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

😋

Looks pretty good already. What's the plan to roll this out? Do you want to test the "debug print only" version (3/4) for one week in prod before merging this?

crates/autopilot/src/domain/settlement/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/domain/settlement/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/on_settlement_event_updater.rs Outdated Show resolved Hide resolved
crates/autopilot/src/on_settlement_event_updater.rs Outdated Show resolved Hide resolved
crates/autopilot/src/on_settlement_event_updater.rs Outdated Show resolved Hide resolved
return Err(anyhow!("{hash:?}, infra error {err}"))
}
Err(err) => {
tracing::warn!(?hash, ?err, "invalid settlement");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe add a todo here that this is where we would trigger the circuit breaker.

@sunce86
Copy link
Contributor Author

sunce86 commented Aug 7, 2024

What's the plan to roll this out?

Ideally, #2865 is merged and running it for a week in prod would give high confidence we can merge this PR.

sunce86 added a commit that referenced this pull request Aug 8, 2024
# Description
Related to comment
#2861 (comment)

Attaches `auction_id` to all errors that happen only once the
`auction_id` is successfully recovered from calldata. This enables
callers of the `Settlement::new` to fetch `auction_id` if it exists.

# Changes
<!-- List of detailed changes (how the change is accomplished) -->

- [ ] Added auction id to `SettlementError`
- [ ] Dependent error types refactored to fit nicely into `Settlement`
needs
sunce86 added a commit that referenced this pull request Aug 8, 2024
Dependent on #2866 

# Description
Instead of logging settlement object, it automatically compares old and
new implementation and prints if it detects a difference of behaviour.

Note that code quality is not of paramount importance here as the code
will be removed once testing and done and
#2861 lands.
sunce86 added a commit that referenced this pull request Aug 9, 2024
# Description
Extracts part of https://github.com/cowprotocol/services/pull/2861/files
that can be merged now, and don't have to be part of that PR.

This enables me to have a nice pub interface of `Settlement` and
continue adding functions for
#2862 and not wait for ☝️
PR to be merged.

Also #2861 becomes smaller
and easier to revert if we encounter some issue on weekly release.
@sunce86 sunce86 marked this pull request as ready for review August 14, 2024 15:36
@sunce86 sunce86 requested a review from a team as a code owner August 14, 2024 15:36
@sunce86 sunce86 changed the title DRAFT: Autopilot refactor 4/4: Use domain::Settlement in OnSettlementEventUpdater Autopilot refactor 4/4: Use domain::Settlement in OnSettlementEventUpdater Aug 14, 2024
@sunce86 sunce86 requested review from squadgazzz and fleupold August 14, 2024 15:37
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally the same issues as in the other PRs (similarly named structs without clear logical differentiation nor documentation, opportunity for more though through error handling instead of simply creating new variants), but otherwise looks good.

I'm really looking forward to making use of the new domain for impactful tasks such as listing CoW AMM orders and protocol fee amounts.

crates/autopilot/src/domain/settlement/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/domain/eth/mod.rs Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Show resolved Hide resolved
crates/autopilot/src/domain/settlement/mod.rs Outdated Show resolved Hide resolved
Comment on lines +413 to +414
auction_id: domain::auction::Id,
settlement: Option<&domain::settlement::Settlement>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit unclear from just reading the function signature under which circumstances it makes sense to have settlement be None.
I assume this happens in the case where we index a settlement event coming from the other environment so we are not able to find the necessary data in the DB?
In that case does it not make sense to split this function into 2?
One for handling just the event and one for storing the settlement information (in case there is some).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case does it not make sense to split this function into 2

Probably, but then we would have two commit statements, for which we would need to significantly change how the OnSettlementEventUpdater works. It would have to be split in two updates, one update for <auction, settlement> relation and other update that would have to check maybe settlement_observation table if it misses an entry for any of the settlement events etc. Anyway, I think this should be done but outside of scope of this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have the full context back in my head but is passing the transaction to the second function not an option to get around the 2-commit issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, but the point of infra is to abstract away specifics of underlying persistence layer and exposing transaction object to the caller is breaking this. I think we already discussed this somewhere whether we should allow it or not (few months ago).

Copy link
Contributor

@squadgazzz squadgazzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a nit.

crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@MartinquaXD MartinquaXD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly nits remaining

crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
crates/autopilot/src/infra/persistence/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking, but if we could keep the databases errors sqlx related and instead turn data inconsistencies into the already existing subtype, I think this PR would look even better.

@@ -96,7 +96,7 @@ impl Settlement {
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("failed communication with the database: {0}")]
Infra(sqlx::Error),
Infra(anyhow::Error),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did this change? Where are we turning a non sqlx error into a database error (since the to_string annotation still mentions databased communication failure)?

I think having an event index that doesn't fit into the right type could be considered InconsistentData (the existing error variant just below)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DatabaseError(sqlx::Error) was changed to DatabaseError(anyhow::Error) since I wanted to get rid of the error::Settlement and error::SettlementEvent and convert InvalidEvent into DatabaseError, and this was not possible for DatabaseError(sqlx::Error).

Converting DatabaseError(sqlx::Error) to DatabaseError(anyhow::Error) is probably fine since all db errors are treated the same at the callsite.

@sunce86
Copy link
Contributor Author

sunce86 commented Aug 20, 2024

Merging 🤞

@sunce86 sunce86 enabled auto-merge (squash) August 20, 2024 08:59
@sunce86 sunce86 merged commit d66f677 into main Aug 20, 2024
9 of 10 checks passed
@sunce86 sunce86 deleted the use-domain-settlement branch August 20, 2024 09:03
@github-actions github-actions bot locked and limited conversation to collaborators Aug 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants