Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cost-model): implement compute_operation_cost #44

Merged
merged 7 commits into from
Nov 21, 2024
Merged

Conversation

lanlou1554
Copy link
Contributor

  1. Implement compute_operation_cost
  2. Get and store cost / estimated_statistic in the cost model
  3. Enable ORM to store cost and estimated_statistic separately

@lanlou1554 lanlou1554 requested a review from xx01cyx November 20, 2024 22:50
@lanlou1554
Copy link
Contributor Author

lanlou1554 commented Nov 20, 2024

@xx01cyx It would be awesome if you can double check the important logic, like store_cost in the ORM layer. Thx :)

Comment on lines +205 to +214
if store_output_statistic {
let res = self
.storage_manager
.store_cost(context.expr_id, None, Some(output_statistic.clone()), None)
.await;
if res.is_err() {
eprintln!("Failed to store output statistic");
}
};
Ok(output_statistic)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might cause problem if some stats are missed from the storage. We should in the future consider having sth like a retry queue to store failed database requests. But I'm good with this for now. Maybe we can add a comment here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I think the retry should be handled by the storage manager, and I will add a comment there. I don't think it is a big deal, since maybe we can recalculate it again?

optd-cost-model/src/cost_model.rs Show resolved Hide resolved
optd-persistent/src/cost_model/orm.rs Outdated Show resolved Hide resolved
optd-persistent/src/cost_model/orm.rs Show resolved Hide resolved
@xx01cyx
Copy link
Member

xx01cyx commented Nov 21, 2024

Also, the logic for invalidating a cost seems to be missing here. I did not comment on the code because I'm not sure what is meant by "valid". The logic should be added after #44 (comment) is resolved.

@lanlou1554
Copy link
Contributor Author

Also, the logic for invalidating a cost seems to be missing here. I did not comment on the code because I'm not sure what is meant by "valid". The logic should be added after #44 (comment) is resolved.

I think the logic of invalidating a cost is handled by update_stats?

@xx01cyx
Copy link
Member

xx01cyx commented Nov 21, 2024

Also, the logic for invalidating a cost seems to be missing here. I did not comment on the code because I'm not sure what is meant by "valid". The logic should be added after #44 (comment) is resolved.

I think the logic of invalidating a cost is handled by update_stats?

Oh yeah you're righg

@lanlou1554 lanlou1554 merged commit dfaee9a into main Nov 21, 2024
11 checks passed
@lanlou1554 lanlou1554 deleted the cost-model-orm branch November 21, 2024 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants