Schema changes for application-level aggregation metrics. #3315
Conversation
| task_id BIGINT NOT NULL, -- task ID the counter is associated with | ||
| ord BIGINT NOT NULL, -- the ordinal index of the task aggregation counter | ||
|
|
||
| success BIGINT NOT NULL DEFAULT 0, -- reports successfully aggregated |
There was a problem hiding this comment.
Do we not want to track preparation failures by reason the way we do upload failures? If nothing else, keeping track of either total failures or total prep attempts would let us compute success rate.
There was a problem hiding this comment.
See response to inahga's comment:
I'm implementing the least amount of functionality that gives us what we need; I'm OK leaning on operational metrics (e.g. prometheus/grafana) & logging for other issues, at least until we have a justification for splitting them out. (And yeah, I think it would be easy to add more to this later.)
Success rate, specifically, can be tracked via the operational metrics.
|
|
||
| CONSTRAINT task_aggregation_counters_unique_id UNIQUE(task_id, ord), | ||
| CONSTRAINT fk_task_id FOREIGN KEY(task_id) REFERENCES tasks(id) ON DELETE CASCADE | ||
| ) WITH (fillfactor = 50); |
There was a problem hiding this comment.
Given that uploading these counter tables can be performance sensitive, I'm surprised we don't create an index on task ID. Maybe it's irrelevant because this table will be smallish anyway?
There was a problem hiding this comment.
CONSTRAINT task_aggregation_counters_unique_id UNIQUE(task_id, ord) implicitly creates an index on (task_id, ord). We only ever INSERT on (task_id, ord) or SELECT task_id and aggregate over ord.
I have some query plans from task upload counters #2508 (comment).
There was a problem hiding this comment.
UNIQUE constraints generate a (unique) index on the relevant fields, so we have an index on (task_id, ord).
See https://www.postgresql.org/docs/current/ddl-constraints.html#DDL-CONSTRAINTS-UNIQUE-CONSTRAINTS:
Adding a unique constraint will automatically create a unique B-tree index on the column or group of columns listed in the constraint.
| task_id BIGINT NOT NULL, -- task ID the counter is associated with | ||
| ord BIGINT NOT NULL, -- the ordinal index of the task aggregation counter | ||
|
|
||
| success BIGINT NOT NULL DEFAULT 0, -- reports successfully aggregated |
There was a problem hiding this comment.
I see we're not going to track the other failures that could come up, like task upload counters do. Rationale? I'm nominally OK with this, since if we want this functionality it should be fairly trivial to bolt it on later.
There was a problem hiding this comment.
I'm implementing the least amount of functionality that gives us what we need; I'm OK leaning on operational metrics (e.g. prometheus/grafana) & logging for other issues, at least until we have a justification for splitting them out. (And yeah, I think it would be easy to add more to this later.)
No description provided.