You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Counters are used to record the task status, they will be periodically logged in monitor.log (configuration: [pipeline] checkpoint_interval_secs).
Time window counters
This type of counter is an array of sub-counters. During task execution, whenever there is a state change (e.g., successfully writing a batch entries to target), a new sub-counter is generated to record the increment info (e.g., number of entries written to target).
The counter has a time window (configuration: [pipeline] counter_time_window_secs), expired sub-counters will be discarded.
The counter is used for real-time monitoring, such as the number of synchronized entries in time window.
The counter has aggregation algorithms, such as the average count of synchronized entries per second.
Aggregation algorithms
Aggregation
Description
Example
sum
sum of sub-counters
count of synchronized entries in last 10 seconds
avg
sum of sub-counters / number of sub-counters
average time cost for each write to target in last 10 seconds
avg_by_sec
sum of all sub-counters / time window
average number of entries written to target per second in last 10 seconds
max
the sub-counter with the maximum value
maximum number of entries written to target in a single batch in last 10 seconds
max_by_sec
sums the sub-counters for each second, and finds the second with the maximum sum
Maximum number of entries written to target in a single second
No window counter
A simple counter to record accumulated data, such as the number of migrated MySQL records.