Traffic caused by Bull is higher than expected #2422
-
|
At the moment I am using Bull via the Nest.js integration hosted on a provisioned Redis instance on Railway. I was consindering to move to a different provider for better latency and was considering Upstash. (I am aware Bull is currently incompatible with Upstash!) Since Upstash charges dynamically per 100k commands I checked my current instance on Railway and to my surprise it had ~3m commands after a month of mostly idling queues indicating that 1-1.5 commands are executed per second. The readme however claims that Bull has "Minimal CPU usage due to a polling-free design." and I was wondering whether I misunderstood the term polling-free in this context because the sheer amount of polling done makes Upstash completely useless for personal projects since the costs would be at least $10-15 compared to Railway where the costs were barely $1. Can't imagine what this would look like under heavy load with thousands of delayed jobs (which is my primary use-case). My question is simply thus: is it normal that Bull causes this many commands to be executed with no active jobs at all? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
You can use "redis-cli monitor" to see what commands are being executed. There are some process that are run at different intervals, such as the stalled job checker, by default every 30 seconds, or if you have many idling workers, since the blocking command BRPOPLPUSH expires after 5 seconds, it may explain the executed commands. But if you check the CPU for you Redis instance it will be 0% if no queue is processing, since these are super low overhead. |
Beta Was this translation helpful? Give feedback.
-
|
Alright, I guess I'll migrate to Redis Cloud then since they don't charge by the amount of commands. Regarding the CPU I can confirm that which keeps my bill on Railway pretty low which is nice. Just wanted to have a confirmation this is normal behavior, which to be honest, sounds reasonable in hindsight as there is no other way than to check queues once per second... thanks for the clarification. |
Beta Was this translation helpful? Give feedback.
You can use "redis-cli monitor" to see what commands are being executed.
There are some process that are run at different intervals, such as the stalled job checker, by default every 30 seconds, or if you have many idling workers, since the blocking command BRPOPLPUSH expires after 5 seconds, it may explain the executed commands.
But if you check the CPU for you Redis instance it will be 0% if no queue is processing, since these are super low overhead.