Regexp freeze in user agent plugin for sdk-core consumes excessive CPU time?

[s-karki]

We run background job processors on a Fargate cluster, which pull jobs from an SQS queue (with Shoryuken). We've recently encountered an issue where the job processors' throughput starts to decrease when CPU usage reaches approximately 60%. Memory usage on the workers is normal, as is the time taken to fetch jobs off the queue.

We ran a profiler to analyze this degradation in performance, and saw that this code to freeze a regex is taking up dramatically more CPU time during the degradation (occasionally up to 10x more). I was hoping to see if anyone has faced a similar issue, and what the potential root cause might be.

I believe Regexes are immutable in Ruby – do we need freeze them? Or is there another fix for this?

We started noticing this degradation in early March.

[mullermp]

Thanks for opening up an issue. This sounds strange. When there isn't a degradation (under 60% cpu usage), does this code functional normally and quickly?

Can you try a monkey patch to see if it would resolve your issue?

My thinking would be that, even when fixed, you're still reaching bottlenecks somehow, and some other block of code would become the problem. But the root cause seems like maybe your cluster is overloaded? Have you looked at scaling up your instances?