Running jobs that are not thread safe

[steveh]

Hi all, one of my jobs uses a library that I know is not thread safe.

To mitigate the problem I use good_job_control_concurrency_with(perform_limit: 1), but ideally I'd like each process to be able to run one of these jobs concurrently, whereas this concurrency control is more restrictive - only a single process can run one of these jobs concurrently. Is there a blessed way to accomplish that? Maybe adding the pid in the concurrency key?

[bensheldon]

Ooh, good question! Because this would be per process I think you could use a simple mutex. e.g.

class YourJob < ApplicationJob
  MUTEX = Mutex.new
  ResourceLockedError = Class.new(StandardError)

  retry_on ResourceLockedError, wait: :polynomially_longer, attempts: :unlimited

  around_perform do |_job, block|
    MUTEX.try_lock { block.call } || raise ResourceLockedError
  end
end

Raising an exception and retrying later is how GoodJob's Concurrency Controls work, though you could tune the wait: value to your particular usage.

[steveh]

Thanks!