discard.active_job doesn't get triggered when jobs are discarded

[TAGraves]

I'm not sure if this is an issue in good_job, an issue in ActiveJob, or not an issue at all 😅.

It appears that the discard.active_job notification is only triggered when you explicitly define discard_on and not when failed jobs are automatically discarded. We noticed this because Datadog relies on this notification and we weren't getting any of our errors reported as a result.

[bensheldon]

I think the interaction between Active Job and the Backend when exceptions are raised through the backend is very undefined.

• "discard" is an explicit action to be taken in Active Job
• Active Job doesn't define what a Backend should do when a job errors
• Good Job reuses the term "discard" to cover:
  • using discard_on (event: discard.activejob)
  • using retry_on when attempts: are exceeded (event: retry_stopped.active_job)
  • implicitly when a job error and is not retried

Some thoughts about it:

• I don't think I want GoodJob emitting that event itself (though it does make me think it should be emitted when discarding jobs via the Dashboard UI, which is itself a hack because Active Job doesn't expose this behavior outside the context of job execution)
• I guess I could make a separate GoodJob event ("unhandled", "unretried", "abandoned"?) though Datadog wouldn't instrument it
• You could use retry_on StandardError, attempts: 1 which would trigger a retry_stopped.active_job on an exception which it does look like Datadog instruments