Allow Probe Server's /connect to handle a certain number of reconnects before statusing

[bensheldon]

Fixes #1068.

Once GoodJob has successfully connected, Probe Server's /connected endpoint will continue returning 200-success if the database connection is lost and reconnections are attempted. This is now aligned with the existing thread-error reporting behavior:

good_job/lib/good_job/notifier.rb

Lines 152 to 160 in 3c45787

	if connection_error
	@connection_errors_count.increment
	if @connection_errors_reported.false? && @connection_errors_count.value >= CONNECTION_ERRORS_REPORTING_THRESHOLD
	GoodJob._on_thread_error(thread_error)
	@connection_errors_reported.make_true
	end
	else
	GoodJob._on_thread_error(thread_error)
	end

I also increased the number of reconnect attempts, so it will now be 6 attempts, once every 5 seconds = 30 seconds of reconnecting before the Notifier#connected? will become false and the Probe Server will start returning a 503 and the GoodJob.on_thread_error callback will be triggered.

Also some slight refactoring so that the utility methods listen? and wait? are now condition variables and have a timeout.

[jgrau]

FWIW I feel the design and code is good :)

[bensheldon]

@jgrau This has been released! https://github.com/bensheldon/good_job/releases/tag/v3.18.3