Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#48717] Replace DelayedJob with GoodJob. #12998

Merged
merged 25 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
fd6a899
[#48717] Replace DelayedJob with GoodJob.
ba1ash Jun 30, 2023
f82090d
Merge branch 'dev' into task/48717-replace-delayedjob
ba1ash Feb 14, 2024
6b8e1dc
Modify good_job_settings only if required.
ba1ash Feb 16, 2024
bddc471
Replace background check with good_job check
oliverguenther Feb 19, 2024
e1ce114
Extract health checks into separate files
oliverguenther Feb 19, 2024
8c86d12
Enable GoodJob dashboard only in development.
ba1ash Feb 20, 2024
e6dcb82
Resurrect removed migrations.
ba1ash Feb 20, 2024
e615525
Run ReorderHierarchyJob synchronously in migration
ba1ash Feb 20, 2024
f17ad24
Remove the reorder_hierarchy_job and inline it in the migration
oliverguenther Feb 20, 2024
8ab39f1
Fix testing value for cron
oliverguenther Feb 22, 2024
227029c
Merge branch 'dev' into task/48717-replace-delayedjob
ba1ash Feb 26, 2024
dab95b6
Use plain sql to cleanup removed cron_jobs.
ba1ash Feb 26, 2024
2267a0a
React on comments from review.
ba1ash Feb 27, 2024
e054adb
Merge remote-tracking branch 'origin/dev' into task/48717-replace-del…
ba1ash Feb 27, 2024
f2ffac9
Make execute_sql work.
ba1ash Feb 28, 2024
f296c54
Fix #upsert_status race condition.
ba1ash Feb 28, 2024
d32d332
Merge branch 'dev' into task/48717-replace-delayedjob
ba1ash Feb 28, 2024
8eb5674
Remove TODO comment in favor of dedicated ticket.
ba1ash Feb 28, 2024
a59e01e
Restore delayed_job related migration statements.
ba1ash Feb 29, 2024
0294e4b
Rename delayed_job_statuses to job_statuses.
ba1ash Feb 29, 2024
fa67e04
Use execute over execute_sql.
ba1ash Feb 29, 2024
ee88c89
Merge branch 'dev' into task/48717-replace-delayedjob
ba1ash Mar 4, 2024
c69d5eb
Migrate 'in queue' jobs to good_jobs table.
ba1ash Mar 5, 2024
dbab61a
Remove delayed_job dependecy. Lock jobs.
ba1ash Mar 5, 2024
ccb275b
Merge branch 'dev' into task/48717-replace-delayedjob
ba1ash Mar 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,7 @@ gem 'multi_json', '~> 1.15.0'
gem 'oj', '~> 3.16.0'

gem 'daemons'
gem 'delayed_cron_job', '~> 0.9.0'
gem 'delayed_job_active_record', '~> 4.1.5'
gem 'good_job'

gem 'rack-protection', '~> 3.2.0'

Expand Down
17 changes: 8 additions & 9 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -457,13 +457,6 @@ GEM
deckar01-task_list (2.3.3)
html-pipeline
declarative (0.0.20)
delayed_cron_job (0.9.0)
fugit (>= 1.5)
delayed_job (4.1.11)
activesupport (>= 3.0, < 8.0)
delayed_job_active_record (4.1.8)
activerecord (>= 3.0, < 8.0)
delayed_job (>= 3.0, < 5)
descendants_tracker (0.0.4)
thread_safe (~> 0.3, >= 0.3.1)
diff-lcs (1.5.1)
Expand Down Expand Up @@ -577,6 +570,13 @@ GEM
i18n (>= 0.7)
multi_json
request_store (>= 1.0)
good_job (3.21.5)
activejob (>= 6.0.0)
activerecord (>= 6.0.0)
concurrent-ruby (>= 1.0.2)
fugit (>= 1.1)
railties (>= 6.0.0)
thor (>= 0.14.1)
google-apis-core (0.13.0)
addressable (~> 2.5, >= 2.5.1)
googleauth (~> 1.9)
Expand Down Expand Up @@ -1159,8 +1159,6 @@ DEPENDENCIES
date_validator (~> 0.12.0)
debug
deckar01-task_list (~> 2.3.1)
delayed_cron_job (~> 0.9.0)
delayed_job_active_record (~> 4.1.5)
disposable (~> 0.6.2)
doorkeeper (~> 5.6.6)
dotenv-rails
Expand All @@ -1178,6 +1176,7 @@ DEPENDENCIES
friendly_id (~> 5.5.0)
fuubar (~> 2.5.0)
gon (~> 6.4.0)
good_job
google-apis-gmail_v1
googleauth
grape (~> 2.0.0)
Expand Down
2 changes: 1 addition & 1 deletion Procfile.dev
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
web: bundle exec rails server -p 3000 -b ${HOST:="127.0.0.1"} --environment ${RAILS_ENV:="development"}
angular: npm run serve
worker: bundle exec rake jobs:work
worker: bundle exec good_job start
5 changes: 4 additions & 1 deletion app/contracts/settings/working_days_params_contract.rb
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,11 @@ def working_days_are_present
end
end

# TODO: consider implementing using GoodJob concurrency control mechanisms
ba1ash marked this conversation as resolved.
Show resolved Hide resolved
def unique_job
if WorkPackages::ApplyWorkingDaysChangeJob.scheduled?
if GoodJob::Job
machisuji marked this conversation as resolved.
Show resolved Hide resolved
.where(finished_at: nil)
.exists?(job_class: 'WorkPackages::ApplyWorkingDaysChangeJob')
errors.add :base, :previous_working_day_changes_unprocessed
end
end
Expand Down
6 changes: 3 additions & 3 deletions app/seeders/root_seeder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -123,13 +123,13 @@ def prepare_seed!
ActionMailer::Base.perform_deliveries = false

# Avoid asynchronous DeliverWorkPackageCreatedJob
previous_delay_jobs = Delayed::Worker.delay_jobs
Delayed::Worker.delay_jobs = false
previous_execution_mode = Rails.configuration.good_job.execution_mode
Rails.configuration.good_job.execution_mode = :inline

yield
ensure
ActionMailer::Base.perform_deliveries = previous_perform_deliveries
Delayed::Worker.delay_jobs = previous_delay_jobs
Rails.configuration.good_job.execution_mode = previous_execution_mode
end

def seed_basic_data
Expand Down
5 changes: 5 additions & 0 deletions app/workers/application_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,11 @@ def reload_mailer_settings!
Setting.reload_mailer_settings!
end

def good_job_scheduled_at
ba1ash marked this conversation as resolved.
Show resolved Hide resolved
GoodJob::Job.where(id: job_id)
.pick(:scheduled_at)
end

private

def prepare_job_context
Expand Down
7 changes: 2 additions & 5 deletions app/workers/attachments/cleanup_uncontainered_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,9 @@
# See COPYRIGHT and LICENSE files for more details.
#++

class Attachments::CleanupUncontaineredJob < Cron::CronJob
class Attachments::CleanupUncontaineredJob < ApplicationJob
queue_with_priority :low

# runs at 10:03 pm
self.cron_expression = '03 22 * * *'

def perform
Attachment
.where(container: nil)
Expand All @@ -50,7 +47,7 @@ def too_old
attachment_table = Attachment.arel_table

attachment_table[:created_at]
.lteq(Time.now - OpenProject::Configuration.attachments_grace_period.minutes)
.lteq(Time.zone.now - OpenProject::Configuration.attachments_grace_period.minutes)
.to_sql
end
end
41 changes: 0 additions & 41 deletions app/workers/concerns/scheduled_job.rb

This file was deleted.

5 changes: 1 addition & 4 deletions app/workers/cron/clear_old_sessions_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,9 @@
#++

module Cron
class ClearOldSessionsJob < CronJob
class ClearOldSessionsJob < ApplicationJob
include ::RakeJob

# runs at 1:15 nightly
self.cron_expression = '15 1 * * *'

def perform
super('db:sessions:expire', 7)
end
Expand Down
5 changes: 1 addition & 4 deletions app/workers/cron/clear_tmp_cache_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,9 @@
#++

module Cron
class ClearTmpCacheJob < CronJob
class ClearTmpCacheJob < ApplicationJob
include ::RakeJob

# runs at 02:45 sundays
self.cron_expression = '45 2 * * 7'

def perform
super('tmp:cache:clear')
end
Expand Down
5 changes: 1 addition & 4 deletions app/workers/cron/clear_uploaded_files_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,9 @@
#++

module Cron
class ClearUploadedFilesJob < CronJob
class ClearUploadedFilesJob < ApplicationJob
include ::RakeJob

# Runs 23pm fridays
self.cron_expression = '0 23 * * 5'

def perform
super('attachments:clear')
end
Expand Down
5 changes: 1 addition & 4 deletions app/workers/ldap/synchronization_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,7 @@
#++

module Ldap
class SynchronizationJob < ::Cron::CronJob
# Run once per night at 11:30pm
self.cron_expression = '30 23 * * *'

class SynchronizationJob < ApplicationJob
def perform
run_user_sync
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,11 @@

module Notifications
# Creates date alert jobs for users whose local time is 1:00 am.
class ScheduleDateAlertsNotificationsJob < Cron::CronJob
# runs every quarter of an hour, so 00:00, 00:15,..., 15:30, 15:45, 16:00, ...
self.cron_expression = '*/15 * * * *'

class ScheduleDateAlertsNotificationsJob < ApplicationJob
def perform
return unless EnterpriseToken.allows_to?(:date_alerts)

service = Service.new(times_from_scheduled_to_execution)
service.call
Service.new(times_from_scheduled_to_execution).call
end

# Returns times from scheduled execution time to current time in 15 minutes
Expand All @@ -57,7 +53,7 @@ def times_from_scheduled_to_execution
end

def scheduled_time
self.class.delayed_job.run_at.then { |t| t.change(min: t.min / 15 * 15) }
good_job_scheduled_at.then { |t| t.change(min: t.min / 15 * 15) }
end
end
end
13 changes: 4 additions & 9 deletions app/workers/notifications/schedule_reminder_mails_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,13 @@
#++

module Notifications
class ScheduleReminderMailsJob < Cron::CronJob
# runs every quarter of an hour, so 00:00, 00:15...
self.cron_expression = '*/15 * * * *'

class ScheduleReminderMailsJob < ApplicationJob
def perform
User.having_reminder_mail_to_send(run_at).pluck(:id).each do |user_id|
User.having_reminder_mail_to_send(good_job_scheduled_at)
.pluck(:id)
.each do |user_id|
Mails::ReminderJob.perform_later(user_id)
end
end

def run_at
self.class.delayed_job.run_at
end
end
end
5 changes: 1 addition & 4 deletions app/workers/oauth/cleanup_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,9 @@
#++

module OAuth
class CleanupJob < ::Cron::CronJob
class CleanupJob < ApplicationJob
include ::RakeJob

# runs at 1:52 nightly
self.cron_expression = '52 1 * * *'

queue_with_priority :low

def perform
Expand Down
5 changes: 1 addition & 4 deletions app/workers/paper_trail_audits/cleanup_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,7 @@
#++

module PaperTrailAudits
class CleanupJob < ::Cron::CronJob
# runs at 4:03 on Saturday
self.cron_expression = '3 4 * * 6'

class CleanupJob < ApplicationJob
# Clean any paper trails older than 60 days
def perform
::PaperTrailAudit
Expand Down
73 changes: 0 additions & 73 deletions app/workers/projects/reorder_hierarchy_job.rb

This file was deleted.

1 change: 0 additions & 1 deletion app/workers/work_packages/apply_working_days_change_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@

class WorkPackages::ApplyWorkingDaysChangeJob < ApplicationJob
queue_with_priority :above_normal
include ::ScheduledJob

def perform(user_id:, previous_working_days:, previous_non_working_days:)
user = User.find(user_id)
Expand Down
2 changes: 1 addition & 1 deletion bin/check-worker-readiness
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash

if ps aux | grep 'rake jobs:work' | grep -v grep; then
if ps aux | grep 'good_job start' | grep -v grep; then
echo "background worker running"
exit 0
fi
Expand Down
5 changes: 0 additions & 5 deletions bin/delayed_job

This file was deleted.

Loading
Loading