Add more filters to the different job statuses

[rosa]

For example, filter scheduled jobs by scheduled_at time ranges, or finished jobs by finished_at time ranges, or blocked jobs by the block (Blocked by) key. The current filters by queue name and job class are generic and apply to all statuses, but we could have some filters that apply only to certain statuses. The Adapter#supported_filters already takes a JobsRelation as argument, so it can easily return different filters depending on jobs_relation.status.

[bruno-oliveira]

I'd like to tackle this one!

[Xeitor]

hi @bruno-oliveira, are u still working on this?

[Xeitor]

hi @rosa, is it okay if give this a try?

[rosa]

@Xeitor sure thing!

[nhorton]

One thing that would be awesome here would be if we could filter on a job argument. The basic scenario is that we do a lot of jobs on different accounts, and often we have questions of "what is still running for Account X". Right now we use sidekiq and do something kludgy to access this in the UI, and doing it properly with Mission Control would be really awesome.

[Xeitor]

@nhorton: sure, sounds useful, will take a look

[Xeitor]

hi @rosa, sorry for the delay, I've been getting myself familiar with the codebase

I was hoping you could help me out with a doubt, I am confused about what you said in the description about using Adapter#supported_filters (a method that seems to no longer exist) for returning filters based on the JobsRelation status, for what I can see, Adapters#supported_job_filters is used for returning filters suppoted natively by the adapter.

[Xeitor]

I just realized there already is some logic for raising errors if a filter is set for a non supported job status

example: raise ActiveJob::Errors::QueryError, "Filtering by worker id is not supported for status #{jobs_relation.status}" in lib/active_job/queue_adapters/solid_queue_ext.rb

[Xeitor]

I am guessing we could have one hand an error raised if a filter is used for the wrong status (like in the example above) and in the other a method like the one you mention (Adapter#supported_filters) for checking in the views if the filter should be used for the current status

[rosa]

Hey @Xeitor, sorry, my description was from over 2 months ago, and things have changed since then. Adapters#supported_job_filters takes a job_relation as argument and can use that to decide what filters are supported, just like this:

mission_control-jobs/lib/active_job/queue_adapters/resque_ext.rb

Lines 50 to 54 in 3a826cf

	def supported_job_filters(jobs_relation)
	if jobs_relation.pending? then [ :queue_name ]
	else []
	end
	end

It takes into account what filters the adapter supports but also the jobs_relation.status.

[Xeitor]

hi, no problem, from what I can tell, Adapters#supported_job_filters will return the natively supported filters for the adapter (taking into account the jobs_relation status), for example:

mission_control-jobs/lib/active_job/queue_adapters/resque_ext.rb

Lines 50 to 54 in a64a9d0

	def supported_job_filters(jobs_relation)
	if jobs_relation.pending? then [ :queue_name ]
	else []
	end
	end

in the resque adapter :queue_name filter can be performed natively if jobs_relation is pending and it will perform it in memory otherwise, :job_class_name will be performed in memory no matter the jobs_relation status

so I suppose we need another method that tells you if the filter is supported for the status, unless I am missing something

[rosa]

@Xeitor ah yes! Sorry, you're totally correct. Other filters could not be natively supported but are fine if performed in memory. I was getting confused with supported_job_statuses, which has nothing to do 😅

I'll continue the discussion in the PR, ultimately these are all implementation details that might not matter in the end.

[Xeitor]

@rosa no problem, thanks for your help :)

[MacLove13]

#209

I opened a PR related to this discussion, adding date filters using enqueued_at, scheduled_at and finished_at.

I didn't think it was necessary to limit any filter by Job type, considering that regardless of the status, the field is always there, and can only be null or not, so it could just be a filter that will make the query return nothing, such as trying to filter pending by a finished_at date.

But this won't break your code and you won't be prevented from doing it, you just won't see any results until you change your filter to something that makes more sense. (After all, of course a pending job shouldn't have a finished_at)

I left a deal to resolve Date Ranges as well and it seems to be fine.

The only point is: Applying filters to queues (or statuses) with /MANY/ jobs has an impact on performance, but I think we already know that at this point, right?