getJobRetryUntil on Import Jobs

[ahinkle]

We encountered an issue where a bad import continuously retried for approximately a day. This led to significantly increased costs due to services like AWS SQS and exception logging. Why is this the default behavior? In Laravel, the default retry count is 3, which would limit retries to 3 per record. Why isn't this the standard setting instead?

via the documentation:

By default, the import system will retry a job for 24 hours. This is to allow for temporary issues, such as the database being unavailable, to be resolved. That functionality is defined in the getJobRetryUntil() method on the importer class:

use Carbon\CarbonInterface;

public function getJobRetryUntil(): ?CarbonInterface
{
return now()->addDay();
}

If you'd like to customize the retry time for jobs of a certain importer, you may override this method in your importer class. You can read more about job retries in the Laravel docs.

https://filamentphp.com/docs/3.x/actions/prebuilt-actions/import#customizing-the-import-job-retries

This is to allow for temporary issues, such as the database being unavailable, to be resolved.

I understand this but I find this to be an unlikely scenario vs. a bug in the code such as incorrect invalidation.

Otherwise love this importer! Thanks!

[spcbfr]

@ahinkle Hey, just came across this, I am curious, what was the bug that made the import job retry indefinitely? I also am using the importer and I got the same problem

[ahinkle]

Hey there @spcbfr,

It's not necessarily a bug, but the default Filament configuration is set to retry indefinitely for 24 hours when an import fails.

In my case, I encountered a simple validation issue where a field was being sent as nullable when it was not permitted.

This was an easy fix, but I believe the retry limit should be set to use Laravel's default (which is 3 attempts) to prevent increased service costs and avoid unnecessary stress for developers.

[markdynamo6]

Where did you find the "default Filament configuration"? Can we also configure retries?

[ahinkle]

The getJobRetryUntil method; you would extend that.

[markdynamo6]

Cheers. Yes, we created a trait and used the following to only allow a single attempt:

public function getJobRetryUntil(): ?CarbonInterface
{
    return now();
}

[sh4msi]

How did you apply $tries? I did not find a method for this
It looks like I need to rewrite the ExportCsv job?

@ahinkle

[mikailhamzat]

Any update on this?

[markdynamo6]

We had a similar issue with exports whereby a failed one that was perpetually retrying flooded our logs with errors. We were also using a load balancer and multiple hosts which unfortunately compounded the issue. We had set retries=3 but this was ignored. It appears that Export Job doesn't appear to use atomic locks or error throttling, hence the multi host problem.

What's the recommended Filament approach to better control this?

[ahinkle]

Hi all, opened a PR to fix this here: #14856

Open to feedback 👍