Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run missed task from persistent storage #328

Open
Cheesy008 opened this issue May 22, 2024 · 4 comments
Open

Run missed task from persistent storage #328

Cheesy008 opened this issue May 22, 2024 · 4 comments

Comments

@Cheesy008
Copy link

Cheesy008 commented May 22, 2024

Hello. Is it possible to add functionality so i could run missed tasks if scheduler was restarted?
For example, task runs every 10 hours. After some amount of time when there are 10 minutes left before the task starts, the scheduler is being restarted due to some technical issues. After that the task is missed and scheduler has to start cycle from scratch. What i want to do is to store scheduler tasks in Redis and on restarts check if task is missed.
I went through the source code but I couldn't find a proper way how to solve it. Can you give some recommendations please?

@s3rius
Copy link
Member

s3rius commented May 22, 2024

Sure. The easiest way will be to create your own schedule source that checks in database what tasks have run and execute required tasks accordingly.

@Cheesy008
Copy link
Author

@s3rius I was thinking about this idea. However if i put tasks to database then after restart scheduler couldn't find out when to start missed task because it doesn't have created date. A task has crontab string which means that it has to run every N minutes/hours. And when this task will pass through the function get_task_delay it doesn't run it immediately

@s3rius
Copy link
Member

s3rius commented May 22, 2024

You can remember when was the last run of the task and decide whether to run the task based on this value. It might be a bit tricky with cron, but should be possible.

@Cheesy008
Copy link
Author

Cheesy008 commented May 22, 2024

@s3rius Where should i store this date value? I would like to put in ScheduledTask but I don't think that extending this model is a good idea.
I've came up with the following logic:

  1. In the schedule soure startup method I get all the tasks and put them to Redis. I add default datetime value to each task.
  2. In the get_schedules I pull all the tasks from Redis. I calculate time of each task according to creation date, crontab and current time, and decide whether this task must run immediately by changing time field, so it could pass through get_task_delay. If time doesn't expire then it will work as it is.
  3. In the post_send method I delete the task from Redis and set the new one with current timestamp.

Maybe I overcomplecated some things or came up with the wrong idea. Would be very grateful to see your comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants