Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unhandled KeyboardInterruptError from interrupt_handler #372

Open
rtnpro opened this issue Oct 22, 2024 · 1 comment
Open

Unhandled KeyboardInterruptError from interrupt_handler #372

rtnpro opened this issue Oct 22, 2024 · 1 comment

Comments

@rtnpro
Copy link

rtnpro commented Oct 22, 2024

Description

When running a taskiq worker that spawns subprocesses, KeyboardInterrupt exceptions are not being caught and handled as expected. This results in an unhandled exception traceback being printed when attempting to gracefully shut down the worker.

Steps to Reproduce

  1. Set up a taskiq worker that spawns subprocesses in its tasks
  2. Run the worker
  3. Send a SIGINT or SIGTERM to the worker process (e.g., by pressing Ctrl+C)

Expected Behavior

The worker should catch the KeyboardInterrupt, log a warning message, and proceed to shut down the broker gracefully.

Actual Behavior

An unhandled KeyboardInterrupt exception is raised, and a traceback is printed:

  File "taskiq/receiver/receiver.py", line 271, in run_task
    returned = await target_future
  File "myapp/tasks.py", line 94, in process_call
    spawn_bot(
  File "myapp/bot_spawner.py", line 115, in spawn_bot
    subprocess.check_call(
  File "subprocess.py", line 408, in check_call
    retcode = call(*popenargs, **kwargs)
  File "subprocess.py", line 391, in call
    return p.wait(timeout=timeout)
  File "subprocess.py", line 1264, in wait
    return self._wait(timeout=timeout)
  File "subprocess.py", line 2053, in _wait
    (pid, sts) = self._try_wait(0)
  File "subprocess.py", line 2011, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
  File "taskiq/cli/worker/run.py", line 107, in interrupt_handler
    raise KeyboardInterrupt

Environment

  • taskiq dependencies' versions:
    • taskiq: 0.11.3
    • taskiq-dependencies: 1.5.3
    • taskiq-redis: 0.5.6
  • Python version: 3.12
  • OS: debian bookworm-slim
  • Command: taskiq worker myapp.worker:broker myapp.tasks --workers=4 --max-threadpool-threads=1 --max-async-tasks=1 --ack-type=when_executed --wait-tasks-timeout=600 --shutdown-timeout=630 --max-tasks-per-child=1000
  • Broker used: taskiq_redis.ListQueueBroker

Additional Context

The issue seems to occur because the KeyboardInterrupt is being raised while waiting for a subprocess to complete. The exception is propagating through the async call stack before it can be caught by the main try-except block in the start_listen function.

@pySilver
Copy link

pySilver commented Nov 8, 2024

@rtnpro this is probably related ? taskiq-python/taskiq-redis#72

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants