"RuntimeError: Event loop is closed" running doc homage demo

[ksamuel]

When running the "Simple Usage" demo code from the doc (https://arq-docs.helpmanual.io/#simple-usage) verbatim on Ubuntu 24.04 with Python 3.12.3, the tasks are correctly enqueued and executed. However, the Redis connection starts shutting down only after the event loop is closed:

$ python main.py 
Exception ignored in: <function AbstractConnection.__del__ at 0x7fcf6575b380>
Traceback (most recent call last):
  File "/tmp/project_LzzRtU/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 217, in __del__
    self._close()
  File "/tmp/project_LzzRtU/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 224, in _close
    self._writer.close()
  File "/usr/lib/python3.12/asyncio/streams.py", line 358, in close
    return self._transport.close()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/selector_events.py", line 1210, in close
    super().close()
  File "/usr/lib/python3.12/asyncio/selector_events.py", line 875, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "/usr/lib/python3.12/asyncio/base_events.py", line 795, in call_soon
    self._check_closed()
  File "/usr/lib/python3.12/asyncio/base_events.py", line 541, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

While this doesn't technically prevents you from using arq, it will:

• Make people spend time figuring out why a hello word demo crashed on their machine, believing there is a bug. I tried it with several versions of Python (from 3.7), moved httpx session around and ran a debugger until I realized what was going on.
• Return an error code in the terminal, making error handling believe task enqueuing has failed.
• Raise an unexpected exception which means embedding the code forces the user to wrap all calls to it in a try/except for lifetime handling.

Changing main() to:

async def main():
    redis = await create_pool(REDIS_SETTINGS)
    for url in ('https://facebook.com', 'https://microsoft.com', 'https://github.com'):
        await redis.enqueue_job('download_content', url)
    await redis.aclose() # <-- close redis connection after the job is done

Solves the problem. Note that adding the redis pool lifecycle in startup() and shutdown() doesn't. I haven't investigated why.

This could be Linux specific since the event loop lifetime events may be slightly different than on Windows and Mac, so if you can't reproduce it (which seems likely otherwise you would probably have caught it sooner), it may be because the timing aligns nicely on those OS. I don't have access to those to test that right now.

However, this is not distro specific, I just tried on the latest Arch Linux with the same result as with Ubuntu.