zio: Avoid sleeping in the I/O path

[markjdb]

The change modifies zio_delay_interrupt(), used in testing environments, to avoid sleeping, as this is not allowed on FreeBSD.

Motivation and Context

When running the ZTS in a FreeBSD VM, I often see the following panic:

panic: sleepq_add: td 0xfffff80104a2f000 to sleep on wchan 0xffffffff813ebb21 with sleeping prohibited
cpuid = 1
time = 1732040928
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d8d5aa30
vpanic() at vpanic+0x136/frame 0xfffffe00d8d5ab60
panic() at panic+0x43/frame 0xfffffe00d8d5abc0
sleepq_add() at sleepq_add+0x338/frame 0xfffffe00d8d5ac10
_sleep() at _sleep+0x1f3/frame 0xfffffe00d8d5acb0
pause_sbt() at pause_sbt+0x7d/frame 0xfffffe00d8d5ace0
zio_delay_interrupt() at zio_delay_interrupt+0x1f9/frame 0xfffffe00d8d5ad40
g_io_deliver() at g_io_deliver+0x2b0/frame 0xfffffe00d8d5ad90
g_disk_done() at g_disk_done+0xee/frame 0xfffffe00d8d5add0
vtblk_done_completed() at vtblk_done_completed+0x38/frame 0xfffffe00d8d5ae10
vtblk_vq_intr() at vtblk_vq_intr+0xaf/frame 0xfffffe00d8d5ae60
ithread_loop() at ithread_loop+0x266/frame 0xfffffe00d8d5aef0
fork_exit() at fork_exit+0x82/frame 0xfffffe00d8d5af30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8d5af30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

In particular, we shouldn't assume that it's ok to sleep in I/O completion threads.

This change modifies zio_delay_interrupt() to avoid violating this invariant.

Description

The patch removes a special case in zio_delay_interrupt() such that the I/O is always deferred to a taskqueue.

How Has This Been Tested?

I've run through the ZTS on FreeBSD with this patch.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[amotin]

Makes sense to me. Just makes me wonder about usefulness of higher-resolution taskq_dispatch_delay() variant. FreeBSD could do it easily.

[markjdb]

Makes sense to me. Just makes me wonder about usefulness of higher-resolution taskq_dispatch_delay() variant. FreeBSD could do it easily.

Looking at the existing callers, I don't see much demand for higher-resolution timeouts. Though, ddi_get_lbolt64() seems fairly coarse on FreeBSD, I guess the maximum resolution there is ~10us.