Re: io-wrk threads on socket vs non-socket

[Jens Axboe <axboe@kernel.dk>]

On 6/14/23 8:09?AM, Marek Majkowski wrote:
> Hi!
> 
> I'm playing with io-uring, and I found the io-wrk thread situation confusing.
> 
> (A) In one case, I have a SOCK_DGRAM socket (blocking), over which I
> do IORING_OP_RECVMSG. This works well, and unless I mark the sqe as
> IOSQE_ASYNC, it doesn't seem to start an io-wrk kernel thread.
> 
> (B) However, the same can't be said of another situation. In the
> second case I have a tap file descriptor (blocking), which doesn't
> support "Socket operations on non-socket", so I must do
> IORING_OP_READV. This however seems to start a new io-wrk for each
> readv request:
> 
> $ pstree -pt `pidof tapuring`
> tapuring(44932)???{iou-wrk-44932}(44937)
>                 ??{iou-wrk-44932}(44938)
>                 ??{iou-wrk-44932}(44939)
>                 ??{iou-wrk-44932}(44940)
>                 ??{iou-wrk-44932}(44941)
>                 ??{iou-wrk-44932}(44942)
> 
> I would expect both situations to behave the same way.
> 
> The manpage for IOSQE_ASYNC:
> 
>        IOSQE_ASYNC
>               Normal operation for io_uring is to try and issue an sqe
>               as non-blocking first, and if that fails, execute it in an
>               async manner. To support more efficient overlapped
>               operation of requests that the application knows/assumes
>               will always (or most of the time) block, the application
>               can ask for an sqe to be issued async from the start. Note
>               that this flag immediately causes the SQE to be offloaded
>               to an async helper thread with no initial non-blocking
>               attempt.  This may be less efficient and should not be
>               used liberally or without understanding the performance
>               and efficiency tradeoffs.
> 
> This seems to cover the tap file descriptor case. It tries to readv
> and when that fails a new io-wrk is spawned. Fine. However, as I
> described it seems this is not true for sockets, as without
> IOSQE_ASYNC the io-wrk thread is _not_ spawned there?
> 
> Is the behaviour different due to socket vs non-socket or readv vs
> recvmsg?

What kernel are you using? tap just recently got FMODE_NOWAIT support,
which should trigger poll instead of needing to spawn an io worker.

Also, as usual, a test case would be appreciated. Particularly if this
is on a current kernel where we would not expect to see io-wq activity
for a read of tap.

-- 
Jens Axboe