sendmsg fails when it 'blocks'

sendmsg fails when it 'blocks'

[Jonas Bonn]

Hi Jens,

I've been trying to use io_uring to flood a network link with UDP 
packets.  Essentially, the program just pushes a series of sendmsg() 
calls through the SQE ring and keeps topping it up with new calls as 
soon as the completions come in.

When the sendmsg() calls complete immediately then everything works 
fine; however, when the calls 'block' and get queued up in the kernel 
then the calls return either errno 97 or 22 when they're retried through 
the workqueue (effectively, bad address or invalid iovec length).

My gut-feeling is that there's some issue copying the msghdr struct so 
that the call that's retried isn't exactly the same one that was 
requested.  I looked into the kernel code a bit, but couldn't really 
make heads or tails of it so I though I'd ask for some input while I 
keep investigating.

I noticed that liburing has a simple test for sendmsg that sends a 
single message; the 'punted' case doesn't seem to be tested.  Is this 
something you've tried?

Tested on kernels 5.3 to 5.6-pre and the behaviour's pretty much the 
same with regards to the above.

Any help greatly appreciated.

Best regards,
Jonas

Re: sendmsg fails when it 'blocks'

[Jens Axboe]

On 2/9/20 2:25 AM, Jonas Bonn wrote:
> Hi Jens,
> 
> I've been trying to use io_uring to flood a network link with UDP 
> packets.  Essentially, the program just pushes a series of sendmsg() 
> calls through the SQE ring and keeps topping it up with new calls as 
> soon as the completions come in.
> 
> When the sendmsg() calls complete immediately then everything works 
> fine; however, when the calls 'block' and get queued up in the kernel 
> then the calls return either errno 97 or 22 when they're retried through 
> the workqueue (effectively, bad address or invalid iovec length).
> 
> My gut-feeling is that there's some issue copying the msghdr struct so 
> that the call that's retried isn't exactly the same one that was 
> requested.  I looked into the kernel code a bit, but couldn't really 
> make heads or tails of it so I though I'd ask for some input while I 
> keep investigating.
> 
> I noticed that liburing has a simple test for sendmsg that sends a 
> single message; the 'punted' case doesn't seem to be tested.  Is this 
> something you've tried?
> 
> Tested on kernels 5.3 to 5.6-pre and the behaviour's pretty much the 
> same with regards to the above.

You could try and pull:

git://git.kernel.dk/linux-block io_uring-5.6

into the current 5.6-pre tree and see if it works for you.

In any case, do you have a test case for this?


-- 
Jens Axboe