Re: IOSQE_IO_LINK vs. short send of SOCK_STREAM

[Jens Axboe <axboe@kernel.dk>]

On 1/11/23 8:27?PM, Ming Lei wrote:
> Hi Stefan and Jens,
> 
> Thanks for the help.
> 
> BTW, the issue is observed when I write ublk-nbd:
> 
> https://github.com/ming1/ubdsrv/commits/nbd
> 
> and it isn't completed yet(multiple send sqe chains not serialized
> yet), the issue is triggered when writing big chunk data to ublk-nbd.

Gotcha

> On Wed, Jan 11, 2023 at 05:32:00PM +0100, Stefan Metzmacher wrote:
>> Hi Ming,
>>
>>> Per my understanding, a short send on SOCK_STREAM should terminate the
>>> remainder of the SQE chain built by IOSQE_IO_LINK.
>>>
>>> But from my observation, this point isn't true when using io_sendmsg or
>>> io_sendmsg_zc on TCP socket, and the other remainder of the chain still
>>> can be completed after one short send is found. MSG_WAITALL is off.
>>
>> This is due to legacy reasons, you need pass MSG_WAITALL explicitly
>> in order to a retry or an error on a short write...
>> It should work for send, sendmsg, sendmsg_zc, recv and recvmsg.
> 
> Turns out there is another application bug in which recv sqe may cut in the
> send sqe chain.
> 
> After the issue is fixed, if MSG_WAITALL is set, short send can't be
> observed any more. But if MSG_WAITALL isn't set, short send can be
> observed and the send io chain still won't be terminated.

Right, if MSG_WAITALL is set, then the whole thing will be written. If
we get a short send, it's retried appropriately. Unless an error occurs,
it should send the whole thing.

> So if MSG_WAITALL is set, will io_uring be responsible for retry in case
> of short send, and application needn't to take care of it?

Correct. I did add a note about that in the liburing man pages after
your email earlier:

https://git.kernel.dk/cgit/liburing/commit/?id=8d056db7c0e58f45f7c474a6627f83270bb8f00e

since that wasn't documented as far as I can tell.

-- 
Jens Axboe