Re: io-uring and tcp sockets

[Stefan Metzmacher <metze@samba.org>]

[-- Attachment #1.1: Type: text/plain, Size: 2311 bytes --]

Am 04.11.20 um 16:38 schrieb David Ahern:
> On 11/4/20 7:50 AM, Jens Axboe wrote:
>> On 11/4/20 4:21 AM, Stefan Metzmacher wrote:
>>> Hi David,
>>>
>>>> New to io_uring but can't find this answer online, so reaching out.
>>>>
>>>> I was trying out io_uring with netperf - tcp stream sockets - and
>>>> noticed a submission is called complete even with a partial send
>>>> (io_send(), ret < sr->len). Saving the offset of what succeeded (plus
>>>> some other adjustments) and retrying the sqe again solves the problem.
>>>> But the issue seems fundamental so wondering if is intentional?
>>>
>>> I guess this is just the way it is currently.
>>>
>>> For Samba I'd also like to be sure to never get short write to a socket.
>>>
>>> There I'd like to keep the pipeline full by submitting as much sqe's as possible
>>> (without waiting for completions on every single IORING_OP_SENDMSG/IORING_OP_SPLICE)
>>> using IOSQE_IO_DRAIN or IOSQE_IO_LINK and maybe IOSQE_ASYNC or IORING_SETUP_SQPOLL.
>>>
>>> But for now I just used a single sqe with IOSQE_ASYNC at a time.
>>>
>>> Jens, do you see a way to overcome that limitation?
>>>
>>> As far as I understand the situation is completely fixed now and
>>> it's no possible to get short reads and writes for file io anymore, is that correct?
>>
>> Right, the regular file IO will not return short reads or writes, unless a
>> blocking attempt returns 0 (or short). Which would be expected. The send/recvmsg
>> side just returns what the socket read/write would return, similarly to if you
>> did the normal system call variants of those calls.
>>
>> It would not be impossible to make recvmsg/sendmsg handle this internally as
>> well, we just need a good way to indicate the intent of "please satisfy the
>> whole thing before return".
>>
> 
> Attached patch handles the full send request; sendmsg can be handled
> similarly.
> 
> I take your comment to mean there should be an sq flag to opt-in to the
> behavior change? Pointers to which flag set?

sendmsg has msg_control, I think we'll need more interaction with the socket layer here
in order to wait in a single low level ->sendmsg_locked() call.

I know IORING_OP_SENDMSG doesn't support msg_control currently, but I hope to get that fixed in future.

metze



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]