Adding read_exact and write_all OPs?

Adding read_exact and write_all OPs?

[Artyom Pavlov]

Greetings!

In application code it's quite common to write/read a whole buffer and 
only then continue task execution. The traditional approach is to wrap 
read/write sycall/OP in loop, which is often done as part of a language 
std. In synchronous context it makes sense because it allows to process 
things like EINTR. But in asynchronous (event-driven) context I think it 
makes a bit less sense.

What do you think about potential addition of OPs like read_exact and 
write_all, i.e. OPs which on successful CQE guarantee that an input 
buffer was processed completely? They would allow to simplify user code 
and in some cases to significantly reduce ring traffic.

Best regards,
Artyom Pavlov.

Re: Adding read_exact and write_all OPs?

[Jens Axboe]

On 8/2/22 3:04 PM, Artyom Pavlov wrote:
> Greetings!
> 
> In application code it's quite common to write/read a whole buffer and
> only then continue task execution. The traditional approach is to wrap
> read/write sycall/OP in loop, which is often done as part of a
> language std. In synchronous context it makes sense because it allows
> to process things like EINTR. But in asynchronous (event-driven)
> context I think it makes a bit less sense.
> 
> What do you think about potential addition of OPs like read_exact and
> write_all, i.e. OPs which on successful CQE guarantee that an input
> buffer was processed completely? They would allow to simplify user
> code and in some cases to significantly reduce ring traffic.

That may make sense, and there's some precedence there for sockets with
the WAITALL flags.

-- 
Jens Axboe

Re: Adding read_exact and write_all OPs?

[Artyom Pavlov]

Is there a tracking list of potential features for io-uring?

Unfortunately, I am not a C programmer (I mostly work with Rust), so I 
probably would not be able to implement it myself.

But either way, could you point to relevant parts of code? In theory the 
changes look relatively simple, but I am not sure how hard it will be to 
implement the new OP codes in practice.

Best regards,
Artyom Pavlov.

Re: Adding read_exact and write_all OPs?

[Stefan Metzmacher]

Hi Jens,

>> In application code it's quite common to write/read a whole buffer and
>> only then continue task execution. The traditional approach is to wrap
>> read/write sycall/OP in loop, which is often done as part of a
>> language std. In synchronous context it makes sense because it allows
>> to process things like EINTR. But in asynchronous (event-driven)
>> context I think it makes a bit less sense.
>>
>> What do you think about potential addition of OPs like read_exact and
>> write_all, i.e. OPs which on successful CQE guarantee that an input
>> buffer was processed completely? They would allow to simplify user
>> code and in some cases to significantly reduce ring traffic.
> 
> That may make sense, and there's some precedence there for sockets with
> the WAITALL flags.

I remember we got short reads/writes from some early kernel versions
and had to add a retry in samba.

But don't we already have a retry in current kernels?
At least for the need_complete_io() case?
io_rw_should_reissue() also seem to be related and also handles S_ISREG,
but it's hidden behind CONFIG_BLOCK.

Are there really systems without CONFIG_BLOCK?

metze

Re: Adding read_exact and write_all OPs?

[Pavel Begunkov]

On 8/16/22 19:46, Stefan Metzmacher wrote:
> Hi Jens,
> 
>>> In application code it's quite common to write/read a whole buffer and
>>> only then continue task execution. The traditional approach is to wrap
>>> read/write sycall/OP in loop, which is often done as part of a
>>> language std. In synchronous context it makes sense because it allows
>>> to process things like EINTR. But in asynchronous (event-driven)
>>> context I think it makes a bit less sense.
>>>
>>> What do you think about potential addition of OPs like read_exact and
>>> write_all, i.e. OPs which on successful CQE guarantee that an input
>>> buffer was processed completely? They would allow to simplify user
>>> code and in some cases to significantly reduce ring traffic.
>>
>> That may make sense, and there's some precedence there for sockets with
>> the WAITALL flags.
> 
> I remember we got short reads/writes from some early kernel versions
> and had to add a retry in samba.
> 
> But don't we already have a retry in current kernels?

Right, both short reads and writes will be retried when possible. Not
all file types are covered but fs and block should be fine. We also try
to handle short send/recv.

> At least for the need_complete_io() case?
> io_rw_should_reissue() also seem to be related and also handles S_ISREG,
> but it's hidden behind CONFIG_BLOCK.
> 
> Are there really systems without CONFIG_BLOCK?

-- 
Pavel Begunkov