Re: [PATCHSET v2 0/2] io_uring: handle short reads internally

[Jens Axboe <axboe@kernel.dk>]

On 8/17/20 9:12 PM, Jens Axboe wrote:
> On 8/17/20 8:29 PM, Jens Axboe wrote:
>> On 8/17/20 2:25 AM, Stefan Metzmacher wrote:
>>> Hi Jens,
>>>
>>>> Since we've had a few cases of applications not dealing with this
>>>> appopriately, I believe the safest course of action is to ensure that
>>>> we don't return short reads when we really don't have to.
>>>>
>>>> The first patch is just a prep patch that retains iov_iter state over
>>>> retries, while the second one actually enables just doing retries if
>>>> we get a short read back.
>>>>
>>>> This passes all my testing, both liburing regression tests but also
>>>> tests that explicitly trigger internal short reads and hence retry
>>>> based on current state. No short reads are passed back to the
>>>> application.
>>>
>>> Thanks! I was going to ask about exactly that :-)
>>>
>>> It wasn't clear why returning short reads were justified by resulting
>>> in better performance... As it means the application needs to do
>>> a lot more work and syscalls.
>>
>> It mostly boils down to figuring out a good way to do it. With the
>> task_work based retry, the async buffered reads, we're almost there and
>> the prep patch adds the last remaining bits to retain the iov_iter state
>> across issues.
>>
>>> Will this be backported?
>>
>> I can, but not really in an efficient manner. It depends on the async
>> buffered work to make progress, and the task_work handling retry. The
>> latter means it's 5.7+, while the former is only in 5.9+...
>>
>> We can make it work for earlier kernels by just using the thread offload
>> for that, and that may be worth doing. That would enable it in
>> 5.7-stable and 5.8-stable. For that, you just need these two patches.
>> Patch 1 would work as-is, while patch 2 would need a small bit of
>> massaging since io_read() doesn't have the retry parts.
>>
>> I'll give it a whirl just out of curiosity, then we can debate it after
>> that.
> 
> Here are the two patches against latest 5.7-stable (the rc branch, as
> we had quite a few queued up after 5.9-rc1). Totally untested, just
> wanted to see if it was doable.
> 
> First patch is mostly just applied, with various bits removed that we
> don't have in 5.7. The second patch just does -EAGAIN punt for the
> short read case, which will queue the remainder with io-wq for
> async execution.
> 
> Obviously needs quite a bit of testing before it can go anywhere else,
> but wanted to throw this out there in case you were interested in
> giving it a go...

Actually passes basic testing, and doesn't return short reads. So at
least it's not half bad, and it should be safe for you to test.

I quickly looked at 5.8 as well, and the good news is that the same
patches will apply there without changes.

-- 
Jens Axboe