Re: [PATCH] io_uring/rsrc: send exact nr_segs for fixed buffer

[Pavel Begunkov <asml.silence@gmail.com>]

On 4/16/25 21:30, Jens Axboe wrote:
> On 4/16/25 2:29 PM, Pavel Begunkov wrote:
>> On 4/16/25 21:01, Jens Axboe wrote:
>>> On 4/16/25 1:57 PM, Nitesh Shetty wrote:
>> ...
>>>>>                   /*
>>>>> @@ -1073,7 +1075,6 @@ static int io_import_fixed(int ddir, struct iov_iter *iter,
>>>>>                    * since we can just skip the first segment, which may not
>>>>>                    * be folio_size aligned.
>>>>>                    */
>>>>> -               const struct bio_vec *bvec = imu->bvec;
>>>>>
>>>>>                   /*
>>>>>                    * Kernel buffer bvecs, on the other hand, don't necessarily
>>>>> @@ -1099,6 +1100,27 @@ static int io_import_fixed(int ddir, struct iov_iter *iter,
>>>>>                   }
>>>>>           }
>>>>>
>>>>> +       /*
>>>>> +        * Offset trimmed front segments too, if any, now trim the tail.
>>>>> +        * For is_kbuf we'll iterate them as they may be different sizes,
>>>>> +        * otherwise we can just do straight up math.
>>>>> +        */
>>>>> +       if (len + offset < imu->len) {
>>>>> +               bvec = iter->bvec;
>>>>> +               if (imu->is_kbuf) {
>>>>> +                       while (len > bvec->bv_len) {
>>>>> +                               len -= bvec->bv_len;
>>>>> +                               bvec++;
>>>>> +                       }
>>>>> +                       iter->nr_segs = bvec - iter->bvec;
>>>>> +               } else {
>>>>> +                       size_t vec_len;
>>>>> +
>>>>> +                       vec_len = bvec->bv_offset + iter->iov_offset +
>>>>> +                                       iter->count + ((1UL << folio_shift) - 1);
>>>>> +                       iter->nr_segs = vec_len >> folio_shift;
>>>>> +               }
>>>>> +       }
>>>>>           return 0;
>>>>>    }
>>>> This might not be needed for is_kbuf , as they already update nr_seg
>>>> inside iov_iter_advance.
>>>
>>> How so? If 'offset' is true, then yes it'd skip the front, but it
>>> doesn't skip the end part. And if 'offset' is 0, then no advancing is
>>> done in the first place - which does make sense, as it's just advancing
>>> from the front.
>>>
>>>> How about changing something like this ?
>>>
>>> You can't hide this in the if (offset) section...
>>
>> Should we just make it saner first? Sth like these 3 completely
>> untested commits
>>
>> https://github.com/isilence/linux/commits/rsrc-import-cleanup/
>>
>> And then it'll become
>>
>> nr_segs = ALIGN(offset + len, 1UL << folio_shift);
> 
> Let's please do that, certainly an improvement. Care to send this out? I
> can toss them at the testing. And we'd still need that last patch to

I need to test it first, perhaps tomorrow

> ensure the segment count is correct. Honestly somewhat surprised that

Right, I can pick up the Nitesh's patch to that.

> the only odd fallout of that is (needlessly) hitting the bio split path.

It's perfectly correct from the iter standpoint, AFAIK, length
and nr of segments don't have to match. Though I am surprised
it causes perf issues in the split path.

Btw, where exactly does it stumble in there? I'd assume we don't
need to do the segment correction for kbuf as the bio splitting
can do it (and probably does) in exactly the same way?

-- 
Pavel Begunkov