Re: [PATCH v3 19/25] fuse: add io-uring kernel-managed buffer ring

[Bernd Schubert <bschubert@ddn.com>]

On 2/5/26 22:29, Joanne Koong wrote:
> On Thu, Feb 5, 2026 at 12:49 PM Bernd Schubert <bschubert@ddn.com> wrote:
>>
>>
>>
>> On 2/5/26 21:24, Joanne Koong wrote:
>>> On Tue, Feb 3, 2026 at 3:58 PM Bernd Schubert <bernd@bsbernd.com> wrote:
>>>>
>>>>
>>>>
>>>> On 12/23/25 01:35, Joanne Koong wrote:
>>>>> Add io-uring kernel-managed buffer ring capability for fuse daemons
>>>>> communicating through the io-uring interface.
>>>>>
>>>>> This has two benefits:
>>>>> a) eliminates the overhead of pinning/unpinning user pages and
>>>>> translating virtual addresses for every server-kernel interaction
>>>>>
>>>>> b) reduces the amount of memory needed for the buffers per queue and
>>>>> allows buffers to be reused across entries. Incremental buffer
>>>>> consumption, when added, will allow a buffer to be used across multiple
>>>>> requests.
>>>>>
>>>>> Buffer ring usage is set on a per-queue basis. In order to use this, the
>>>>> daemon needs to have preregistered a kernel-managed buffer ring and a
>>>>> fixed buffer at index 0 that will hold all the headers, and set the
>>>>> "use_bufring" field during registration. The kernel-managed buffer ring
>>>>> will be pinned for the lifetime of the connection.
>>>>>
>>>>> Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
>>>>> ---
>>>>>  fs/fuse/dev_uring.c       | 423 ++++++++++++++++++++++++++++++++------
>>>>>  fs/fuse/dev_uring_i.h     |  30 ++-
>>>>>  include/uapi/linux/fuse.h |  15 +-
>>>>>  3 files changed, 399 insertions(+), 69 deletions(-)
>>>>>
>>>>> diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c
>>>>> @@ -824,21 +1040,29 @@ static void fuse_uring_add_req_to_ring_ent(struct fuse_ring_ent *ent,
>>>>>  }
>>>>>
>>>>>  /* Fetch the next fuse request if available */
>>>>> -static struct fuse_req *fuse_uring_ent_assign_req(struct fuse_ring_ent *ent)
>>>>> +static struct fuse_req *fuse_uring_ent_assign_req(struct fuse_ring_ent *ent,
>>>>> +                                               unsigned int issue_flags)
>>>>>       __must_hold(&queue->lock)
>>>>>  {
>>>>>       struct fuse_req *req;
>>>>>       struct fuse_ring_queue *queue = ent->queue;
>>>>>       struct list_head *req_queue = &queue->fuse_req_queue;
>>>>> +     int err;
>>>>>
>>>>>       lockdep_assert_held(&queue->lock);
>>>>>
>>>>>       /* get and assign the next entry while it is still holding the lock */
>>>>>       req = list_first_entry_or_null(req_queue, struct fuse_req, list);
>>>>> -     if (req)
>>>>> -             fuse_uring_add_req_to_ring_ent(ent, req);
>>>>> +     if (req) {
>>>>> +             err = fuse_uring_next_req_update_buffer(ent, req, issue_flags);
>>>>> +             if (!err) {
>>>>> +                     fuse_uring_add_req_to_ring_ent(ent, req);
>>>>> +                     return req;
>>>>> +             }
>>>>
>>>> Hmm, who/what is going to handle the request if this fails? Let's say we
>>>> have just one ring entry per queue and now it fails here - this ring
>>>> entry will go into FRRS_AVAILABLE and nothing will pull from the queue
>>>> anymore. I guess it _should_ not happen, some protection would be good.
>>>> In order to handle it, at least one other ent needs to be in flight.
>>>
>>> If the queue only has one ring ent and this fails, the request gets
>>> reassigned to the ent whenever ->send_req() is next triggered. I don't
>>> think this is a new edge case introduced by kmbufs; in the existing
>>> code, fuse_uring_commit_fetch() -> fuse_uring_get_next_fuse_req() ->
>>> fuse_uring_send_next_to_ring() -> fuse_uring_prepare_send() could fail
>>> if any of the copying fails, in which case we end up in the same
>>> position of the ent getting assigned the next request whenever
>>> ->send_req() is next triggered.
>>
>> I don't manage to check right now (need to solve another imbalance with
>> reduced rings right now), but every failed copy is *supposed* to end up
> 
> Thanks for your work on the reduced rings, I'm looking forward to
> seeing your patchset.
> 
>> in a request failure. Why should it block, if the copy failed?
>> It would be a bug if it does not right now and should be solved.
>>
>> Regarding your copy, I don't think waiting for for the next ->send_req()
>> is an option, it might be never.
>> One solution might be a single entry in any of the queues or in a
>> separate queue that doesn't have buf-rings - i.e. it can go slowly, but
>> it must not block. Some wake-up task retry might also work, but would be
>> timeout based.
> 
> Ah, so your concern is about the request taking too long to complete,
> not about the ent not being available again to send requests. In the
> existing code, yes if the next request can't be sent after a commit
> then that next request is immediately terminated. For the kmbuf case,
> the fuse_uring_next_req_update_buffer() call only fails if all the
> buffers are currently being used. The request will be picked up when
> the next buffer becomes available / is recycled back, which happens
> when the request(s) sent out to the server completes and is committed,
> if a ->send_req() hasn't already been triggered by then.


In the simple one ring entry example there wouldn't be another in-fly
request - the request would basically hang forever, if some reasons the
ring buffer is not available. It *shouldn't* happen, but what if for
example the same buffer would be used for zery-copy another subsystem
now consumes them? No idea if that could happen - with these buffers
there an additional complexity, which I don't understand from fuse point
of view. Let's just assume there would be some kind of ring buffer bug
and requests would now hang - how would we debug this if it just hangs
without any log or failure message?


Thanks,
Bernd