Re: [PATCH v3 19/25] fuse: add io-uring kernel-managed buffer ring

[Joanne Koong <joannelkoong@gmail.com>]

On Thu, Feb 5, 2026 at 1:48 PM Bernd Schubert <bschubert@ddn.com> wrote:
>
>
>
> On 2/5/26 22:29, Joanne Koong wrote:
> > On Thu, Feb 5, 2026 at 12:49 PM Bernd Schubert <bschubert@ddn.com> wrote:
> >>
> >>
> >>
> >> On 2/5/26 21:24, Joanne Koong wrote:
> >>> On Tue, Feb 3, 2026 at 3:58 PM Bernd Schubert <bernd@bsbernd.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 12/23/25 01:35, Joanne Koong wrote:
> >>>>> Add io-uring kernel-managed buffer ring capability for fuse daemons
> >>>>> communicating through the io-uring interface.
> >>>>>
> >>>>> This has two benefits:
> >>>>> a) eliminates the overhead of pinning/unpinning user pages and
> >>>>> translating virtual addresses for every server-kernel interaction
> >>>>>
> >>>>> b) reduces the amount of memory needed for the buffers per queue and
> >>>>> allows buffers to be reused across entries. Incremental buffer
> >>>>> consumption, when added, will allow a buffer to be used across multiple
> >>>>> requests.
> >>>>>
> >>>>> Buffer ring usage is set on a per-queue basis. In order to use this, the
> >>>>> daemon needs to have preregistered a kernel-managed buffer ring and a
> >>>>> fixed buffer at index 0 that will hold all the headers, and set the
> >>>>> "use_bufring" field during registration. The kernel-managed buffer ring
> >>>>> will be pinned for the lifetime of the connection.
> >>>>>
> >>>>> Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
> >>>>> ---
> >>>>>  fs/fuse/dev_uring.c       | 423 ++++++++++++++++++++++++++++++++------
> >>>>>  fs/fuse/dev_uring_i.h     |  30 ++-
> >>>>>  include/uapi/linux/fuse.h |  15 +-
> >>>>>  3 files changed, 399 insertions(+), 69 deletions(-)
> >>>>>
> >>>>> diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c
> >>>>> @@ -824,21 +1040,29 @@ static void fuse_uring_add_req_to_ring_ent(struct fuse_ring_ent *ent,
> >>>>>  }
> >>>>>
> >>>>>  /* Fetch the next fuse request if available */
> >>>>> -static struct fuse_req *fuse_uring_ent_assign_req(struct fuse_ring_ent *ent)
> >>>>> +static struct fuse_req *fuse_uring_ent_assign_req(struct fuse_ring_ent *ent,
> >>>>> +                                               unsigned int issue_flags)
> >>>>>       __must_hold(&queue->lock)
> >>>>>  {
> >>>>>       struct fuse_req *req;
> >>>>>       struct fuse_ring_queue *queue = ent->queue;
> >>>>>       struct list_head *req_queue = &queue->fuse_req_queue;
> >>>>> +     int err;
> >>>>>
> >>>>>       lockdep_assert_held(&queue->lock);
> >>>>>
> >>>>>       /* get and assign the next entry while it is still holding the lock */
> >>>>>       req = list_first_entry_or_null(req_queue, struct fuse_req, list);
> >>>>> -     if (req)
> >>>>> -             fuse_uring_add_req_to_ring_ent(ent, req);
> >>>>> +     if (req) {
> >>>>> +             err = fuse_uring_next_req_update_buffer(ent, req, issue_flags);
> >>>>> +             if (!err) {
> >>>>> +                     fuse_uring_add_req_to_ring_ent(ent, req);
> >>>>> +                     return req;
> >>>>> +             }
> >>>>
> >>>> Hmm, who/what is going to handle the request if this fails? Let's say we
> >>>> have just one ring entry per queue and now it fails here - this ring
> >>>> entry will go into FRRS_AVAILABLE and nothing will pull from the queue
> >>>> anymore. I guess it _should_ not happen, some protection would be good.
> >>>> In order to handle it, at least one other ent needs to be in flight.
> >>>
> >>> If the queue only has one ring ent and this fails, the request gets
> >>> reassigned to the ent whenever ->send_req() is next triggered. I don't
> >>> think this is a new edge case introduced by kmbufs; in the existing
> >>> code, fuse_uring_commit_fetch() -> fuse_uring_get_next_fuse_req() ->
> >>> fuse_uring_send_next_to_ring() -> fuse_uring_prepare_send() could fail
> >>> if any of the copying fails, in which case we end up in the same
> >>> position of the ent getting assigned the next request whenever
> >>> ->send_req() is next triggered.
> >>
> >> I don't manage to check right now (need to solve another imbalance with
> >> reduced rings right now), but every failed copy is *supposed* to end up
> >
> > Thanks for your work on the reduced rings, I'm looking forward to
> > seeing your patchset.
> >
> >> in a request failure. Why should it block, if the copy failed?
> >> It would be a bug if it does not right now and should be solved.
> >>
> >> Regarding your copy, I don't think waiting for for the next ->send_req()
> >> is an option, it might be never.
> >> One solution might be a single entry in any of the queues or in a
> >> separate queue that doesn't have buf-rings - i.e. it can go slowly, but
> >> it must not block. Some wake-up task retry might also work, but would be
> >> timeout based.
> >
> > Ah, so your concern is about the request taking too long to complete,
> > not about the ent not being available again to send requests. In the
> > existing code, yes if the next request can't be sent after a commit
> > then that next request is immediately terminated. For the kmbuf case,
> > the fuse_uring_next_req_update_buffer() call only fails if all the
> > buffers are currently being used. The request will be picked up when
> > the next buffer becomes available / is recycled back, which happens
> > when the request(s) sent out to the server completes and is committed,
> > if a ->send_req() hasn't already been triggered by then.
>
>
> In the simple one ring entry example there wouldn't be another in-fly
> request - the request would basically hang forever, if some reasons the

If there's no in-flight request, then the buffer will always be
available since there's no other request using it. For the one
ring-entry example with 1 registered buffer in the bufring,

request A -> reserves buffer 1 -> request A gets sent to the server

request B gets enqueued -> no ent is available -> request is added to
the list for later servicing

server sends a reply for request A -> kernel processes the reply in
fuse_uring_commit_fetch() -> buffer 1 is recycled -> server gets the
next request (request B) -> server uses buffer 1 to service request B


> ring buffer is not available. It *shouldn't* happen, but what if for
> example the same buffer would be used for zery-copy another subsystem
> now consumes them? No idea if that could happen - with these buffers

The buffer is registered by libfuse for the fuse connection and can
*only* be used by that fuse connection. It cannot be used by another
subsystem.

> there an additional complexity, which I don't understand from fuse point
> of view. Let's just assume there would be some kind of ring buffer bug
> and requests would now hang - how would we debug this if it just hangs
> without any log or failure message?

The ring buffer can only run out of entries if there are in-flight
requests that are currently using all the buffers. Those requests will
complete and recycle back the buffer they used and fetch the next
request, which will use the buffer that was just recycled back.

Thanks,
Joanne
>
>
> Thanks,
> Bernd