Re: [PATCH v4 00/25] fuse/io-uring: add kernel-managed buffer rings and zero-copy

[Joanne Koong <joannelkoong@gmail.com>]

On Fri, Jan 16, 2026 at 3:31 PM Joanne Koong <joannelkoong@gmail.com> wrote:
>
> This series adds buffer ring and zero-copy capabilities to fuse over io-uring.
> This requires adding a new kernel-managed buf (kmbuf) ring type to io-uring
> where the buffers are provided and managed by the kernel instead of by
> userspace.
>
> On the io-uring side, the kmbuf interface is basically identical to pbufs.
> They differ mostly in how the memory region is set up and whether it is
> userspace or kernel that recycles back the buffer. Internally, the
> IOBL_KERNEL_MANAGED flag is used to mark the buffer ring as kernel-managed.
>
> The zero-copy work builds on top of the infrastructure added for
> kernel-managed buffer rings (the bulk of which is in patch 19: "fuse: add
> io-uring kernel-managed buffer ring") and that informs some of the design
> choices for how fuse uses the kernel-managed buffer ring without zero-copy.

Could anyone on the fuse side review the fuse changes in patches 19 and 24?

Thanks,
Joanne

>
> There was a previous submission for supporting registered buffers in fuse [1]
> but that was abandoned in favor of using kernel-managed buffer rings, which,
> once incremental buffer consumption is added in a later patchset, gives
> significant memory usage advantages in allowing the full buffer capacity to be
> utilized across multiple requests, as well as offers more flexibility for
> future additions. As well, it also makes the userspace side setup simpler.
> The relevant refactoring fuse patches from the previous submission are carried
> over into this one.
>
> Benchmarks for zero-copy (patch 24) show approximately the following
> differences in throughput for bs=1M:
>
> direct randreads: ~20% increase (~2100 MB/s -> ~2600 MB/s)
> buffered randreads: ~25% increase (~1900 MB/s -> 2400 MB/s)
> direct randwrites: no difference (~750 MB/s)
> buffered randwrites: ~10% increase (950 MB/s -> 1050 MB/s)
>
> The benchmark was run using fio on the passthrough_hp server:
> fio --name=test_run --ioengine=sync --rw=rand{read,write} --bs=1M
> --size=1G --numjobs=2 --ramp_time=30 --group_reporting=1
>
> This series is on top of commit b71e635feefc in the io-uring tree.
>
> The libfuse changes can be found in [2]. This has a dependency on the liburing
> changes in [3]. To test the server, you can run it with:
> sudo ~/libfuse/build/example/passthrough_hp ~/src ~/mounts/tmp
> --nopassthrough -o io_uring_zero_copy -o io_uring_q_depth=8
>
> Thanks,
> Joanne
>
> [1] https://lore.kernel.org/linux-fsdevel/20251027222808.2332692-1-joannelkoong@gmail.com/
> [2] https://github.com/joannekoong/libfuse/tree/zero_copy
> [3] https://github.com/joannekoong/liburing/tree/kmbuf
>
> v3: https://lore.kernel.org/linux-fsdevel/20251223003522.3055912-1-joannelkoong@gmail.com/
> v3 -> v4:
> * Get rid of likely()s and get rid of going through cmd interface layer (Gabriel)
> * Fix io_uring_cmd_fixed_index_get() to return back the node pointer (Caleb)
> * Add documentation for io_buffer_register_bvec (Caleb)
> * Remove WARN_ON_ONCE() for io_buffer_unregister() call (Caleb)
>
> v2: https://lore.kernel.org/linux-fsdevel/20251218083319.3485503-1-joannelkoong@gmail.com/
> v2 -> v3:
> * fix casting between void * and u64 for 32-bit architectures (kernel test robot)
> * add newline for documentation bullet points (kernel test robot)
> * fix unrecognized "boolean" (kernel test robot), switch it to a flag (me)
>
> v1: https://lore.kernel.org/linux-fsdevel/20251203003526.2889477-1-joannelkoong@gmail.com/
> v1 -> v2:
> * drop fuse buffer cleanup on ring death, which makes things a lot simpler (Jens)
>   - this drops a lot of things (eg needing ring_ctx tracking, needing callback
>     for ring death, etc)
> * drop fixed buffer pinning altogether and just do lookup every time (Jens)
>   (didn't significantly affect the benchmark results seen)
> * fix spelling mistake in docs (Askar)
> * use -EALREADY for pinning already pinned bufring, return PTR_ERR for
>    registration instead of err, move initializations to outside locks (Caleb)
> * drop fuse patches for zero-ed out headers (me)
>
> Joanne Koong (25):
>   io_uring/kbuf: refactor io_buf_pbuf_register() logic into generic
>     helpers
>   io_uring/kbuf: rename io_unregister_pbuf_ring() to
>     io_unregister_buf_ring()
>   io_uring/kbuf: add support for kernel-managed buffer rings
>   io_uring/kbuf: add mmap support for kernel-managed buffer rings
>   io_uring/kbuf: support kernel-managed buffer rings in buffer selection
>   io_uring/kbuf: add buffer ring pinning/unpinning
>   io_uring/kbuf: add recycling for kernel managed buffer rings
>   io_uring: add io_uring_fixed_index_get() and
>     io_uring_fixed_index_put()
>   io_uring/kbuf: add io_uring_is_kmbuf_ring()
>   io_uring/kbuf: export io_ring_buffer_select()
>   io_uring/kbuf: return buffer id in buffer selection
>   io_uring/cmd: set selected buffer index in __io_uring_cmd_done()
>   fuse: refactor io-uring logic for getting next fuse request
>   fuse: refactor io-uring header copying to ring
>   fuse: refactor io-uring header copying from ring
>   fuse: use enum types for header copying
>   fuse: refactor setting up copy state for payload copying
>   fuse: support buffer copying for kernel addresses
>   fuse: add io-uring kernel-managed buffer ring
>   io_uring/rsrc: rename
>     io_buffer_register_bvec()/io_buffer_unregister_bvec()
>   io_uring/rsrc: split io_buffer_register_request() logic
>   io_uring/rsrc: Allow buffer release callback to be optional
>   io_uring/rsrc: add io_buffer_register_bvec()
>   fuse: add zero-copy over io-uring
>   docs: fuse: add io-uring bufring and zero-copy documentation
>
>  Documentation/block/ublk.rst                  |  14 +-
>  .../filesystems/fuse/fuse-io-uring.rst        |  59 +-
>  drivers/block/ublk_drv.c                      |  18 +-
>  fs/fuse/dev.c                                 |  30 +-
>  fs/fuse/dev_uring.c                           | 692 ++++++++++++++----
>  fs/fuse/dev_uring_i.h                         |  42 +-
>  fs/fuse/fuse_dev_i.h                          |   8 +-
>  include/linux/io_uring/buf.h                  |  25 +
>  include/linux/io_uring/cmd.h                  |  97 ++-
>  include/linux/io_uring_types.h                |  10 +-
>  include/uapi/linux/fuse.h                     |  17 +-
>  include/uapi/linux/io_uring.h                 |  17 +-
>  io_uring/kbuf.c                               | 355 +++++++--
>  io_uring/kbuf.h                               |  19 +-
>  io_uring/memmap.c                             | 117 ++-
>  io_uring/memmap.h                             |   4 +
>  io_uring/register.c                           |   9 +-
>  io_uring/rsrc.c                               | 183 ++++-
>  io_uring/uring_cmd.c                          |   6 +-
>  19 files changed, 1447 insertions(+), 275 deletions(-)
>  create mode 100644 include/linux/io_uring/buf.h
>
> --
> 2.47.3
>