Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

[Joanne Koong <joannelkoong@gmail.com>]

On Fri, Feb 27, 2026 at 12:48 PM Pavel Begunkov <asml.silence@gmail.com> wrote:
>
> On 2/27/26 01:12, Joanne Koong wrote:
> ...
> >>> Regions shouldn't know anything about your buffers, how it's
> >>> subdivided after, etc.
> >
> > I still think the memory for the buffers should be tied to the ring
> > itself and allocated physically contiguously per buffer. Per-buffer
> > contiguity will enable the most efficient DMA path for servers to send
> > read/write data to local storage or the network. If the buffers for
> > the bufring have to be allocated as one single memory region, the
> > io_mem_alloc_compound() call will fail for this large allocation size.
> > Even if io_mem_alloc_compound() did succeed, this is a waste as the
> > buffer pool as an entity doesn't need to be physically contiguous,
> > just the individual buffers themselves. For fuse, the server
> > configures what buffer pool size it wants to use, depending on what
> > queue depth and max request size it needs. So for most use cases, at
> > least for high-performance servers, allocation will have to fall back
> > to alloc_pages_bulk_node(), which doesn't allocate contiguously. You
> > mentioned in an earlier comment that this "only violates abstractions"
> > - which abstractions does this break? The pre-existing behavior
> > already defaults to allocating pages non-contiguously if the mem
> > region can't be allocated fully contiguously.
>
> Regions has uapi (see struct io_uring_region_desc) so that users
> can operate with them in a unified manner. If you want regions to
> be allocated in some special way, just extend it.

You can't say "regions shouldn't know anything about your buffers, how
it's subdivided, etc" and then also say "extend the region uapi for
special allocation to make it buffer-compatible". If we extend the
region uapi to specify contiguous chunks of size X starting at offset
Y for len Z, that is basically encoding buffer layout information into
the region. The buffer ring already knows buffer sizes and count - it
is the natural place to express contiguity requirements.

Pushing this into the region abstraction muddies the uapi and forces
awkward indirection where callers now need to manually synchronize
region chunk specifications with their buffer layout. Memory regions
are generic and will be used for purposes beyond kmbufs. forcing
buffer-specific allocation semantics into the region UAPI pollutues a
general abstraction with domain-specific details.

>
> > Going through registered buffers doesn't help either. Fuse servers can
> > be unprivileged and it's not guaranteed that there are enough huge
> > pages reserved or that another process hasn't taken them or that the
> > server has privileges to pre-reserve pages for the allocation. Also
>
> There is THP these days. And FWIW, we should be vigilant about not

THP is opportunistic and not guaranteed. It depends on external
factors like fragmentation, memory pressure, system settings, etc. For
high-performance FUSE servers where deterministic DMA efficiency is
required, this doesn't suffice.

> using io_uring to work around capabilities and mm policies. If user

This isn't working around capabilities / mm policies. The user isn't
getting contiguous physical memory to use freely, the kernel is
allocating it internally to service I/O efficiently. Providing
infrastructure for efficient DMA isn't a capability / mm bypass, this
is standard kernel behavior. When userspace does i/o through sockets
or block devices, the kernel routinely allocates contiguous memory
with dma_alloc_coherent() or alloc_pages() with order > 0. That's
exactly the point I'm trying to make - users shouldn't have to do this
themselves (eg going through registered buffers with user-allocated
buffers). The kernel should handle it internally.

> can't do it, io_uring shouldn't either. It's also all accounted
> against mlock, if the limit is not high enough, you won't be able
> to use this feature at all.

The mlock point is orthogonal. it restricts how much memory a user can
pin, but contiguous and noncontiguous allocations of the same size
consume the same mlock budget.

Thanks,
Joanne

>
> > the 2 MB granularity is inflexible while 1 GB is too much.
>
> --
> Pavel Begunkov
>